Learn about the Australian Research Data Commons (ARDC), which operates the Nectar Research Cloud, and their OpenStack use case

image

Eleven years into the development and adoption of OpenStack, organizations with clouds of all sizes are continuing to grow their footprint and diversify the workloads they are running. In this series of case studies, we talked to different users around the world, learned about the scale and growth of their OpenStack cloud, and what kind of future growth they are expecting.

Today, Paul Coddington, associate director of research cloud and storage, and Carmel Walsh, director, eResearch infrastructure and services at Australian Research Data Commons (ARDC) presented their OpenInfra use case and explained why their OpenStack deployment is growing so quickly.

What is the name of your organization?

Australian Research Data Commons (ARDC), which operates the Nectar Research Cloud, Australia’s national research cloud.

What year did your organization launch its first OpenStack deployment?

2012

How has OpenStack transformed your organization?

The ARDC Nectar Research Cloud, supported by the Australian Government National Collaborative Research Infrastructure Strategy (NCRIS), provides Australia’s research community with fast, interactive, self-service access to large-scale computing infrastructure, software and data, and is a powerful platform for collaboration.

Its versatile cloud infrastructure can be used in many different ways to support research, such as a virtual desktop for a single researcher, or a powerful computational server that can be shared by researchers in Australia and internationally.

The Nectar project began in 2010 with a mandate to provide open cloud infrastructure for research, and platforms to enhance cross-institutional research collaboration. In 2011 OpenStack was chosen as the cloud software from several possible solutions, and that has proven to be an excellent decision, since it has become the standard solution for open cloud infrastructure.

OpenStack has enabled us to provide a very cost-effective, large-scale research cloud infrastructure, federated across multiple organizations and locations in Australia and New Zealand. Since it is open infrastructure we have been able to customise our deployments to meet the requirements of our user community, and leverage off and learn from (and contribute to) the international OpenStack community, and other research organizations that also use OpenStack.

What workloads are you running on OpenStack?

The ARDC Nectar Research Cloud supports a very wide range of research workloads from thousands of projects in all major research areas, including hosting many research platforms (virtual research environments) and other digital research services.

Currently, the ARDC Nectar Research Cloud hosts more than 200 research platforms and services that are used by more than 50,000 researchers in Australia and internationally across all fields of research. https://ardc.edu.au/services/nectar-research-cloud/digital-services/

What is the scale of your OpenStack environment?

830 servers, 33,000 physical cores, approximately 160,000 vcpus, 2 PB object storage, 5 PB volume storage, across 7 sites.

Why has your organization seen such significant growth over the last year?

The Australian federal government, through its National Collaborative Research Infrastructure Strategy (NCRIS), has provided significant capital investment for digital research infrastructure for the period 2018-23, including a refresh and capacity uplift for the ARDC Nectar Research Cloud. Over the last few years some of our partner sites, particularly University of Melbourne and Monash University, have also had large additional investments in cloud infrastructure to support their researchers. Also the ARDC has partnered across research organizations in Australia to evolve existing or develop new national research platforms across multiple research disciplines.

What is next for your OpenStack deployment? Are you expecting similar growth of the upcoming years?

Over the next two years we are focussing more on high-end infrastructure such as GPU and large memory servers, so we expect much slower growth in more generic cloud capacity. Planning for NCRIS investment in national research infrastructure for 2023-28 is currently underway, so we can’t yet say how capacity is likely to change beyond 2023.

Other activities over the next two years include setting up separate OpenStack regions to support analysis of sensitive data, trailing new technology (DPUs) to test micro-segmentation of cyber security for research environments, developing a national service and leading best practice & developing standards for containers and orchestration for research (ARCOS) and developing new services for using Jupyter notebooks and virtual desktops.