NIMBUS User Guide for DARE

Note

Nimbus was discontinued in 2025. This guide is retained for reference purposes.

• 1. Sign up
• 2. Using Nimbus
  • 2.1 Initialising a Nimbus Instance
• 3. Setting up your Nimbus instance
• 4. Using Docker
  • 4.1 Creating a Docker container
  • 4.2 Running a Python script in a Docker container
  • 4.3 Connecting into a Docker container on Nimbus with Visual Studio Code

Pawsey’s cloud service, Nimbus is a national research-specific cloud service, open to any Australian researcher, including from state government. It is flexible and scalable to your research needs. You are able to access it whenever you wish, with no waiting in queues and you have total control over your instances to install and run whatever you want. It provides a great intermediary between the University of Sydney's HPC Artemis (for cluster jobs, using big data, or with large RAM needs) and your own local computer. It can be used to great effect for medium-sized jobs, removing the need to thrash your laptop day and night!

1. Sign up

You can signup for Nimbus through the Pawsey Application Portal. Once you hit the "Login" button and select "The University of Sydney", you will be able to use your SSO credentials.

This guide guides you through the application process. This application asks you to provide details on:

• Your work to justify the allocation.
  • NOTE: If you are asking for a relatively small amount of resources (e.g., 1 or 2 instances) it is highly likely you’ll be approved. If you are asking for larger amounts of resources, then you will need a more detailed description on why you need that amount.
  • You therefore don't need to go into great detail as to your use if you just want a standard "flavour". GPUs are also in high demand and will need to be justified.
• You will be asked to select an instance "flavour" corresponding to the CPU/RAM combination you will require for your research.
  • A guide to choosing from the "flavours" can be found here. If you are unsure of which flavour to choose, please talk to one of the DARE research engineers for assistance.

After submission, there will be a wait time of a few days before your application is approved.

All projects now have a six-month validity period, after which an application must be submitted for an extension if continued Nimbus use is required (an automated email will be sent with a link to a pre-filled form).

2. Using Nimbus

2.1 Initialising a Nimbus Instance

A comprehensive guide can be found here, but please feel free to get in contact with one of the DARE research engineers if you require assistance on any of the below:

https://pawseysc.github.io/using-nimbus/

This guide takes you through the steps of:

• Generating a key pair for authentication.
• Launching an instance, including the specification of compute resources:
  • Choosing an instance image - you can find the various Ubuntu etc images here.
  • Choose a volume size for this image - this should be just enough to cover the operating system. We will create a separate volume for your scripts/data later.
  • Choosing network settings - you should use the Public External network option for most use cases. Otherwise, you will need to allocate a floating IP.
  • Security Group - ensuring that -SSH is added to allow SSH access.
• Accessing your instance.
  • Following the instructions your SSH call should look like ssh -i ~/.ssh/My_Key_Pair.pem ubuntu@148.118.##.## (note the login name is just the instance type e.g., ubuntu).
  • The easiest way to do this for most DARE users is via the Visual Studio Code editor and the "Remote Development" extension pack. You can follow the instructions here to set this up.
  • Alternatively, you can SSH from your favourite terminal.

3. Setting up your Nimbus instance

We will create a separate volume to store our data and mount this to the instance. This avoids any problems that could occur if you fill up the actual instance volume. You can follow the instructions here to:

• On the dashboard:
  • Use the dashboard to create a volume (specifying size).
  • Attach it to your instance.
• From your SSH session (in terminal or via Visual Studio Code):
  • Check the volume exists and format it (only if it's a new volume).
  • Mount the file system - mount this to /project to follow the Docker example below:
    • check the volume: sudo fdisk -l /dev/vdc
    • (ONLY) if this is a new volume then sudo mkfs.ext4 /dev/vdc (this will erase contents of the volume).
    • mount to /project with sudo mount /dev/vdc /project.
  • Set permissions to ensure access.
    • sudo chown ubuntu /project

The best way to transfer files for most use cases is to use an FTP/SFTP client (you can also use command line tools e.g., scp, rsync). FileZilla is a popular client and instructions for its use with Nimbus can be found here. A great choice on Mac is Cyberduck for which you would:

• Click the "Open connection" button.
• Copy your public external IP (148.118.##.##) to server.
• username: ubuntu (or your selected instance flavour).
• Select your SSH private key from the dropdown. This should appear automatically if you have your key stored in ~/.ssh/.
• Connect.

This guide covers some basic use cases, but does not touch on any of the more advanced supercomputing features of Nimbus. For more information on these features, please see the Nimbus User Guide or chat to a DARE research engineer.

4. Using Docker

4.1 Creating a Docker container

Below is an example using Docker to ensure a consistent Python environment from your local computer to the Nimbus instance. If you need further help or need different software, please contact a DARE research engineer.

First, we need to develop a Dockerfile to describe our environment. There is a wealth of information on creating Dockerfiles. For this example, we will generate a container with an Ubuntu version of Linux, with Python installed. A simple introduction can be found here:

The code for our Dockerfile is below:

# syntax=docker/dockerfile:1
FROM ubuntu:18.04
# Create a directory to mount our data volume.
RUN mkdir /project
# Install Ubuntu libraries and packages.
RUN apt-get update -y && \
    apt-get install git curl -y
# Set some environment variables we will need.
ENV PATH="/build/miniconda3/bin:${PATH}"
ARG PATH="/build/miniconda3/bin:${PATH}"
RUN mkdir /build && \
    mkdir /build/.conda
# Install Python3.9 via miniconda.
RUN curl -O https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh &&\
    bash Miniconda3-latest-Linux-x86_64.sh -b -p /build/miniconda3 &&\
    rm -rf /Miniconda3-latest-Linux-x86_64.sh
WORKDIR /build
RUN conda install python=3.9 numpy ipykernel

Briefly, we are:

• Using the Ubuntu 18.04 base image.
• Ensuring its packages are up to date (using apt-get).
• Creating the /project directory.
  • NOTE we will mount the created Nimbus volume to this.
• Adding paths to the conda install so that we can use "conda install …".
• Downloading and installing the latest version of miniconda.
• Ensuring Python 3.9 and numpy are installed.

You can then add code on the end to install any Python package you would like as you would on your usual conda install, i.e., via pip, conda-forge or a yml file.

4.2 Running a Python script in a Docker container

Now let's build this Docker image on the Nimbus instance remembering the instructions for SSH connection from Visual Studio Code here:

1. Save your Dockerfile from the code above to a file pytest.Dockerfile and transfer it to Nimbus using Cyberduck (or FileZilla) to the location /project/docker/pytest.Dockerfile.
2. Using your SSH session or Visual Studio Code, navigate to the location of your Dockerfile.
   • cd /project/docker/
3. Build the image from our pytest.Dockerfile and name it pytest using the command:
   • docker build --tag pytest -f pytest.Dockerfile .
4. Wait for Docker to build the image, downloading the packages etc that it needs.
5. We can run interactive sessions (say for the pytest image) to try things out with something like docker run -v "/project:/project" -w /project -it pytest.
   • This creates an interactive session with the image.
   • Maps your directory (mounted volume) /project to an equivalent location /project within the Docker image so that you can access the files you have on Nimbus.
   • Changes the working directory in the Docker instance to /project.
6. You can terminate the session with exit.

Below, you can see an example checking Python works correctly on Nimbus running a Docker container (as per the Dockerfile installation above). This example uses Visual Studio Code to connect to Nimbus.

As a further test, we will copy the files "test.py" and "test.ipynb" from the examples directory of this repository to a folder on our nimbus instance: /project/python. We can run our script test.py from the command line using:

docker run -v "/project:/project" -w /project pytest python /project/python/test.py

The output is shown below.

4.3 Connecting into a Docker container on Nimbus with Visual Studio Code

With a little configuration of Visual Studio Code we can connect directly to a Docker container on Nimbus. This can be useful to interactively run code on Nimbus either directly from a .py file (including debugging) or from a Jupyter Notebook. Starting from the same point as above, with the /project folder open in Visual Studio Code via SSH, we will do the following:

• Ensure you have the "Remote - Containers" extension installed in Visual Studio Code (it may ask you to reinstall on your SSH session).

  

• In the /project directory create a subdirectory /project/.devcontainer/ and file /project/.devcontainer/devcontainer.json.
  • In this file we need the following to replicate the command from above (docker run -v "/project:/project" -w /project -it pytest).
  • Again, we are telling Nimbus/Docker to use our built Docker image pytest, mapping the /project folder to the container and changing the working directory in the Docker container to /project.

      {
          "image": "pytest",
          "workspaceMount": "source=/project,target=/project,type=bind,consistency=cached",
          "workspaceFolder": "/project"
      }

  

• In the bottom left, click the green "Remote Development" icon which should be showing your current SSH connection. This will show your remote development options and, if you have the "Remote - Containers" extension installed, you will see "Remote-Containers" settings as below.

  

• Select: "Open Folder in Container...".
• We will open our /project folder.
• If you have followed the instructions for the devcontainer.json above, you will now have a Visual Studio Code window open with a pytest Docker container on Nimbus - you can check that the "Remote Development" status in green in the bottom left reads something like:
  • Dev Container @ssh://146.118.##.##
• We can trial this by opening our /project/python/test.ipynb notebook. You should be able to select your conda environment, run and confirm we are in the correct working directory and with the correct CPU/Memory configuration of your Nimbus instance.
• You can use the remote development options to reopen the folder in SSH again to exit the Docker container.
• docker rm $(docker ps -a -q) will kill all containers.
• docker image rm pytest will delete your image.

Acknowledgements

Main author: Joshua Simmons. Secondary updates: Travis Stenborg.