Introduction to CANFAR Computing

Are you an astronomer and would like to perform reproducible research? With the Canadian Advanced Network for Astronomical Research infrastructure you can build your data processing pipeline, interactively analyze results, and launch thousands of batch jobs on multiple clusters, using exactly the same operating system and software stack.

Virtual Machines

The CANFAR infrastructure uses Virtual Machines to deploy your software. A Virtual Machine is a computer system running inside another computing system. The primary (or real) computing system provides the link to the hardware on which the computation is being done and provides an environment in which the ‘Virtual Machine’ can be created. This extra level of abstraction comes with a small performance price which is easily overcome by considering the savings in development costs. Once your computation works in your VM you can be certain that the computation will work on the computers that your processing will execute on.

Compute Canada and OpenStack

The CANFAR computing resources are currently provided as an OpenStack cloud managed by Compute Canada.

Before you start

You will need to register to CANFAR. The CANFAR team will take care of your registration to Compute Canada infrastructure.

Managing your resources

With the user interface

The OpenStack dashboard is a web interface to manage your resources for your persistent computing resources. Compute Canada has a visual quick start guide to show you how to use it.

Some other tutorials that go into greater depth may be also be found at other OpenStack clouds, which all look similar. The RAC documentation at Cybera is another good source.

With the command line tools

To automate the management of your project resources, you can automate them using the OpenStack command line clients. We document some of the command line tools to help you getting started.

Batch Processing

If you need to run large processing jobs, we recommend to make use of the CANFAR batch services. The resources are much larger and more adequate than for the personal resources you get with your project. The batch documentation has the basics you need to help you using the CANFAR batch services.

Tutorial

A tutorial covering all the above use cases is also available. The documentation will show you how to setup Virtual Machines, install software and process some data, and storing results in your VOSpace.