Skip to content

Installation

Jump to bottom
dr1315 edited this page Aug 18, 2025 · 25 revisions

Licensing

Some dependencies of the ORAC library require users to obtain a license. Before installing the software, please review the terms and conditions at the following sites:

You must have obtained these licenses before using any pre-compiled binaries related to this project. If this is a problem, please contact us on DEVORAC at jiscmail.ac.uk to discuss alternatives.

Installation

There are three methods to install the ORAC processor. We currently only support Linux. Due to licensing issues, we cannot distribute the library through a public channel but it is available on the JASMIN cluster; just apply for access to the Aerosol CCI group workspace. If you are not eligible for access, please contact us on DEVORAC at jiscmail.ac.uk and we can direct you to the files.

Standard Installation: Compile ORAC from source

The simplest way to install ORAC is to:

  • Clone the repo to an appropriate place.
  • Build the conda environment.
  • Install and compile ORAC.

NB// This should be done in one go to ensure that any variables aren't lost or overridden in the overall process, which can cause issues.

Cloning ORAC repo

Make sure to cd to an appropriate location where the ORAC repo can be cloned into. Also ensure that you have git available on your system (this can be done using the command which git, which should show you the loaction of the git binary file if it is available on your system).

Then, run the following code block:

git clone https://github.com/ORAC-CC/orac.git
cd orac
export ORACDIR=$PWD

This ensures that the path to the ORAC repo is set in your environment and makes the rest of the installation process simpler.

Building the conda environment

It is best to use the conda package manager for handling the dependencies ORAC requires to run. Conda creates self-contained virtual environments that can be turn "on" and "off" as needed, preventing potential conflicts with other software you may want to use when not using ORAC.

There are 2 installers for conda:

  • Miniconda3 from Anaconda; this will install with the Anaconda defaults channel, which is ONLY FREE TO INDIVIDUALS AND SMALL ORGANISATIONS and requires a paid licence for larger organisations.
  • Miniforge3 from conda-forge; this is a more minimal, open-source version of conda that does not use the defaults channel, instead only using conda-forge by default.

NB// Conda virtual environments can become quite large, using a lot of disk space. It is recommended to install conda somewhere with a large amount of space available.

It is recommended to install miniforge3, as ORAC does not need any packages from the Anaconda defaults channel. It will be assumed that you have done this for the rest of the installation process. Installing miniforge3 is done by running the following commands:

wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh
bash Miniforge3-$(uname)-$(uname -m).sh
rm Miniforge3-$(uname)-$(uname -m).sh

You can then activate conda using the activate conda command. However, you may need to initialise conda using:

source "miniforge3/etc/profile.d/conda.sh"

which will allow you to create the ORAC virtual environment.

IMPORTANT If you are on JASMIN and want to use the conda channel available through aerosol_cci, you will need to point conda at the right place so it can grab packages from the local channel. This is done by adding the path to your miniforge3 dir:

ln -s /gws/nopw/j04/aerosol_cci/conda_channel $(conda info --base)/conda-bld

Similarly, if you are not on JASMIN, you can place the conda channel into your miniforge3 dir with the name conda-bld.

NB// The conda channel MUST be called conda-bld for conda to pick it up as the local channel. However, this is not best practice if you intended to build your own conda channel at a later date. Instead, you can point to the conda channel using -c /path/to/aerosol_cci_conda_channel when setting up the environment.

We can now build the conda environment. This is simply done with the following command:

conda env create -y --file ${ORACDIR}/feedstocks/dependencies.nonmpi.yml

Assuming that you would like to build ORAC with nompi, which is necessary on JASMIN. There is also an openmpi .yml file for systems that support multi-threaded compilation.

Note on the default channel

Conda distributes software through "channels", which can be a folder or web address. By runnnig the command:

ln -s /gws/nopw/j04/aerosol_cci/conda_channel $CONDA_PREFIX/conda-bld

we have made it so that we do not need to point to that channel every time we build an ORAC environment with conda.

NB// You must have access to the Aerosol CCI group workspace to access the /gws/nopw/j04/aerosol_cci/conda_channel. If not, replace that path with wherever you decompressed the ORAC channel.

Thereafter, you can use -c local to tell conda to took in the ORAC channel for packages. (This is a bad idea if you ever intend to build your own packages with conda.) The instructions to follow assume you did this.

We can further save effort by creating the file .condarc in your home folder to define the default channels that should be searched. We recommend the file contain,

channels:
  - conda-forge
  - avalentino
  - defaults
  - local

14 Feb 2022: At the moment, an alternative to conda, called mamba, cannot use the avalentino channel as it has unexpected formatting in its metadata. As such, the third line of the above should be removed if you wish to use mamba.

Installing and compiling ORAC

Now that the ORAC conda environment is set up and all dependecies should be available, the following script should install ORAC (as long as conda is activated):

cd $ORACDIR
# Make directories
mkdir common/obj
mkdir pre_processing/obj
mkdir src/obj
mkdir post_processing/obj
mkdir derived_products/broadband_fluxes/obj
mkdir derived_products/broadband_fluxes/obj/bugsrad
mkdir derived_products/broadband_fluxes/obj/fu_liou
# Overwrite activate and deactivate for orac_git
SCRIPT_DIR=$(conda info --base)/envs/orac_git/etc/conda
for CHANGE in "activate" "deactivate"; do
    mkdir -p "${SCRIPT_DIR}/${CHANGE}.d"
    cp "${ORACDIR}/feedstocks/orac/${CHANGE}.sh" "${SCRIPT_DIR}/${CHANGE}.d/${CHANGE}-orac.sh"
done
cat <<EOF >> $SCRIPT_DIR/activate.d/activate-orac.sh
# Point to this environment's libraries
if [ -n "\${ORAC_LIB}" ]; then
    export ORAC_LIB_SAVE="\${ORAC_LIB}"
fi
export ORAC_LIB="\${ORACDIR}/config/lib.conda.inc"
if [ -n "\${ORAC_ARCH}" ]; then
    export ORAC_ARCH_SAVE="\${ORAC_ARCH}"
fi
export ORAC_ARCH="\${ORACDIR}/config/arch.conda.inc"

if [ -n "\${PYTHONPATH}" ]; then
    export PYTHONPATH_SAVE="\${PYTHONPATH}"
fi
export PYTHONPATH="\${ORACDIR}/tools:\${CONDA_PREFIX}/lib:\${PYTHONPATH}"
EOF
cat <<EOF >> $SCRIPT_DIR/deactivate.d/deactivate-orac.sh
if [ -n "\${ORAC_LIB_SAVE}" ]; then
    export ORAC_LIB="\${ORAC_LIB_SAVE}"
    unset ORAC_LIB_SAVE
else
    unset ORAC_LIB
fi
if [ -n "\${ORAC_ARCH_SAVE}" ]; then
    export ORAC_ARCH="\${ORAC_ARCH_SAVE}"
    unset ORAC_ARCH_SAVE
else
    unset ORAC_ARCH
fi

if [ -n "\${PYTHONPATH_SAVE}" ]; then
    export PYTHONPATH="\${PYTHONPATH_SAVE}"
    unset PYTHONPATH_SAVE
else
    unset PYTHONPATH
fi
EOF

And then compile ORAC with the following script:

# Activate the ORAC conda environment; if it is already activated, you do not need to do this
conda activate orac_git
# Go to the top-level ORAC dir
cd $ORACDIR
# Compile ORAC
make

NB// If you wish to recompile ORAC in the future, you do not need to run the installation script, only the compilation script. It is also recommend to set ORACDIR in your start-up script, e.g. write export ORACDIR=/full/path/to/orac in your .bashrc, to ensure it is always set for future use.

ORAC should now be installed on your system, i.e. the main processor, ${ORACDIR}/src/orac, should have been generated.

Notes on installation and running ORAC

You'll see that we have added three clauses to the activation files during the installation process. These define three environment variables,

  • ORAC_LIB is used by the makefiles and Python scripts to direct ORAC to the location of the external libraries. Those are all in the conda installation.
  • ORAC_ARCH is used by the makefiles to define the arguments given to the compiler.
  • PYTHONPATH tells Python where to go looking for modules. We add the pyORAC directory to it, as well as the conda library folder where RTTOV's Python wrapper is stored. Other means of getting pyORAC into Python are available if you know what you're doing.

If you wish to run ORAC using the pyorac commands, you then need to create a local defaults file in tools/pyorac. A generic example can be found in that directory while the specific example distributed with the binaries can be found in feedstocks/orac.

However, ORAC is run using driver files, which pyorac sets up automatically for you. There is no expilict need to create a local defaults file if you do not wish to use pyorac and are more comfortable generating your own ORAC driver files.

Alternative Installation: Pre-compiled binaries (minimal use)

If you wish to use the code as-is (e.g. without compiling or changing anything), the ORAC software library and it's dependencies can be installed using conda. We recommend installing ORAC into a fresh environment as many of the dependencies clash with common packages from popular conda channels and (until they fix feature management) our installation is likely to break any existing environment.

To install ORAC call,

conda create -y -n orac_release --override-channels -c local -c conda-forge pyorac

You may replace orac_release with any other name you desire and local with the path to the ORAC channel. If you would like multi-threaded processing instead call,

conda create -y -n orac_release --override-channels -c local -c conda-forge pyorac "orac=*=mpi_openmpi*"

Auxiliary files

ORAC requires a variety of satellite, meteorological, and geographical data to run. A brief summary of them can be found here. A more detailed summary SHOULD BE DRAFTED.

The code requires a number of Static Application Data (SAD) files,

  • instrument channel characteristics (one file per instrument channel),
  • aerosol or cloud radiative properties for each supported cloud class in each instrument channel (Look Up Tables). These are stored in the Aerosol and Cloud CCI group workspaces or can be requested from us.

UPDATE: 04-Apr-2025 ORAC has now moved to using netCDF files for the LUTs. These are self-contained, i.e. a single file for all instrument channel and radiative properties per aerosol or cloud type, and are available at /gws/nopw/j04/oracluts/public in JASMIN. For external users, LUTs are available online.

Python scripts

The recommended method to run the ORAC suite is with the provided Python script, orac.py, which is installed by pyorac and is described here. To avoid typing lengthy paths with every call, these scripts use a file called local_defaults.py to list the default locations of all auxiliary files. The version of that file distributed with the binaries assumes that a user:

If either of these are not true, or you wish to control other optional information, please edit the file $CONDA_PREFIX/envs/orac_release/lib/python{pyversion}/site-packages/pyorac/tools/pyorac/local_defaults.py, where ${pyversion} is the main release number you have installed, of the form 3.X, and can be determined with python --version`

An example local defaults is kept at here. For the alternative/minimal installation, you will desire:

ORAC_DIR = os.environ["CONDA_PREFIX"] + "/bin"
ORAC_LIB  = os.environ["CONDA_PREFIX"] + '/share/orac/lib.inc'

which points the Python scripts to an ORAC settings file.

Intense use: Compile all dependencies from source

This is the previous method of installing ORAC and is not recommended for most users as many of these dependencies can be difficult to compile. If you intend to run on JASMIN, you may find this page useful as it outlines the existing libraries and resources.

Prerequisites

A Fortran 2003 compiler is required. The ORAC code is developed and tested on GNU/Linux platforms using gfortran (versions 7.5.0, 12.1.0), the Intel ifort compiler (versions 17, 18.0.5), and PGI (version 21.9-0). We expect other compilers will work with minimal effort and the code is also known to compile and run on MacOS using gfortran.

The bison and flex tools are required, both of which are standard development tools available on GNU/Linux. Version 3 of bison is preferred. Version 2 will work, but you must comment line 17 and uncomment line 22 here.

Libraries

Numerous libraries are required by the ORAC code and their installation is explained here. For each, there is a directory in feedstocks. The location of the source code is specified in the meta.yaml file under the source: tag with sub-headers,

  • git_url pointing to a git repository;
  • url pointing to a downloadable tarball;
  • path pointing to files in the feedstocks directory which should be manually copied across (to the destination specified by folder or the top directory);
  • patch specifying patch files that should be called with patch -p1 -i $PATCHFILE from the top directory.

The build.sh file outlines how we expect the library to be compiled, but should not be taken as definitive instructions for installation. The exact flags and patches required appear to be highly dependent on your machine architecture. If you are having difficulty, please email the ORAC Developer list (DEVORAC at jiscmail.ac.uk) for advice.

Some specific notes that may be of use to you but may have been fixed in later version:

  • Your Linux distribution may include some of these libraries already. If you are compiling them yourself (e.g. to include additional features), make doubly certain that your linker points to your compiled library and not the system default. This is our most common cause of compilation difficulties.
  • EMOS library The ECMWF EMOS library is currently required but has proven to be difficult to install (recommendations for an alternative happily accepted). It can be found at https://software.ecmwf.int/wiki/display/EMOS/Releases.
    • This library conflicts with the HDF library as both contain a routine called init, which is repaired with a patch.
    • You must include -DENABLE_GRIBEX_ABORT=OFF as we still use the old syntax.
    • The default configuration for ifort is incorrect. The LARGE_FILE flag should be set to -D_LARGEFILE64_SOURCE, -Vaxlib has been depreciated, the DEBUG flag should be set to -O2, and you may wish to use the icc compiler rather than gcc. (See the attached script.)
    • The default configuration for gfortran is partially incorrect. The FFLAGS variable should be set to -ffixed-line-length-0. (See the attached script.)
    • Verbose output from the library, for debugging, can be turned on by setting an environment variable JDCNDBG=1.
    • This links against the FFTW library, which is usually available in your distribution, but can be downloaded at http://www.fftw.org. Please build both the single and double precision libraries.
  • NetCDF The NetCDF library is required to read/write input/output files. The code has been successfully compiled with NetCDF versions 3 and 4 although NetCDF 4 is recommended. The core NetCDF C library and API is required along with the NetCDF Fortran API, which is shipped separately. They can be downloaded from the Unidata website: http://www.unidata.ucar.edu/downloads/netcdf/index.jsp
    • The library should be compiled from source with Fortran support enabled. If ORAC is to be run on a GNU/Linux machine, using the gfortran compiler, a pre-compiled version the NetCDF library can usually be installed directly using the distributions package manager. Both the NetCDF library and the associated development package, a.k.a "dev" package, should be installed.
  • RTTOV RTTOV is used to calculate the clear-sky transmittances and radiances and can be downloaded from http://nwpsaf.eu/site/software/rttov/downloads. The current version of ORAC requires versions 13.
    • You will need to edit Makefile.local to point to your netCDF4 or HDF5 library directory. We recommend using HDF5 rather than netCDF4. The rttov_compile.sh script can be used to simplify compilation, we recommend compiling with OpenMP support (if appropriate to your architecture) as this enables a significant improvement in execution time.
    • You will also require the emissivity and BRDF atlases along with the RT coefficient files for any sensors you wish to process. At the time of writing, these could be found (at the same website) in:
      • uw_ir_emis_atlas_hdf5.tar
      • uw_ir_emis_atlas_covariances_hdf5.tar
    • The RT coefficient files for ORAC-supported sensors are included as part of the RTTOV distribution. Any future coefficient updates will be available from https://nwpsaf.eu/site/software/rttov/download/coefficients/coefficient-download/
  • No issues have yet been encountered installing the following libraries with their default settings:
  • Optional Dependencies The following libraries are optional, but recommended, in the current ORAC code:

Building the code

All parts of ORAC are compiled using Makefiles. This is controlled by two files:

  • The archfile, which sets the Fortran and C compilers to be used and the settings for those compilers. It is specified by the environment variable ORAC_ARCH.
  • The libfile, which lists the paths to all the libraries required by the compilers. It is specified by the environment variable ORAC_LIB.

Examples can be found in the config folder.

The OBJS variable controls where the compiled object and module files are stored. We recommend setting and, when checking out a fresh copy from the repository, execute the following,

mkdir common/obj
mkdir pre_processing/obj
mkdir src/obj
mkdir post_processing/obj
mkdir derived_products/broadband_fluxes/obj
mkdir derived_products/broadband_fluxes/obj/bugsrad
mkdir derived_products/broadband_fluxes/obj/fu_liou

You can also set OBJS to a full path and all object and module files will be stored in a single folder.

Calling make from the trunk folder should compile all of ORAC. A single part may be compiled by calling make from within it's folder, though all parts of ORAC are dependent on the contents of the common folder. To remove the compilation, call make clean. The command make depend is available to automatically generate file dependencies, which is useful if you add modules to the code.

Note that ORAC responds very well to compiler optimisations, so if you are planing to do extensive processing with the code, you should consider using the appropriate flags (e.g. -O2 or -O3 will work wonders on most compilers). ORAC is parallelized using the OpenMP API for parallel execution on multiple cores. The flags are -qopenmp on ifort and -fopenmp on gfortran.

The -cpp compiler flag is used to enable pre-processing via the C pre-processor. This flag may vary depending on your compiler and version.

Optional modes

A number of optional features are included within the ORAC code. These are implemented using C-pre-processor statements and can be activated by adding -DX to the INC variable defined in your archfile, where X is the desired macro.

  • DEBUG Debug mode, which prints additional warning messages and text outputs.`
  • BKP Breakpoint mode, where details of most mathematical calculations are printed to stdout. While not depreciated, this feature is poorly maintained. It is possible to compile only a single routine with BKP on, but you must re-compile the executable as well or the breakpoint file will not be opened. Note that breakpoint output from different routines within ORAC can be switched in and out by changing the breakpoint level set in OracConstants_m or setting the breakpoint level in the driver file. Only run in the breakpoints configuration for a small number of pixels as the code runs slowly and produces substantial output.
  • WRAPPER The default installation of ORAC creates three executables. Wrapper mode compiles these programs as subroutines that can be called from a single executable.
  • USE_TIMING Prints details of how long each step of the main processor took to execute.`
  • USE_SLATEC Uses the SLATEC package for matrix inversion rather than LAPACK.
  • COMPATIBILITY_MODE Is a debugging mode for the Cox-Munk sea surface parameterisation, using sin and cos to calculate all angles directly.
  • INCLUDE_RTTOV_OPENMP Use multithreading in RTTOV, which must have been compiled with multithreading support.
  • The optional dependencies each have an appropriate macro: INCLUDE_ATSR_SUPPORT, INCLUDE_NR, INCLUDE_FU_LIOU_SUPPORT, INCLUDE_HIMARAWI_SUPPORT, INCLUDE_SEVIRI_SUPPORT.

Running the code

By default ORAC is compiled to use dynamic linking for HDF, HDF5, NetCDF and EPR-API and therefore the LD_LIBRARY_PATH must contain paths to these libraries. The following python one-liner may be of use,

LD_LIBRARY_PATH=$(cd $ORACDIR/trunk/tools; python -c "import os; import orac_utils as ou; print(ou.build_orac_library_path(ou.read_orac_libraries(os.environ['\''ORAC_LIB'\''])))")

History

  • 23-May-2011, Andy Smith
  • 13-Dec-2011, Matthias Jerg
  • 12-Jan-2012, Gareth Thomas
  • 14-Jun-2012, Caroline Poulsen
  • 08-Nov-2013, Adam Povey
  • 25-Jul-2016, Adam Povey
  • 26-Jul-2016, Greg McGarragh
  • 16-Oct-2016, Caroline Poulsen
  • 24-Mar-2017, Simon Proud
  • 19-Feb-2019, Adam Povey
  • 14-Feb-2022, Adam Povey
  • 04-Apr-2025, Daniel Robbins

Clone this wiki locally