TIC-Ollama-CSF

This repository contains jobscripts to install, setup, and run Ollama on the University of Manchester's CSF3 HPC system, and basic .py files to run Ollama through python. Ollama is a wrapper for llama.cpp and is an easy way to locally run off-the-shelf LLMs like Llama or Gemma with minimal faff.

Instructions

1. Transfer this repo to your CSF3 account.
2. (Optional) Edit ollama.config to choose the version of Ollama to install. This must be one of the releases on the Ollama GitHub.
3. On CSF3, run sbatch jobscript_ollama_install_and_test. This will call the three other jobscripts in turn:
   1. jobscript_ollama_install: Install Ollama to the /software folder in your home directory.
   2. jobscript_ollama_setup_py: Setup a python environment (called ollama) to run Ollama in python
   3. jobscript_ollama_test: Test the existing installation in python
4. Check the slurm .out file to see if the install was successful- there should be no errors and Ollama should be running on the GPU
5. You can now run your own python scripts using Ollama. A basic example is shown in jobscript_ollama_run and the associated python file ollama_run.py. Note that you have to start ollama-server each time.

Running Ollama on a GPU

Users should note that all of these jobscripts use a GPU (L40S by default) so the jobs may have to wait in the queue until one is available. Ollama can also run on a CPU but will be much slower.

To check if the GPU is working, look in the log file for the ollama-setup jobscript. Near the end you will see something that looks like this:

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A         2760804      C   ...are/ollama/v0.15.6/bin/ollama       3132MiB |
+-----------------------------------------------------------------------------------------+

Here, you see that ollama is running on GPU 0. If you don't see ollama listed here, it means it is not using the GPU for some reason. Below, you will then see sections that look like this:

index, utilization.gpu [%], utilization.memory [%], memory.total [MiB], memory.used [MiB], memory.free [MiB]
0, 57 %, 55 %, 46068 MiB, 3141 MiB, 42318 MiB

Looking at process 0, you see that Ollama is utilising 57% of the GPU power.

Contact details

If you have any questions, please contact jonathan.taylor@manchester.ac.uk