Building llama.cpp with SYCL

[batonac]

I'm not sure what your motivations are in packaging this, but it would be swell to have instructions on how to utilize this to build llama.cpp with SYCL support on NixOS.

Thanks for all your efforts here!

[kilyanni]

This is currently just a staging area. I'm working on getting everything to a point where llama.cpp and similar can be built, but I'm not quite there yet.
Getting ggml (the library behind it) to work is my current goal

I hope to ultimately get all of this upstreamed to nixpkgs, btw.

If you want to experiment around before that: The main idea is to compile ggml or similar with this repos src.llvm.stdenv.mkDerivation instead of regular stdenv.mkDerivation, and it'll compile with the open-source version of the intel compiler.

[batonac]

I gotcha. I'll try to keep tracking with your efforts and tinkering with this, as I have time and energy. I deeply appreciate all your efforts. Hopefully eventually we can use services.ollama.acceleration = "sycl" in NixOS 🤩

[batonac]

BTW, this is cool: https://www.phoronix.com/news/LLVM-Upstream-libsycl