Skip to content

catboost: use llvm 14 for cuda#333562

Merged
RossComputerGuy merged 1 commit into
NixOS:masterfrom
ExpidusOS:feat/catboost-llvm-upgrade
Mar 6, 2025
Merged

catboost: use llvm 14 for cuda#333562
RossComputerGuy merged 1 commit into
NixOS:masterfrom
ExpidusOS:feat/catboost-llvm-upgrade

Conversation

@RossComputerGuy

Copy link
Copy Markdown
Member

Description of changes

Upgrade to a newer LLVM version for CUDA (related #305146), format so CI doesn't complain.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@ofborg ofborg Bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-linux: 1 This PR causes 1 package to rebuild on Linux. labels Aug 10, 2024
Comment thread pkgs/top-level/all-packages.nix Outdated
@RossComputerGuy RossComputerGuy force-pushed the feat/catboost-llvm-upgrade branch from e19db93 to 53914c7 Compare August 16, 2024 01:15
@ofborg ofborg Bot added 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. and removed 10.rebuild-linux: 1 This PR causes 1 package to rebuild on Linux. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Aug 16, 2024
@tomberek tomberek requested a review from SomeoneSerge August 16, 2024 05:39
Comment thread pkgs/top-level/all-packages.nix Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah ideally we'd accommodate in cudaPackages (the same logic that handles gcc version ranges) but admittedly that might take a non-trivial refactoring and it's ok to postpone it. CC @NixOS/cuda-maintainers

@wegank wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Sep 27, 2024
@RossComputerGuy RossComputerGuy force-pushed the feat/catboost-llvm-upgrade branch from 53914c7 to 290a347 Compare October 10, 2024 16:01
@ofborg ofborg Bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Oct 10, 2024
@wegank wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Nov 1, 2024
@RossComputerGuy RossComputerGuy force-pushed the feat/catboost-llvm-upgrade branch from 290a347 to ab66c4d Compare January 23, 2025 16:25
@RossComputerGuy RossComputerGuy changed the title catboost: use llvm 14 for cuda & format with nixfmt-rfc-style catboost: use llvm 14 for cuda Jan 23, 2025
@ofborg ofborg Bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jan 23, 2025
@emilazy

emilazy commented Jan 23, 2025

Copy link
Copy Markdown
Member

See #370663 (comment).

@RossComputerGuy

Copy link
Copy Markdown
Member Author

Is catboost/catboost#2540 still a problem? I tried bumping CUDA to 12 but ran into errors like this:

catboost>   /nix/store/ccm40rk4zlaarhqncxrfmav3yxkrf89h-glibc-2.40-36-dev/include/bits/string_fortified.h(107):
catboost>   error: linkage specification is incompatible with previous "stpncpy"
catboost>   (declared at line 499 of
catboost>   /nix/store/ccm40rk4zlaarhqncxrfmav3yxkrf89h-glibc-2.40-36-dev/include/string.h)
catboost>
catboost>
catboost>      stpncpy (char * const __attribute__ ((__pass_object_size__ (2 > 1))) __dest, const char *__src, size_t __n) noexcept (true)
catboost>      ^

@SomeoneSerge

Copy link
Copy Markdown
Contributor

Is catboost/catboost#2540 still a problem? I tried bumping CUDA to 12 but ran into errors like this

I didn't check but I'd not expect this to change: they use a higher level tool (yatool) to generate CMakeLists that hard-code most of the relevant parameters

@RossComputerGuy

Copy link
Copy Markdown
Member Author

I've brought up the CUDA 11 thing with upstream - catboost/catboost#2755. I don't consider it a blocker for this PR. I'm mainly worried about getting things off LLVM 12.

@andrey-khropov

andrey-khropov commented Feb 4, 2025

Copy link
Copy Markdown

Is catboost/catboost#2540 still a problem? I tried bumping CUDA to 12 but ran into errors like this:

catboost>   /nix/store/ccm40rk4zlaarhqncxrfmav3yxkrf89h-glibc-2.40-36-dev/include/bits/string_fortified.h(107):
catboost>   error: linkage specification is incompatible with previous "stpncpy"
catboost>   (declared at line 499 of
catboost>   /nix/store/ccm40rk4zlaarhqncxrfmav3yxkrf89h-glibc-2.40-36-dev/include/string.h)
catboost>
catboost>
catboost>      stpncpy (char * const __attribute__ ((__pass_object_size__ (2 > 1))) __dest, const char *__src, size_t __n) noexcept (true)
catboost>      ^

This does not look like an error caused by unsupported CUDA architecture related to catboost/catboost#2540. This is something else.

In conda catboost is built successfully with CUDA 12.0 (and CUDA 12.1 and 12.2 should work as well) using patching CMakeLists by removing unsupported architectures like this:

https://github.com/conda-forge/catboost-feedstock/blob/234a16cbeeeb12bb02bb3eff7282a1cbce9c979f/recipe/build.sh#L57

Static vs shared CUDA libraries could also be an issue (but it does not depend on CUDA version): https://github.com/conda-forge/catboost-feedstock/blob/234a16cbeeeb12bb02bb3eff7282a1cbce9c979f/recipe/build.sh#L64-L66

@RossComputerGuy

Copy link
Copy Markdown
Member Author

Ok, I'm going to definitely just consider the CUDA bump a non-issue for this PR. I don't have the time or knowledge to figure out this problem. My concern has only been getting things off LLVM 12 so we can drop it.

@RossComputerGuy RossComputerGuy merged commit 22d24c1 into NixOS:master Mar 6, 2025
@RossComputerGuy RossComputerGuy deleted the feat/catboost-llvm-upgrade branch March 6, 2025 08:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants