make save and load os independent; add uv to pre-commit to .toml#10
Open
marquesafonso wants to merge 1 commit into
Open
make save and load os independent; add uv to pre-commit to .toml#10marquesafonso wants to merge 1 commit into
marquesafonso wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi there,
Based on this issue I am proposing to you a PR that makes the save and load methods of sparse base index + dense faiss and usearch indices.
The goal is to make baguetter windows compatible.
The problem comes from the fsspec lib and its usage in the LocalFileRepository component.
My approach was to work backwards from there and using the os module to decouple baguetter from fsspec. In the end, the solution practically does away with the LocalFileRepository component, which is rendered useless. But, I kept it as this should be a bigger design decision. Let me know your thoughts.
Right now on a windows machine we get 100% test coverage, whereas before there were 3 errors (on test_base.py, test_faiss.py, and test_usearch.py) which all stemmed from the save and load functions.
Some changes were done to the tests to align with dropping the LocalFileRepository component after first getting 100% code coverage while keeping the component intact.
Please test it in Linux and/or Mac and let me know if it breaks anything there.
Also added uv to pre-commit and pyproject.toml to enable direct usage with uv (by simply running
uv syncon your clone/fork of the repo).