AIUsage

AI Usage in Open Chat

Source Code

• Notebook LM was used to understand technical articles relevant to the creation of code
• CodeRabbit AI was used for PR review

Tokenizer

• Made add_token() (tokenizer.hpp) and the max_frequency (tokenizer_trainer.cpp) more robust to prevent duplicate tokens