Telegram Chat Finetuning

This repo was meant to experiment mimicking my typing style in my groupchat. Obviously as this is a privacy sensitive thing I chose to finetune and host my bot locally.

Using this Repo

1. Download Telegram Data
   1. Telegram -> Settings -> Advanced -> Export Telegram Data -> JSON
2. Run json_preprocessing.ipynb, using your respective filename and intended groupchat
   1. Creates data_export/{grpchat_name}.json
   2. Creates {grpchat}-df.pkl
3. Follow steps in Qwen3_(14B)_Reasoning_Conversational, taken from the official repo to create a model directory i.e. lora_model_John
   1. If the training steps take too long you can reduce the number of "steps" taken. I think 6000 training steps is not too bad and takes about 1 night of training
4. Configure Bot
   1. Search Botfather on Telegram and follow the wizard to create a telegram bot and get its name, username (@) and token
   2. For each bot you have, copy sample.env and rename it, and fill in the name, username (@) and token, as well as your prompt injection
   3. Leave .env_global as the default
5. Go to main.py and add the paths to your respective envs to PERSONALITY_FILES
   1. If adding multiple bots, remember that the first entry is the MasterBot so all messages will filter through this
6. Run main.py

Communicating with the bots

The bots will take the last 10 minutes of messages as context and generate new messages with /generate

Bots can also operate in groupchats, but BotFather -> Bot -> Bot Settings -> Allow Groups ENABLED -> Group Privacy Disabled must be done. The MasterBot in your main must also be in the groupchat to read and ask the other bots to generate messages.

Config Commands

You can also play around with some of the parameters through /config especially if responses are either too random or too repetitive: Temperature [0.00 : 2.00] : Affects the randomness of responses /config temp 0.7

TOP_P [0.00 : 1.00] : Confidence Interval of responses. Hence a higher value creates more random responses /config top_p 0.97

Top_K [0 : 1000??] : The generated message is chosen from a random sample among the top K responses. Having a higher top_k means less likely responses are also considered. /config top_k 150

/clear will clear the context. Do this if the bot is repeating itself

/recent shows the context. Mainly for debug

Other notes

Each bot takes up a certain amount of DRAM so at least my computer with 16 GB of DRAM is limited to ~4 bots at max. The inference is sequential so the GPU is not the bottleneck to scalability.

Choice of Model

At the time of writing (August 2025) I chose to go with QWEN3 4-bit 8B model as my hardware was just a RTX4070 Super with 12GB of VRAM and 16GB of DRAM

TODO:

[X] Let the bots communicate with each other (MasterBot architecture) [ ] Add sticker support (i.e. emoji support) [ ] Smarter responses [ ] Logging