tried using ollama; it works

[omfgroflmfaol]

I had some issues in the beginning setting up the config.ini file, but found their causes after some experimenting. I don't know how to create a pull request, but here is a slightly changed section of the AI setup page of the wiki that should help other people set up their config.ini for ollama quicker:

---

Using Other AI Models or GPT-4 via Microsoft Azure

If you have access to other Large Language Models—on cloud platforms, university servers, or even your own PC—you can try integrating them into QualCoder. This, however, is a highly experimental option and has not been very well tested. We would be interested to hear about your experiences.

• The service you wish to use must provide an interface compatible with the OpenAI API. This is often the case, since this API has become a de facto standard over the past few months. Ollama is a popular example of a server for local LLMs that also offers a (mostly) OpenAI-compatible API.
• The available models in QualCoder are defined in the file "config.ini," located in the ".qualcoder" subfolder in the user's home directory. Model definitions have the following format (omit the "<>" marks):

[ai_model_your_model_name]
desc = A description shown in the UI.
       Can have more than one line.
access_info_url = <URL pointing to a website with model info. Can be empty>
large_model = <The exact name of the model in the API>
large_model_context_window = <The maximum number of tokens in a single request>
fast_model = <Can be identical to large_model or name a smaller model, used for simple tasks>
fast_model_context_window = <The maximum number of tokens for the small model>
api_base = <The URL of the API base, e.g., http://localhost:11434/v1/ is the default setting for a local Ollama>
api_key = <The API-key if needed, or None or some random_text instead, e.g. `ollama` for local ollama. Do not leave this field empty.>

The section name must start with the prefix 'ai_model_'.

Again, if you get other services running in QualCoder (or if you tried but failed), we would like to hear about your endeavors.

---

Summary of suggested changes:

• api_base url was missing a / at the end
• setting api_key = None, at least for me, caused errors downstream (presumably because of how these openAI libraries handle NoneType object). Any random_string works - ollama just ignores it. Maybe this issue with None is not limited to ollama. I couldn't test it.
• added link to ollama API docs
• removed "(untested)" because it no longer is :)

[kaixxx]

Thank you very much, I'm happy to include these updates in wiki.
One question: Do you know what error messages you would get when entering "None" as the API-key? Internally, this is translated into an empty string, not an actual None-object.

[omfgroflmfaol]

Thank you for creating this awesome piece of software! Happy to help 👍

Here is the error message I get when using api_key = None:

File "qualcoder/__main__.py", line 3068, in <module>

 File "qualcoder/__main__.py", line 2997, in gui

 File "qualcoder/__main__.py", line 1227, in __init__

 File "qualcoder/ai_llm.py", line 207, in init_llm

 File "langchain_core/load/serializable.py", line 125, in __init__

 File "pydantic/main.py", line 214, in __init__

 File "langchain_openai/chat_models/base.py", line 578, in validate_environment

 File "openai/_client.py", line 110, in __init__

OpenAIError: The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable

[kaixxx]

Thank you. It seems that the Ollama OpenAI-API doesn't accept an empty string. I will mention that in the wiki. I think I'm going to make a separate section about Ollama based on your settings.

[kaixxx]

I have integrated Ollama in the Wiki: https://github.com/ccbogel/QualCoder/wiki/2.3.-AI-Setup and also fixed the error with the API key set to "None".

[ccbogel]

Wonderful