Support Custom Model Parameters In Request Body

[nitg16]

What feature would you like to see?

Allow us to send custom model parameters in request body for custom openai compatible providers.

Example: temperature, top_p.

[zemaj]

What sort of context would this be for? As in during the main chat requests?

[nitg16]

Yes. When we send a request to chat completion api endpoint, include additional params in request body.

Example:

curl https://api.openai.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "Provide three suggestions about time management."
      }
    ],
    "temperature": 0.7,
    "top_p": 0.8
  }'

[freelanceagent1]

Yes. When we send a request to chat completion api endpoint, include additional params in request body.

Example:

curl https://api.openai.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful assistant."
      },
      {
        "role": "user",
        "content": "Provide three suggestions about time management."
      }
    ],
    "temperature": 0.7,
    "top_p": 0.8
  }'

can't you just attach those via an translate proxy as a temp workaround?