Skip to content
OpenGateLLM

Configuration file

OpenGateLLM requires configuring a configuration file. This defines models, dependencies, and settings parameters. Playground and API can share the same configuration file; the Scope column in each table indicates whether a field applies to the API, the Playground, or both.

By default, the configuration file must be ./config.yml file.

You can change the configuration file by setting the CONFIG_FILE environment variable.

Secrets

Section titled “Secrets”

You can pass environment variables in configuration file with pattern ${ENV_VARIABLE_NAME}. All environment variables will be loaded in the configuration file.

Example

models:
  [...]
  - name: my-language-model
    type: text-generation
    providers:
      - type: openai
        url: https://api.openai.com
        key: ${OPENAI_API_KEY}
        model_name: gpt-4o-mini

Example

Section titled “Example”

The following is an example of configuration file:

# -------------------------------- dependencies ---------------------------------
dependencies:
  postgres: # required
    url: postgresql+asyncpg://postgres:${POSTGRES_PASSWORD:-changeme}@${POSTGRES_HOST:-postgres}:${POSTGRES_PORT:-5432}/postgres
    echo: False
    pool_size: 5
    connect_args:
      server_settings:
        statement_timeout: "120s"
      command_timeout: 60


  redis: # required
    url: redis://:${REDIS_PASSWORD:-changeme}@${REDIS_HOST:-redis}:${REDIS_PORT:-6379}
    max_connections: 200
    socket_connect_timeout: 5
    retry_on_timeout: True
    health_check_interval: 30
    decode_responses: False
    socket_keepalive: True


  # sentry:
  #   dsn: ${SENTRY_DSN}


  # langfuse:
  #   public_key: ${LANGFUSE_PUBLIC_KEY}
  #   secret_key: ${LANGFUSE_SECRET_KEY}
  #   base_url: http://localhost:3000


# ---------------------------------- settings -----------------------------------
settings:
  # disabled_routers: ["admin", "audio"]
  # hidden_routers: ["auth"]
  # usage_tokenizer: tiktoken_gpt2
  # app_title: My OpenGateLLM API


  # log_level: INFO
  # log_format: [%(asctime)s][%(process)d:%(name)s][%(levelname)s] %(client_ip)s - %(message)s


  swagger_version: 0.6.0
  # swagger_contact_url: https://github.com/etalab-ia/OpenGateLLM
  # swagger_contact_email: john.doe@example.com
  # swagger_docs_url: /docs
  # swagger_redoc_url: /redoc


  auth_secret_key: changeme
  auth_bootsrap_admin_username: admin
  auth_bootsrap_admin_password: changeme
  # auth_login_type: password
  # auth_login_session_duration: 3600
  # auth_playground_url: http://playground:8501
  # auth_sso_default_role_id: 1
  # auth_sso_default_organization_id: 1
  # auth_sso_oidc_issuer_url: https://...
  # auth_sso_client_id: ${OAUTH2_PROXY_CLIENT_ID}
  # auth_sso_client_secret: ${OAUTH2_PROXY_CLIENT_SECRET}
  # auth_sso_cookie_secret: ${OAUTH2_PROXY_COOKIE_SECRET}
  # auth_sso_cookie_secure: False
  # auth_sso_logout_redirect_uri: https://...
  # auth_sso_oidc_scope: openid email roles given_name usual_name


  # rate_limiting_strategy: fixed_window


  # monitoring_sentry_enabled: True
  # monitoring_postgres_enabled: True
  # monitoring_prometheus_enabled: True


  playground_opengatellm_url: ${OPENGATELLM_URL:-http://api:8000}
  # playground_opengatellm_timeout: 60
  # playground_disabled_pages: []
  # playground_default_model: my-model
  # playground_theme_has_background: True
  # playground_theme_accent_color: purple
  # playground_theme_appearance: dark
  # playground_theme_gray_color: gray
  # playground_theme_panel_background: solid
  # playground_theme_radius: medium
  # playground_theme_scaling: 100%
  # playground_swagger_url: http://localhost:8000/swagger
  # playground_reference_url: http://localhost:8000/redoc
  # playground_documentation_url: https://docs.opengatellm.org
  # playground_sso_access_denied_documentation_url: https://docs.opengatellm.org


# ----------------------------------- models ------------------------------------
# models:
#   - name: albert-testbed
#     type: text-generation
#     # aliases: ["model-alias"]
#     # owned_by: Me
#     # load_balancing_strategy: shuffle
#     # cost_prompt_tokens: 0.10
#     # cost_completion_tokens: 0.10
#     providers:
#       - type: vllm
#         url: http://albert-testbed.etalab.gouv.fr:8000
#         # key: sk-xxx
#         model_name: "gemma3:1b"
#         # timeout: 60
#         # model_hosting_zone: FRA
#         # model_total_params: 8
#         # model_active_params: 8

Configuration

Section titled “Configuration”

Configuration file is composed of 3 sections, models:

  • models: to declare models API exposed to the API.
  • dependencies: to declare both required plugins for the API (e.g. PostgreSQL, Redis) and optional ones (e.g. Sentry).
  • settings: to configure the API.

We don’t recommend to use the configuration file to declare models, prefer to use the API to declare models, by endpoints or on the Playground UI (see Models configuration).

The following parameters allow you to configure the Playground application. The configuration file can be shared with the API, as the sections are identical and compatible. Some parameters are common to both the API and the Playground (for example, app_title).

For Plagroud deployment, some environment variables are required to be set, like Reflex backend URL. See Environment variables for more information.

Attribute ScopeTypeDescriptionDefault Values Examples
dependenciesAPIPlayground Dependencies required by the applications (API and Playground). For details of configuration, see the Dependencies section.required
models API arrayModels used by the API. For details of configuration, see the Model section. required
settings APIPlayground General settings configuration fields. For details of configuration, see the Settings section. required

Model

Section titled “Model”

In the model section, you define a list of models (routers and providers). These models are only used for the initial bootstrap of the API. The model section of the configuration is ignored if any models are already registered in the database.

Attribute ScopeTypeDescriptionDefault Values Examples
aliases APIarray Aliases of the model. It will be used to identify the model by users. [] [‘model-alias’, ‘model-alias-2’]
cost_completion_tokens APInumberModel costs completion tokens for user budget computation. The cost is by 1M tokens. Set to 0.0 to disable budget computation for this model. 0.0 0.1
cost_prompt_tokens APInumberModel costs prompt tokens for user budget computation. The cost is by 1M tokens. 0.0 0.1
load_balancing_strategyAPIstringRouting strategy for load balancing between providers of the model. shuffle least_busy
name APIstringUnique name exposed to clients when selecting the model. required gpt-4o
providers APIarray API providers of the model. If there are multiple providers, the model will be load balanced between them according to the routing strategy. The different models have to the same type. For details of configuration, see the ModelProvider section.required
type APIstringType of the model. It will be used to identify the model type. requiredtext-generation

ModelProvider

Section titled “ModelProvider”
Attribute ScopeTypeDescriptionDefault Values Examples
basic_auth APInull Model provider basic authentication. For details of configuration, see the BasicAuth section. None
key APIstringnullModel provider API key. None
model_active_paramsAPIinteger Active params of the model in billions of parameters for carbon footprint computation. For more information, see https://ecologits.ai 0
model_hosting_zone APIstring Model hosting zone using ISO 3166-1 alpha-3 code format (e.g., WOR for World, FRA for France, USA for United States). This determines the electricity mix used for carbon intensity calculations. For more information, see https://ecologits.aiWOR
model_name APIstring Model name from the model provider. required
model_total_params APIinteger Total params of the model in billions of parameters for carbon footprint computation. For more information, see https://ecologits.ai 0
qos_limit APInumbernullThe value to use for the quality of service. Depends of the metric, the value can be a percentile, a threshold, etc. None
qos_metric APIstringnullThe metric to use for the quality of service policy. If not provided, no QoS policy is applied. None
timeout APIinteger Timeout for the model provider requests, after user receive an 503 error (model is too busy). 300
type APIstring Model provider type. required
url APIstringnullModel provider API url. The url must only contain the domain name (without /v1 suffix for example). Depends of the model provider type, the url can be optional (Albert, OpenAI). None
BasicAuth
Section titled “BasicAuth”
Attribute ScopeTypeDescriptionDefault Values Examples
password APIstring required
username APIstring required

Dependencies

Section titled “Dependencies”
Attribute ScopeTypeDescriptionDefault Values Examples
langfuse API nullSee the LangfuseDependency section for more information. For details of configuration, see the LangfuseDependency section. None
postgres API Postgres is a required dependency of OpenGateLLM to store API data. For details of configuration, see the PostgresDependency section. required
redis APIPlaygroundnullRedis is a required dependency for the API to store rate limiting counters and performance metrics. It is an optional dependency for the Playground to use as stage manage (see Reflex documentation). For details of configuration, see the RedisDependency section.required (API)
None (Playground)
sentry API nullSentry is an optional dependency of OpenGateLLM. Sentry helps you identify, diagnose, and fix errors in real-time. For details of configuration, see the SentryDependency section. None

LangfuseDependency

Section titled “LangfuseDependency”

Langfuse is an optional dependency of OpenGateLLM. Langfuse is used for LLM observability and tracing. In this section, you can pass all Langfuse client arguments, see https://python.reference.langfuse.com/langfuse for more information.

Attribute ScopeTypeDescriptionDefault Values Examples
base_url APIstringLangfuse server URL.http://localhost:3000 http://localhost:3000
public_key APIstringLangfuse public key.required pk-lf-…
secret_key APIstringLangfuse secret key.required sk-lf-…

PostgresDependency

Section titled “PostgresDependency”

Postgres is a required dependency of OpenGateLLM. In this section, you can pass all postgres python SDK arguments, see https://docs.sqlalchemy.org/en/21/core/engines.html#engine-creation-apihttps://docs.sqlalchemy.org/en/21/core/engines.html#engine-creation-api for more information. Only the url argument is required. The connection URL must use the asynchronous scheme, postgresql+asyncpg://. If you provide a standard postgresql:// URL, it will be automatically converted to use asyncpg.

Attribute ScopeTypeDescriptionDefault Values Examples
url APIstringPostgreSQL connection url.required postgresql+asyncpg://postgres:changeme@localhost:5432/postgres

RedisDependency

Section titled “RedisDependency”

Redis is a required dependency of OpenGateLLM. Redis is used to store rate limiting counters and performance metrics. Pass all from_url() method arguments of redis.asyncio.connection.ConnectionPool class, see https://redis.readthedocs.io/en/stable/connections.html#redis.asyncio.connection.ConnectionPool.from_url for more information.

Attribute ScopeTypeDescriptionDefault Values Examples
url APIPlaygroundstringRedis connection url.required redis://:changeme@localhost:6379

SentryDependency

Section titled “SentryDependency”

Sentry is an optional dependency of OpenGateLLM. Sentry helps you identify, diagnose, and fix errors in real-time. In this section, you can pass all sentry python SDK arguments, see https://docs.sentry.io/platforms/python/configuration/options/ for more information.

No settings.

Settings

Section titled “Settings”

General settings configuration fields.

Attribute ScopeTypeDescriptionDefault Values Examples
app_title APIPlaygroundstring The title of the application (dsiplayed on Playground, Swagger and Redoc UI). OpenGateLLM My API
audio_file_size_limit API nullintegerMaximum size of the audio file in bytes. If not provided, the audio file size limit is not applied. None 100000000
auth_bootsrap_admin_password API string Password of the admin user created at the first startup. changeme
auth_bootsrap_admin_username API string Username of the admin user created at the first startup. admin
auth_key_max_expiration_days APIPlaygroundnullintegerMaximum number of days for a new API key to be valid. None
auth_login_session_duration APIPlaygroundinteger Duration of login session for the playground in seconds. Also used as oauth2-proxy cookie expiration when SSO is enabled. 3600
auth_secret_key API stringnull Secret key for the API. It should be a random string with at least 32 characters. This key is used to encrypt user tokens, watch out if you modify the secret key, you’ll need to update all user API keys. If not provided, the master key will be used.None
disabled_routers API array Disabled routers to limits services of the API. [] [‘embeddings’]
hidden_routers API array Routers are enabled but hidden in the swagger and the documentation of the API. [] [‘admin’]
log_format API string Logging format of the API. [%(asctime)s][%(process)d:%(name)s][%(levelname)s] %(client_ip)s - %(message)s
log_level API string Logging level of the API. INFO
monitoring_postgres_enabled API boolean If true, the log usage will be written in the PostgreSQL database. True
monitoring_prometheus_enabled API boolean If true, Prometheus metrics will be exposed in the /metrics endpoint. True
playground_default_model Playground stringnull The first model selected in chat page. None
playground_disabled_pages Playground array List of pages to disable from the navigation bar. required
playground_documentation_url Playground stringnull Documentation URL. If not provided, deactivated documentation link in the navigation bar. https://docs.opengatellm.org
playground_opengatellm_timeout Playground integer The timeout in seconds for the OpenGateLLM API. 60
playground_opengatellm_url Playground string The URL of the OpenGateLLM API. http://localhost:8000
playground_reference_url Playground stringnull Reference URL. If not provided, deactivated reference link in the navigation bar. http://localhost:8000/redoc
playground_sso_access_denied_documentation_urlPlayground stringnull URL displayed in the access denied page when SSO access is denied. If not provided, use the documentation URL. None
playground_swagger_url Playground stringnull Swagger URL. If not provided, deactivated swagger link in the navigation bar. http://localhost:8000/docs
playground_theme_accent_color Playground string The primary color used for default buttons, typography, backgrounds, etc. See available colors at https://www.radix-ui.com/colors. purple
playground_theme_appearance Playground string The appearance of the theme. light
playground_theme_gray_color Playground string The secondary color used for default buttons, typography, backgrounds, etc. See available colors at https://www.radix-ui.com/colors. gray
playground_theme_has_background Playground boolean Whether the theme has a background. True
playground_theme_panel_background Playground string Whether panel backgrounds are translucent: ‘solid’ | ‘translucent’. solid
playground_theme_radius Playground string The radius of the theme. Can be ‘small’, ‘medium’, or ‘large’. medium
playground_theme_scaling Playground string The scaling of the theme. 100%
rate_limiting_strategy API string Rate limiting strategy for the API. fixed_window
routing_max_priority APIPlaygroundinteger Maximum allowed priority in routing tasks. 4
routing_max_retries API integer Maximum number of retries for routing tasks. 3
routing_retry_countdown API integer Number of seconds before retrying a failed routing task. 3
swagger_contact API nullobject Contact informations of the API in swagger UI, see https://fastapi.tiangolo.com/tutorial/metadata for more information. None
swagger_description API string Display description of your API in swagger UI, see https://fastapi.tiangolo.com/tutorial/metadata for more information. See documentation See documentation
swagger_docs_url API string Docs URL of swagger UI, see https://fastapi.tiangolo.com/tutorial/metadata for more information. /docs
swagger_license_info API object Licence informations of the API in swagger UI, see https://fastapi.tiangolo.com/tutorial/metadata for more information. {‘name’: ‘MIT Licence’, ‘identifier’: ‘MIT’, ‘url’: ‘https://raw.githubusercontent.com/etalab-ia/opengatellm/refs/heads/main/LICENSE'\}
swagger_openapi_tags API array OpenAPI tags of the API in swagger UI, see https://fastapi.tiangolo.com/tutorial/metadata for more information. []
swagger_openapi_url API string OpenAPI URL of swagger UI, see https://fastapi.tiangolo.com/tutorial/metadata for more information. /openapi.json
swagger_redoc_url API string Redoc URL of swagger UI, see https://fastapi.tiangolo.com/tutorial/metadata for more information. /redoc
swagger_summary API string Display summary of your API in swagger UI, see https://fastapi.tiangolo.com/tutorial/metadata for more information. OpenGateLLM connect to your models. You can configuration this swagger UI in the configuration file, like hide routes or change the title. My API description.
swagger_terms_of_service API stringnull A URL to the Terms of Service for the API in swagger UI. If provided, this has to be a URL. None https://example.com/terms-of-service
swagger_version API string Display version of your API in swagger UI, see https://fastapi.tiangolo.com/tutorial/metadata for more information. latest 2.5.0
usage_tokenizer API string Tokenizer used to compute usage of the API. tiktoken_gpt2

Authentication specific fields

Section titled “Authentication specific fields”
Attribute ScopeTypeDescriptionDefault Values Examples
auth_login_typeAPIPlaygroundstringLogin type for the API.password