Configuration file

OpenGateLLM requires configuring a configuration file. This defines models, dependencies, and settings parameters. Playground and API can share the same configuration file; the Scope column in each table indicates whether a field applies to the API, the Playground, or both.

By default, the configuration file must be ./config.yml file.

You can change the configuration file by setting the CONFIG_FILE environment variable.

Secrets

You can pass environment variables in configuration file with pattern ${ENV_VARIABLE_NAME}. All environment variables will be loaded in the configuration file.

Example

models:
  [...]
  - name: my-language-model
    type: text-generation
    providers:
      - type: openai
        url: https://api.openai.com
        key: ${OPENAI_API_KEY}
        model_name: gpt-4o-mini

Example

The following is an example of configuration file:

# -------------------------------- dependencies ---------------------------------
dependencies:
  postgres: # required
    url: postgresql+asyncpg://${POSTGRES_USER:-postgres}:${POSTGRES_PASSWORD:-changeme}@${POSTGRES_HOST:-postgres}:${POSTGRES_PORT:-5432}/postgres
    echo: False
    pool_size: 5
    connect_args:
      server_settings:
        statement_timeout: "120s"
      command_timeout: 60

  redis: # required
    url: redis://:${REDIS_PASSWORD:-changeme}@${REDIS_HOST:-redis}:${REDIS_PORT:-6379}
    max_connections: 200
    socket_connect_timeout: 5
    retry_on_timeout: True
    health_check_interval: 30
    decode_responses: False
    socket_keepalive: True

  # sentry:
  #   dsn: ${SENTRY_DSN}

  # langfuse:
  #   public_key: ${LANGFUSE_PUBLIC_KEY}
  #   secret_key: ${LANGFUSE_SECRET_KEY}
  #   base_url: http://localhost:3000

# ---------------------------------- settings -----------------------------------
settings:
  # disabled_routers: ["admin", "audio"]
  # hidden_routers: ["auth"]
  # usage_tokenizer: tiktoken_gpt2
  # app_title: My OpenGateLLM API

  # log_level: INFO
  # log_format: [%(asctime)s][%(process)d:%(name)s][%(levelname)s] %(client_ip)s - %(message)s

  swagger_version: 0.5.0
  # swagger_contact_url: https://github.com/etalab-ia/OpenGateLLM
  # swagger_contact_email: john.doe@example.com
  # swagger_docs_url: /docs
  # swagger_redoc_url: /redoc

  auth_secret_key: changeme
  auth_bootsrap_admin_username: admin
  auth_bootsrap_admin_password: changeme

  # rate_limiting_strategy: fixed_window

  # monitoring_sentry_enabled: True
  # monitoring_postgres_enabled: True
  # monitoring_prometheus_enabled: True

  playground_opengatellm_url: ${OPENGATELLM_URL:-http://api:8000}
  # playground_opengatellm_timeout: 60
  # playground_disabled_pages: []
  # playground_default_model: my-model
  # playground_theme_has_background: True
  # playground_theme_accent_color: purple
  # playground_theme_appearance: dark
  # playground_theme_gray_color: gray
  # playground_theme_panel_background: solid
  # playground_theme_radius: medium
  # playground_theme_scaling: 100%
  # playground_swagger_url: http://localhost:8000/swagger
  # playground_reference_url: http://localhost:8000/redoc
  # playground_documentation_url: https://docs.opengatellm.org

# ----------------------------------- models ------------------------------------
# models:
#   - name: albert-testbed
#     type: text-generation
#     # aliases: ["model-alias"]
#     # owned_by: Me
#     # load_balancing_strategy: shuffle
#     # cost_prompt_tokens: 0.10
#     # cost_completion_tokens: 0.10
#     providers:
#       - type: vllm
#         url: http://albert-testbed.etalab.gouv.fr:8000
#         # key: sk-xxx
#         model_name: "gemma3:1b"
#         # timeout: 60
#         # model_hosting_zone: FRA
#         # model_total_params: 8
#         # model_active_params: 8

Configuration

Configuration file is composed of 3 sections, models:

models: to declare models API exposed to the API.
dependencies: to declare both required plugins for the API (e.g. PostgreSQL, Redis) and optional ones (e.g. Sentry).
settings: to configure the API.

We don’t recommend to use the configuration file to declare models, prefer to use the API to declare models, by endpoints or on the Playground UI (see Models configuration).

The following parameters allow you to configure the Playground application. The configuration file can be shared with the API, as the sections are identical and compatible. Some parameters are common to both the API and the Playground (for example, app_title).

For Plagroud deployment, some environment variables are required to be set, like Reflex backend URL. See Environment variables for more information.

Attribute	Scope	Type	Description	Default
dependencies	APIPlayground		API: Dependencies used by the API. For details of configuration, see the Dependencies section. Playground: Dependencies used by the playground. For details of configuration, see the Dependencies section.	required
models	API	array	Models used by the API. For details of configuration, see the Model section.	required
settings	APIPlayground		API: For details of configuration, see the Settings section. Playground: General settings configuration fields. Some fields are common to the API and the playground. For details of configuration, see the Settings section.	required

Model

In the model section, you define a list of models (routers and providers). These models are only used for the initial bootstrap of the API. The model section of the configuration is ignored if any models are already registered in the database.

Attribute	Scope	Type	Description	Default	Examples
aliases	API	array	Aliases of the model. It will be used to identify the model by users.	[]	[‘model-alias’, ‘model-alias-2’]
cost_completion_tokens	API	number	Model costs completion tokens for user budget computation. The cost is by 1M tokens. Set to `0.0` to disable budget computation for this model.	0.0	0.1
cost_prompt_tokens	API	number	Model costs prompt tokens for user budget computation. The cost is by 1M tokens.	0.0	0.1
load_balancing_strategy	API	string	Routing strategy for load balancing between providers of the model.	shuffle	least_busy
name	API	string	Unique name exposed to clients when selecting the model.	required	gpt-4o
providers	API	array	API providers of the model. If there are multiple providers, the model will be load balanced between them according to the routing strategy. The different models have to the same type. For details of configuration, see the ModelProvider section.	required
type	API	string	Type of the model. It will be used to identify the model type.	required	text-generation

ModelProvider

Attribute	Scope	Type	Description	Default
basic_auth	API	null	Model provider basic authentication. For details of configuration, see the BasicAuth section.	None
key	API	nullstring	Model provider API key.	None
model_active_params	API	integer	Active params of the model in billions of parameters for carbon footprint computation. For more information, see https://ecologits.ai	0
model_hosting_zone	API	string	Model hosting zone using ISO 3166-1 alpha-3 code format (e.g., `WOR` for World, `FRA` for France, `USA` for United States). This determines the electricity mix used for carbon intensity calculations. For more information, see https://ecologits.ai	WOR
model_name	API	string	Model name from the model provider.	required
model_total_params	API	integer	Total params of the model in billions of parameters for carbon footprint computation. For more information, see https://ecologits.ai	0
qos_limit	API	nullnumber	The value to use for the quality of service. Depends of the metric, the value can be a percentile, a threshold, etc.	None
qos_metric	API	nullstring	The metric to use for the quality of service policy. If not provided, no QoS policy is applied.	None
timeout	API	integer	Timeout for the model provider requests, after user receive an 503 error (model is too busy).	300
type	API	string	Model provider type.	required
url	API	nullstring	Model provider API url. The url must only contain the domain name (without `/v1` suffix for example). Depends of the model provider type, the url can be optional (Albert, OpenAI).	None

BasicAuth

Attribute	Scope	Type	Description	Default	Values	Examples
password	API	string		required
username	API	string		required

Dependencies

Attribute	Scope	Type	Description	Default
langfuse	API	null	See the LangfuseDependency section for more information. For details of configuration, see the LangfuseDependency section.	None
postgres	API		Postgres is a required dependency of OpenGateLLM to store API data. For details of configuration, see the PostgresDependency section.	required
redis	APIPlayground	null	API: Redis is a required dependency of OpenGateLLM to store rate limiting counters and performance metrics. For details of configuration, see the RedisDependency section. Playground: Set the Redis connection url to use as stage manager. See https://reflex.dev/docs/api-reference/config/ for more information. For details of configuration, see the RedisDependency section.	required (API) None (Playground)
sentry	API	null	Sentry is an optional dependency of OpenGateLLM. Sentry helps you identify, diagnose, and fix errors in real-time. For details of configuration, see the SentryDependency section.	None

LangfuseDependency

Langfuse is an optional dependency of OpenGateLLM. Langfuse is used for LLM observability and tracing. In this section, you can pass all Langfuse client arguments, see https://python.reference.langfuse.com/langfuse for more information.

Attribute	Scope	Type	Description	Default	Examples
base_url	API	string	Langfuse server URL.	http://localhost:3000	http://localhost:3000
public_key	API	string	Langfuse public key.	required	pk-lf-…
secret_key	API	string	Langfuse secret key.	required	sk-lf-…

PostgresDependency

Postgres is a required dependency of OpenGateLLM. In this section, you can pass all postgres python SDK arguments, see https://docs.sqlalchemy.org/en/21/core/engines.html#engine-creation-apihttps://docs.sqlalchemy.org/en/21/core/engines.html#engine-creation-api for more information. Only the url argument is required. The connection URL must use the asynchronous scheme, postgresql+asyncpg://. If you provide a standard postgresql:// URL, it will be automatically converted to use asyncpg.

Attribute	Scope	Type	Description	Default	Values	Examples
url	API	string	PostgreSQL connection url.	required		postgresql+asyncpg://postgres:changeme@localhost:5432/postgres

RedisDependency

Redis is a required dependency of OpenGateLLM. Redis is used to store rate limiting counters and performance metrics. Pass all from_url() method arguments of redis.asyncio.connection.ConnectionPool class, see https://redis.readthedocs.io/en/stable/connections.html#redis.asyncio.connection.ConnectionPool.from_url for more information.

Attribute	Scope	Type	Description	Default	Values	Examples
url	APIPlayground	string	Redis connection url.	required		redis://:changeme@localhost:6379

SentryDependency

Sentry is an optional dependency of OpenGateLLM. Sentry helps you identify, diagnose, and fix errors in real-time. In this section, you can pass all sentry python SDK arguments, see https://docs.sentry.io/platforms/python/configuration/options/ for more information.

No settings.

Settings

General settings configuration fields.

Attribute	Scope	Type	Description	Default	Examples
app_title	APIPlayground	string	API: Display title of your API in swagger UI, see https://fastapi.tiangolo.com/tutorial/metadata for more information. Playground: The title of the application.	OpenGateLLM	My API
auth_bootsrap_admin_password	API	string	Password of the admin user created at the first startup.	changeme
auth_bootsrap_admin_username	API	string	Username of the admin user created at the first startup.	admin
auth_key_max_expiration_days	APIPlayground	nullinteger	API: Maximum number of days for a new API key to be valid. Playground: Maximum number of days for a token to be valid.	None
auth_login_session_duration	API	integer	Duration of the playground postgres_session in seconds.	3600
auth_secret_key	API	nullstring	Secret key for the API. It should be a random string with at least 32 characters. This key is used to encrypt user tokens, watch out if you modify the secret key, you’ll need to update all user API keys. If not provided, the master key will be used.	None
disabled_routers	API	array	Disabled routers to limits services of the API.	[]	[‘embeddings’]
hidden_routers	API	array	Routers are enabled but hidden in the swagger and the documentation of the API.	[]	[‘admin’]
log_format	API	string	Logging format of the API.	[%(asctime)s][%(process)d:%(name)s][%(levelname)s] %(client_ip)s - %(message)s
log_level	API	string	Logging level of the API.	INFO
monitoring_postgres_enabled	API	boolean	If true, the log usage will be written in the PostgreSQL database.	True
monitoring_prometheus_enabled	API	boolean	If true, Prometheus metrics will be exposed in the `/metrics` endpoint.	True
playground_default_model	Playground	nullstring	The first model selected in chat page.	None
playground_disabled_pages	Playground	array	List of pages to disable from the navigation bar.	required
playground_documentation_url	Playground	nullstring	Documentation URL. If not provided, deactivated documentation link in the navigation bar.	https://docs.opengatellm.org
playground_opengatellm_timeout	Playground	integer	The timeout in seconds for the OpenGateLLM API.	60
playground_opengatellm_url	Playground	string	The URL of the OpenGateLLM API.	http://localhost:8000
playground_reference_url	Playground	nullstring	Reference URL. If not provided, deactivated reference link in the navigation bar.	http://localhost:8000/redoc
playground_swagger_url	Playground	nullstring	Swagger URL. If not provided, deactivated swagger link in the navigation bar.	http://localhost:8000/docs
playground_theme_accent_color	Playground	string	The primary color used for default buttons, typography, backgrounds, etc. See available colors at https://www.radix-ui.com/colors.	purple
playground_theme_appearance	Playground	string	The appearance of the theme.	light
playground_theme_gray_color	Playground	string	The secondary color used for default buttons, typography, backgrounds, etc. See available colors at https://www.radix-ui.com/colors.	gray
playground_theme_has_background	Playground	boolean	Whether the theme has a background.	True
playground_theme_panel_background	Playground	string	Whether panel backgrounds are translucent: ‘solid’ \| ‘translucent’.	solid
playground_theme_radius	Playground	string	The radius of the theme. Can be ‘small’, ‘medium’, or ‘large’.	medium
playground_theme_scaling	Playground	string	The scaling of the theme.	100%
rate_limiting_strategy	API	string	Rate limiting strategy for the API.	fixed_window
routing_max_priority	APIPlayground	integer	Maximum allowed priority in routing tasks.	4 (API) 10 (Playground)
routing_max_retries	API	integer	Maximum number of retries for routing tasks.	3
routing_retry_countdown	API	integer	Number of seconds before retrying a failed routing task.	3
swagger_contact	API	nullobject	Contact informations of the API in swagger UI, see https://fastapi.tiangolo.com/tutorial/metadata for more information.	None
swagger_description	API	string	Display description of your API in swagger UI, see https://fastapi.tiangolo.com/tutorial/metadata for more information.	See documentation	See documentation
swagger_docs_url	API	string	Docs URL of swagger UI, see https://fastapi.tiangolo.com/tutorial/metadata for more information.	/docs
swagger_license_info	API	object	Licence informations of the API in swagger UI, see https://fastapi.tiangolo.com/tutorial/metadata for more information.	{‘name’: ‘MIT Licence’, ‘identifier’: ‘MIT’, ‘url’: ‘https://raw.githubusercontent.com/etalab-ia/opengatellm/refs/heads/main/LICENSE'\}
swagger_openapi_tags	API	array	OpenAPI tags of the API in swagger UI, see https://fastapi.tiangolo.com/tutorial/metadata for more information.	[]
swagger_openapi_url	API	string	OpenAPI URL of swagger UI, see https://fastapi.tiangolo.com/tutorial/metadata for more information.	/openapi.json
swagger_redoc_url	API	string	Redoc URL of swagger UI, see https://fastapi.tiangolo.com/tutorial/metadata for more information.	/redoc
swagger_summary	API	string	Display summary of your API in swagger UI, see https://fastapi.tiangolo.com/tutorial/metadata for more information.	OpenGateLLM connect to your models. You can configuration this swagger UI in the configuration file, like hide routes or change the title.	My API description.
swagger_terms_of_service	API	nullstring	A URL to the Terms of Service for the API in swagger UI. If provided, this has to be a URL.	None	https://example.com/terms-of-service
swagger_version	API	string	Display version of your API in swagger UI, see https://fastapi.tiangolo.com/tutorial/metadata for more information.	latest	2.5.0
usage_tokenizer	API	string	Tokenizer used to compute usage of the API.	tiktoken_gpt2