Skip to content

Redis

OpenGateLLM uses Redis as an in-memory data store. It has two main purposes:

  • Rate limiting: Tracks API usage limits (RPM, RPD, TPM, TPD)
  • Inflight gauges: Counts concurrent requests per provider (INCR / DECR) so least_busy load balancing can pick the least loaded provider

To configure the Redis dependency, you need to add the connection information in the dependencies section of your config.yml file.
Check Redis section in configuration file documentation for more information.

Example:

dependencies:
[...]
redis:
url: redis://:${REDIS_PASSWORD:-changeme}@${REDIS_HOST:-localhost}:${REDIS_PORT:-6379}

The Redis dependency accepts all parameters from the from_url() method of redis.asyncio.connection.ConnectionPool class. Only url parameter is required.

Configuration file documentation