Redis
OpenGateLLM uses Redis as an in-memory data store. It has two main purposes:
- Rate limiting: Tracks API usage limits (RPM, RPD, TPM, TPD)
- Inflight gauges: Counts concurrent requests per provider (
INCR/DECR) soleast_busyload balancing can pick the least loaded provider
Configuration
Section titled “Configuration”To configure the Redis dependency, you need to add the connection information in the dependencies section of your config.yml file.
Check Redis section in configuration file documentation for more information.
Example:
dependencies: [...] redis: url: redis://:${REDIS_PASSWORD:-changeme}@${REDIS_HOST:-localhost}:${REDIS_PORT:-6379}The Redis dependency accepts all parameters from the from_url() method of redis.asyncio.connection.ConnectionPool class. Only url parameter is required.