Redis

Required dependency

Redis is a required dependency for OpenGateLLM to function.

OpenGateLLM uses Redis as an in-memory data store. It has two main purposes:

• Rate limiting: Tracks API usage limits (RPM, RPD, TPM, TPD)
• Performance metrics: Stores time-series latency and TTFT metrics for dynamic routing and request prioritisation (work in progress, check our roadmap for more information)

Configuration

To configure the Redis dependency, you need to add the connection information in the dependencies section of your config.yml file.
Check Redis section in configuration file documentation for more information.

Example:

dependencies:
    [...]
    redis:
        url: redis://:${REDIS_PASSWORD:-changeme}@${REDIS_HOST:-localhost}:${REDIS_PORT:-6379}

The Redis dependency accepts all parameters from the from_url() method of redis.asyncio.connection.ConnectionPool class. Only url parameter is required.

Redis Stack Server required

Redis Stack Server is required (not standard Redis) because OpenGateLLM uses the RedisTimeSeries module for performance metrics.

Configuration file documentation