Quickstart

Prerequisites

• Docker and Docker Compose
• An access to a LLM API, eg. OpenAI, vLLM, Ollama… (see supported model API)

1. Clone the repository

   git clone https://github.com/etalab-ia/OpenGateLLM.git

2. Make quickstart

   Run OpenGateLLM quickstart with the following command:

   make quickstart

   The services are running and available at the following URLs:

   • API – http://localhost:8000/docs
   • Playground – http://localhost:8501

   Quickstart command

   This command will launch the OpenGateLLM API and Playground, and their required dependencies, Postgres and Redis, in Docker containers. Moreover, it will copy these three files if they don’t already exist:

   • config.example.yml → config.yml
   • env.example → .env
   • compose.example.yml → compose.yml

3. Create an admin role and your first admin user

   Run the following command to create the default admin role and admin user:

   make create-admin

   Connect to the Playground

   Open the Playground at http://localhost:8501 and sign in with the default master account:

   • email: master
   • password: changeme

   You can modify the created admin role and user on Playground UI or use the API Reference for role and user management details (endpoints /v1/admin/roles and /v1/admin/users).

4. Setup models

   Now, you can setup your models to expose them to your users.

   Setup your models