Quickstart

Prerequisites

• Docker and Docker Compose
• An access to a LLM API, eg. OpenAI, vLLM, Ollama… (see supported model API)

1. Clone the repository

   git clone https://github.com/etalab-ia/OpenGateLLM.git

2. Make quickstart

   Run OpenGateLLM quickstart with the following command:

   make quickstart

   

   The services are running and available at the following URLs:

   • API – http://localhost:8000/docs
   • Playground – http://localhost:8501

   Quickstart command

   This command will launch the OpenGateLLM API and Playground, and their required dependencies, Postgres and Redis, in Docker containers. Moreover, it will copy these three files if they don’t already exist:

   • config.example.yml → config.yml
   • env.example → .env
   • compose.example.yml → compose.yml

   The bootstrap admin user is created and you can connect to the Playground with the following credentials:

   • email: admin
   • password: changeme

   This user is created at the first startup into a new role called bootstrap_admin with the permission ADMIN. We recommend deleting it after you have created your own admin account. See configuration file documentation for more information about the bootstrap admin user.

3. Setup models

   Now, you can setup your models to expose them to your users.

   Setup your models