AI Gateway
Overview
AI Gateway is a private AI platform built on a concept close to OpenRouter, but restricted to a limited number of users chosen directly by the admin, with no open registration allowed.
Registration and Access
A user registers with an email, a password, and an activation code issued only by the admin. Each code is tied to a specific dollar balance that the admin assigns to its owner at creation time.
User Side
Dashboard
The first page the user sees after logging in, showing:
- Total tokens consumed.
- Per-model consumption: token count and number of requests for each model.
- Number of images generated.
- Total remaining dollar balance in the account.
- All information related to account usage.
API Keys
A dedicated page for creating an API key, which can then be used on any external platform that supports calling AI services.
Models
A section containing two types of models:
- Text models: 419 models.
- Image models: 2 models.
These models were gathered from different sources online after deep and extensive research, including models scraped from websites, models relying on leaked keys, and models hosted by various parties in different ways.
Docs
A section explaining to the user how to use the API key, clarifying that the connection only works through the legacy OpenAI format.
Admin Side
The admin sees everything the user sees, plus 3 additional sections:
Users
Shows all registered users on the platform.
Keys
A section for creating activation codes/keys, where the admin creates a new code and assigns it a dollar balance before handing it to the user.
Add New Model
The most important admin section, built on a JSON-based interpreter understood by an AI assistant model. The process works as follows:
The admin (or the assistant model) is sent a fixed prompt describing the role of a Connection Setup Assistant and the shape of the 6 fields in the connection panel: Model ID, Server URL, API key (optional and secret, never written directly — only where to paste it is shown), API Path, an optional short description, and finally the Connection JSON, the full declarative template for the connection.
This prompt defines the exact shape of the Connection JSON: `kind` must be `chat`, `version` equals 1, with the auth type specified (`none`, `bearer`, `header`, or `query`), request details (method, path, headers, body with the variables `{{model}}`, `{{messages}}`, `{{prompt}}`, `{{api_key}}`) and a timeout, and response details (mode: json, sse, or text) with an output path and an error path, and for streaming (SSE) a chunk path (`chunk_path`) and a done marker (`done_marker`) are added.
The assistant model asks the admin exactly two questions in order: which company or service is being connected (with examples), then the exact model name (with examples), then stops asking and reads the service's documentation or any pasted curl example on its own.
The model verifies that every path mentioned in the JSON actually exists in a real response shape, then outputs the full 6 fields with instructions for pasting each value in its place.
This method allows connecting any model or API in the world, even a private one with a fully custom response format, without writing manual code for each new model.
Settings
Includes setting up a Telegram bot connection, so the admin receives an instant alert on Telegram whenever a new user registers or whenever there is an attempted breach of the platform.
Smart Preference System: auto_model and auto_image
These are two special models that act as a routing layer on top of all the real models (419 text models and 2 image models):
- On first use, the system picks a random model from the list and tries it.
- During each trial, the system measures the model's response time and the validity of its response (whether it returned proper JSON or not), since these are leaked models that don't always follow the correct response format, being hosted inside other websites.
- If a response comes back in the wrong format, the system automatically retries and lowers that model's reliability score.
- The same logic applies to speed, so over time a cumulative ranking (similar to a vector-style scoring) builds up for each model based on how fast and consistent it is at returning correct responses.
- The natural result is that the most efficient and intelligent models are the ones with a stronger capacity to keep returning a correct format consistently, so they automatically rise to the top of the ranking over time and become the first choice used.
- Alongside this, there is a load-balancing system: each request is routed to a different model from the top-ranked group, so the load is distributed across models and tasks are executed in parallel instead of relying on a single model sequentially, which speeds up the platform's overall response time.
Storage and Architecture
Storage is fully local via SQLite. The frontend is built with HTML, CSS, and JavaScript, the server (Web Server) is written in Python, and the entire project is deployed on the developer's own private VPS.
Persistent Browser and Session Scraping
The project also includes a browser that runs persistently in the background on the server, relying on a specialized library for running a browser in a way that reduces detection and avoids protection systems and CAPTCHAs. This browser is used to scrape sessions from specific, carefully studied websites that offer free usage periods without requiring login, then these sessions are used to feed some of the models within the platform.
Problem
Accessing a large number of different AI models, especially leaked or non-uniformly hosted models, normally requires writing separate code for each model because every service has a different request and response format. These models can also return unstable or incorrectly formatted responses, making it hard to rely on them and to distribute usage across them efficiently.
Solution
AI Gateway provides a unified private gateway that lets a limited number of authorized users access hundreds of models through a single API key in the legacy OpenAI format, with an AI-assisted JSON interpreter that allows connecting any new model regardless of its response format, and a smart routing system (auto_model and auto_image) that automatically tests models, evaluates their speed and accuracy, ranks them, and distributes load across them for the best possible efficiency.
Highlights
Private Platform for a Limited Set of Users
There is no open registration; only the admin decides who the users are, by issuing an activation code tied to a specific dollar balance.
- Email + Password + Activation Code
Three-Part Registration
A user registers with an email, a password, and an activation code issued only by the admin in order to access the platform.
Complete Usage Dashboard
Shows total tokens consumed, per-model tokens and requests, number of images, and the remaining dollar balance in the account.
API Key Creation
The user creates an API key from a dedicated page to use it on any external platform that supports the legacy OpenAI format.
- 419 models
Large Number of Text Models
The platform includes 419 text models gathered from multiple sources after deep research into models available online.
- 2 models
Image Models
The platform includes two image generation models alongside the text models.
AI-Assisted JSON Connection Interpreter
An assistant AI model asks the admin only two questions (service name, then model name), then reads the service's documentation itself and builds a complete Connection JSON that links any model or API in the world, even one with a custom response format.
- kind: chat, version 1
Unified Connection JSON Template
Defines the auth type and the request and response details through fixed variables like {{model}}, {{messages}}, and {{api_key}}, supporting json, sse, or text response modes.
Admin Users Section
Shows the admin every registered user on the platform.
Admin Activation Keys Section
Lets the admin create new activation codes and assign a dollar balance to each one before handing it to a user.
Add New Model Section
Relies on the AI-assisted interpreter to connect any new model to the platform without writing manual code for each case.
Telegram Alerts for the Admin
Linking a Telegram bot in the platform's settings to send instant alerts to the admin when a new user registers or when there are attempted breaches of the platform.
auto_model and auto_image System
Two models that automatically route requests across all models, starting with a random pick then evaluating each model's speed and response format accuracy over time.
Cumulative Reliability Score for Models
When a response comes back in the wrong format, the system automatically retries and lowers that model's reliability score, while fast and correct responses gradually raise a model's ranking.
Load Balancing Across Models
Each request is routed to a different model from the top-ranked group to distribute load and execute tasks in parallel instead of sequentially.
- SQLite
Fully Local Storage
The platform relies entirely on a local SQLite database with no external storage service.
Persistent Browser for Session Scraping
A persistent background browser uses a specialized library to reduce detection and avoid CAPTCHAs, scraping sessions from carefully studied websites that offer free usage without login.
Hosted on a Private VPS
The entire project, including the web server and the persistent browser, is deployed on the developer's own private VPS.
Features
- A dashboard showing total tokens, dollar balance, image count, and per-model token and request consumption
- Over 420 AI models (419 text models and 2 image models) gathered from multiple sources through deep research, usable via a single API key in the legacy OpenAI format
- An AI-assisted JSON connection interpreter that asks the admin only two questions, then reads the documentation itself and builds a complete connection template linking any model or API in the world regardless of its response format
- A smart routing system, auto_model and auto_image, that automatically tests models, measures their speed and response accuracy, ranks them cumulatively, and balances load across models to execute requests in parallel
- A separate admin panel for managing users and dollar-balance-linked activation codes, plus adding new models, with instant Telegram alerts on registrations or attempted breaches
Tech stack
Architecture
Unified Gateway Layer Over Hundreds of Models
The platform gathers 419 text models and 2 image models from multiple sources with varied hosting setups, and exposes them all to the user through a single interface and API key in the legacy OpenAI format, so the large differences in how each model is called disappear behind one unification layer.
AI-Assisted Connection JSON Interpreter
Adding a new model relies on a fixed prompt that turns any AI service into a defined JSON template (kind, version, auth type, and request and response details with fixed variables like {{model}}, {{messages}}, and {{api_key}}), after the assistant model asks the admin only two questions and infers the rest from the service's documentation itself, verifying the validity of every extraction path.
Model Scoring and Ranking System (auto_model / auto_image)
A routing layer that starts with a random pick among models, then measures response time and response format validity on each attempt. Leaked models that return an incorrect format are automatically retried and have their reliability score lowered, while fast and consistent models gradually climb the ranking until they automatically become the first choice over time.
Load Balancing Across Models
Alongside the ranking system, each new request is distributed to a different model from the top-ranked group, allowing several requests to run in parallel instead of relying on a single model sequentially, which reduces the platform's overall response time.
Persistent Browser for Scraping Free Sessions
A browser runs persistently in the background on the same VPS, relying on a specialized library to reduce browser detection and avoid CAPTCHAs and protection systems, to scrape sessions from carefully studied websites that offer free usage without login, and feed some of the platform's models with these sessions.
Users
Private project for a limited number of users chosen by the admin