Jupyter AI¶
Data scientists ask a model about the data in front of them, which is often the data you most need to protect. Point
Jupyter AI at the gateway and every chat message and %%ai cell is checked against your policies, charged to the right
team and recorded in Request logs, with Claude and Gemini as reachable as OpenAI's models.
What it is¶
Jupyter AI adds an AI chat (Jupyternaut) and the %%ai magic commands to JupyterLab. It talks to models in the OpenAI
format, which the gateway accepts at /v1. Its settings changed between version 2 and version 3: both are below. Run
pip show jupyter-ai to see yours.
The two settings¶
The simplest way, for both versions: set two environment variables where the Jupyter server starts (on JupyterHub, in each user's server environment):
| Setting | Environment variable | Value |
|---|---|---|
| Base address | OPENAI_BASE_URL |
https://<your-gateway>/v1 |
| Key | OPENAI_API_KEY |
Your Vidai key (starts vk_), never a provider's key |
Or set them in Jupyter AI itself:
Open the Settings tab, then Jupyternaut Settings:
- Chat model:
openai/followed by a model name, for exampleopenai/gpt-4o-mini. Theopenai/part means "speak the OpenAI format"; the name after it is any name your organisation has approved. - Under the API keys, set
OPENAI_API_KEYto your Vidai key. - Under the model parameters, choose + Add Parameter:
api_base=https://<your-gateway>/v1.
For the magics, install jupyter-ai-magic-commands and pass the address on the cell:
%load_ext jupyter_ai_magic_commands
%%ai openai/gpt-4o-mini --api-base https://<your-gateway>/v1
Explain a left join in one sentence.
Open the chat panel's settings:
- Language model: OpenAI (general interface), then type the model name in Model ID, for example
gpt-4o-mini. - Base API URL (optional):
https://<your-gateway>/v1. - OpenAI API key (
OPENAI_API_KEY): your Vidai key.
Or in jupyter_jupyter_ai_config.py:
c.AiExtension.model_parameters = {
"openai-chat-custom:gpt-4o-mini": {"openai_api_base": "https://<your-gateway>/v1"},
}
The magics (%load_ext jupyter_ai_magics, then %%ai openai-chat-custom:gpt-4o-mini) read the environment
variables above.
In the 2.x settings, the key goes under API Keys, in the field labelled OPENAI_API_KEY.
The gateway's address, and the base URL for each kind of client, are under Connect your app on the Keys page and in System › Settings › Gateway. You get the key from Keys › Issue a key; its secret is shown once.
A minimal call¶
Run a cell:
%%ai openai/gpt-4o-mini --api-base https://<your-gateway>/v1
Reply with exactly: ok
(%%ai openai-chat-custom:gpt-4o-mini in version 2.) Or ask Jupyternaut in the chat panel.
The same settings call Claude or Gemini: use a name such as openai/claude-sonnet-4.5 (version 3) or Model ID
claude-sonnet-4.5 (version 2). Jupyter AI sends the OpenAI format; the gateway translates it for that provider and
the answer back.
Streaming¶
Jupyternaut streams its answers in the chat panel; a stream from Claude or Gemini arrives in the OpenAI format it expects. Nothing to set.
Tool calls¶
Jupyternaut (version 3) can read and edit notebooks through tools. Those tool calls travel inside the model calls, which go through the gateway on every provider it translates to; the tools themselves run in your Jupyter server. In Request logs, the Tool filter under More filters finds the calls that offered or called a tool by its name.
Reading what the gateway did¶
Jupyter AI doesn't show response headers. Find the call in Request logs: filter by Key or Model under More filters. Each call shows the model that answered, its cost, and which policy or rule acted on it.
Errors¶
A refused call shows the gateway's message in the chat or under the cell.
| Status | Gateway code | What to do |
|---|---|---|
| 401 | invalid_api_key, missing_api_key |
Check OPENAI_API_KEY; restart the server after changing an environment variable |
| 403 | model_not_allowed, denied_by_routing_rule |
Use a model your key may call |
| 403 | content_guardrail_blocked |
A guardrail stopped the content, for example personal data in a cell |
| 404 | model_not_found |
Check the name after openai/ against Models |
| 429 | rate_limit_exceeded, token_limit_exceeded, concurrency_limit_exceeded |
Wait and run the cell again |
Model names¶
Use the names your organisation has approved, as listed under Models in the Control room. Jupyter AI's own model
lists show the provider's catalogue, not yours; type the name instead. A GET to https://<your-gateway>/v1/models
with your key lists exactly the models the key may call.
What goes through the gateway¶
| In Jupyter AI | Through the gateway |
|---|---|
Jupyternaut chat and %%ai with the OpenAI provider pointed at the gateway |
Yes, to every provider your organisation has set up |
| Other providers in Jupyter AI (Anthropic, Bedrock, Ollama…) with their own keys | No: they call their provider directly |
| Code in a cell that uses an SDK | Only if that SDK points at the gateway: see OpenAI SDK for Python |
Check it works in a minute¶
Run the minimal cell above. ok means the address and key are right. The call shows in Request logs within
seconds.
Good to know¶
- On a shared JupyterHub, give each user their own key (through their server's environment) so each person's spend shows under their name.
- Jupyter AI moves fast and its settings screens change between releases; the environment variables are the steadiest way to set it up.