Run Claude Code with open-source models: setup with environment variables

Pick the model

For a coding agent you want strong reasoning and good tool handling: glm-5.3-nvfp4 or qwen-3.8-27b are good choices. For light tasks and fast replies, qwen-3.5-9b.

Tools (function calling) are what the agent uses to read and edit your files and run commands, so without them it cannot do its job. The catalogue marks which models have them; pick one that does.

This applies if you are configuring it on your machine, the next step. If you prefer a ready-made Sandbox, the template fixes the model and we cover that there.

Configure it on your machine

Claude Code reads the base URL, the key and the models from environment variables. Put the model from the previous step in GPUFLOW_MODEL, and the key you created in the dashboard in ANTHROPIC_AUTH_TOKEN: that way it travels as Authorization: Bearer and Claude Code does not ask you to confirm it at startup.

The other lines prevent two surprises. Claude Code does not use a single model: subagents and background tasks (titles, summaries) use the opus, sonnet and haiku aliases, and unless you point them at your model they will ask for Anthropic models that Token Factory does not serve. And in auto mode, Claude Code asks the server to review each action, which only the Anthropic API does: the two auto mode variables have your model review it and remove the session-not-eligible warning.

export GPUFLOW_MODEL="glm-5.3-nvfp4"

export ANTHROPIC_BASE_URL="https://api.gpuflow.ai"
export ANTHROPIC_AUTH_TOKEN="$GPUFLOW_API_KEY"
export ANTHROPIC_MODEL="$GPUFLOW_MODEL"
export ANTHROPIC_DEFAULT_OPUS_MODEL="$GPUFLOW_MODEL"
export ANTHROPIC_DEFAULT_SONNET_MODEL="$GPUFLOW_MODEL"
export ANTHROPIC_DEFAULT_HAIKU_MODEL="$GPUFLOW_MODEL"
export CLAUDE_CODE_AUTO_MODE_MODEL="$GPUFLOW_MODEL"
export CLAUDE_CODE_AUTO_MODE_SERVER=0

claude

Or boot it ready-made in a Sandbox

If you create a Sandbox with one of the assistant templates, the environment boots already configured against the cluster inference: the API key is injected for you. You connect over SSH and start working.

Mind the model, because you do not pick it the same way here. The Claude Code template runs on your own Anthropic account or on our tokens with GLM 5.3 NVFP4, its fixed model. The one that lets you choose any model in the catalogue is Open Code.

It is still usually the cleanest path: you do not manage keys by hand.

Check it works

Run a simple task ("summarise this repo" or "add a test to this function") and verify in the dashboard that token usage shows up under your account. If you see spend, Claude Code is talking to Token Factory.