llmadapter

Getting Started

This guide gets you from a fresh checkout to a working CLI inference call, config inspection, gateway, and Docker run.

1. Verify The Repo

env GOCACHE=/tmp/go-cache go test ./...
env GOCACHE=/tmp/go-cache go vet ./...
env GOCACHE=/tmp/go-cache GOMODCACHE=/tmp/go-mod-cache go build ./...

These checks do not require provider credentials.

2. Add Credentials

Set at least one provider credential:

export ANTHROPIC_API_KEY=...
export OPENAI_API_KEY=...
export OPENROUTER_API_KEY=...
export MINIMAX_API_KEY=...
# Bedrock Mantle Responses/Messages:
export BEDROCK_API_KEY=...
# Bedrock Converse and SDK-backed Bedrock auth:
export AWS_PROFILE=...
export AWS_REGION=us-east-1

Aliases also supported:

export OPENAI_KEY=...
export OPENROUTER_KEY=...
export MINIMAX_KEY=...

Local OAuth providers:

Inspect what llmadapter sees:

go run ./cmd/llmadapter providers --auto

The output reports provider endpoint status without printing secrets.

3. Run CLI Inference

Use the auto-detected mux client:

go run ./cmd/llmadapter infer -m anthropic/claude-haiku-4-5-20251001 "reply with one short sentence"

The command prints route/model resolution first, streams reasoning/text when available, then prints usage and cost data when providers report it.

Resolve without calling a provider:

go run ./cmd/llmadapter resolve anthropic/claude-haiku-4-5-20251001
go run ./cmd/llmadapter resolve openai/gpt-5.5

4. Inspect The Example Config

The repository ships a load-tested config:

go run ./cmd/llmadapter serve --config examples/llmadapter.example.json --inspect-config

This validates routing, provider endpoint metadata, modeldb overlays, aliases, capability provenance, and pricing availability without constructing live provider clients.

Useful inspection commands:

go run ./cmd/llmadapter routes --config examples/llmadapter.example.json
go run ./cmd/llmadapter models --config examples/llmadapter.example.json
go run ./cmd/llmadapter resolve --config examples/llmadapter.example.json example-fast

5. Run The Gateway

Start the gateway:

go run ./cmd/llmadapter serve --config examples/llmadapter.example.json

Call the OpenAI Responses-compatible endpoint:

curl -sS http://localhost:8080/v1/responses \
  -H 'content-type: application/json' \
  -d '{
    "model": "example-fast",
    "input": "Reply with exactly: llmadapter ok"
  }'

The same gateway also exposes:

6. Run In Docker

Build:

docker build -t llmadapter:local .

Run with the example config:

docker run --rm -p 8080:8080 -w /app \
  -v "$PWD/examples:/app/examples:ro" \
  -e ANTHROPIC_API_KEY -e OPENAI_API_KEY -e OPENROUTER_API_KEY \
  llmadapter:local serve --config examples/llmadapter.example.json

Run with your own config:

docker run --rm -p 8080:8080 \
  -v "$PWD/llmadapter.json:/etc/llmadapter/config.json:ro" \
  -e LLMADAPTER_CONFIG=/etc/llmadapter/config.json \
  -e ANTHROPIC_API_KEY \
  llmadapter:local

7. Run Live Smoke Tests

Run the full live matrix:

env GOCACHE=/tmp/go-cache TEST_INTEGRATION=1 go test ./tests/e2e -count=1 -v

Run focused slices:

env GOCACHE=/tmp/go-cache TEST_INTEGRATION=1 go test ./tests/e2e -run TestSmokeTextStream -count=1 -v
env GOCACHE=/tmp/go-cache TEST_INTEGRATION=1 go test ./tests/e2e -run TestSmokeToolUse -count=1 -v
env GOCACHE=/tmp/go-cache TEST_INTEGRATION=1 go test ./tests/e2e -run TestSmokeOpenAIResponsesWebSocket -count=1 -v
env GOCACHE=/tmp/go-cache TEST_INTEGRATION=1 go test ./tests/e2e -run 'TestSmokeCodexWebSocket(Continuation|PromptCache)' -count=1 -v
env GOCACHE=/tmp/go-cache TEST_INTEGRATION=1 go test ./tests/e2e -run TestGatewaySmoke -count=1 -v

Tests skip cleanly when credentials are missing or a provider endpoint does not advertise a feature.

Next Steps