Skip to main content

Documentation Index

Fetch the complete documentation index at: https://afk.arpan.sh/llms.txt

Use this file to discover all available pages before exploring further.

Providers translate between AFK’s normalized contracts (LLMRequest/LLMResponse) and provider-specific APIs. AFK ships with three built-in providers and supports custom providers for internal deployments.

​
Built-in providers

OpenAI

Direct integration via the OpenAI Python SDK. Supports all GPT-4.1 and o-series models.

Anthropic

Direct integration via the Anthropic SDK. Supports Claude Opus 4.5 and Opus.

LiteLLM

Proxy adapter for 100+ providers (Azure, Bedrock, Gemini, Mistral, local models, etc.).

​
Capability comparison

FeatureOpenAIAnthropicLiteLLM
Text generation
Tool calling (provider-dependent)
Structured outputProvider-dependent
Streaming
Vision (image input)Provider-dependent
Custom endpoints

​
Usage

from afk.llms import LLMBuilder

# OpenAI
openai_client = LLMBuilder().provider("openai").model("gpt-4.1-mini").build()

# Anthropic
anthropic_client = LLMBuilder().provider("anthropic").model("claude-opus-4-5").build()

# LiteLLM (any provider)
gemini_client = LLMBuilder().provider("litellm").model("gemini/gemini-2.5-pro").build()

​
Custom provider

Register your own provider for unsupported inference servers:
1

Implement provider and transport contracts

from afk.llms import LLMProvider, LLMRequest, LLMResponse, LLMTransport

class MyTransport(LLMTransport):
    provider_id = "my-provider"

    async def chat(self, request: LLMRequest, *, response_model=None) -> LLMResponse:
        # Translate LLMRequest → your provider's format
        payload = self._build_payload(request)
        resp = await self._post(payload)
        return self._parse_response(resp.json())

class MyProvider(LLMProvider):
    provider_id = "my-provider"

    def create_transport(self, *, settings, middlewares=None, observers=None, provider_settings=None):
        return MyTransport()
2

Register the provider

from afk.llms import register_llm_provider

register_llm_provider(MyProvider())
3

Use it

client = LLMBuilder().provider("my-provider").model("my-model").build()

agent = Agent(name="demo", model=client, instructions="...")

​
Custom transport

Use provider-specific settings for custom endpoints, proxy URLs, or credentials: 
client = (
    LLMBuilder()
    .provider("openai")
    .model("gpt-4.1-mini")
    .with_provider_settings("openai", {"base_url": "https://proxy.example.com/v1"})
    .build()
)

​
Next steps

Control & Session

Retry, caching, rate limiting, and circuit breaking.

Agent Integration

How agents resolve and use LLM clients.