Custom models & providers

Connect a model omp does not already knowCopy Anchor Link

Use a custom provider when your model runs at a private URL, behind a company gateway, or under an id that omp does not list. You can declare the exact model yourself or let omp discover the models exposed by the server.

You often do not need this file for local inference. omp automatically looks for Ollama at http://127.0.0.1:11434, llama.cpp at http://127.0.0.1:8080, and LM Studio at http://127.0.0.1:1234/v1. Start the server and open /model; if its models are already there, select one and stop.

Otherwise, create ~/.omp/agent/models.yml. This is the smallest valid configuration for an unauthenticated server that implements OpenAI Chat Completions:

providers:
  local-openai:
    baseUrl: http://127.0.0.1:8000/v1
    api: openai-completions
    auth: none
    models:
      - id: Qwen/Qwen2.5-Coder-32B-Instruct

The provider id, local-openai, is yours to choose. The model id must be the exact id accepted by the server. Use openai-responses instead if the server implements /v1/responses rather than /v1/chat/completions.

For a newly declared model, omitted metadata defaults to its id as the display name, text-only input, no configurable reasoning, a 128,000-token context window, a 16,384-token output limit, and zero cost. Those defaults make a minimal entry usable, but you should set the real limits and capabilities when they differ.

Verify and select itCopy Anchor Link

First ask omp to load the file and list just your provider:

omp models local-openai

A valid, available model appears under local-openai with its context, output, thinking, and image capabilities. A YAML or schema problem instead prints models.yml validation failed and the field that failed.

Then exercise the endpoint with an exact selector:

omp -p --model local-openai/Qwen/Qwen2.5-Coder-32B-Instruct "Reply with only OK"

Exact provider/model-id selectors avoid ambiguity when several providers expose the same model id. To make it your normal interactive model, start omp, enter /model, find the provider and model, and assign it to Default. The model hub reloads models.yml when it opens and saves the role assignment. /switch (or Alt+P) switches only the current session.

See Model roles for assigning the same model to smol, slow, plan, or another role.

Add authenticationCopy Anchor Link

providers:
  myco:
    baseUrl: https://llm.internal.example/v1
    apiKey: MYCO_LLM_API_KEY
    api: openai-responses
    models:
      - id: myco-large
        name: MyCo Large
        reasoning: true
        input: [text, image]
        contextWindow: 200000
        maxTokens: 32000
        cost:
          input: 3
          output: 15
          cacheRead: 0.3
          cacheWrite: 3.75

export MYCO_LLM_API_KEY='…'
omp models myco

auth defaults to apiKey. Use auth: none only for a genuinely unauthenticated endpoint. auth: oauth is valid only when that provider's OAuth support is already supplied by omp or an extension; models.yml cannot define a new OAuth flow.

Set authHeader: true only when a gateway specifically needs Authorization: Bearer <resolved-apiKey> injected as an ordinary header. Standard provider clients already apply their normal authentication scheme.

Discover a server's model listCopy Anchor Link

If an OpenAI-compatible server exposes GET /v1/models, discovery avoids maintaining a models list by hand:

providers:
  lab:
    baseUrl: http://127.0.0.1:8000/v1
    api: openai-completions
    auth: none
    discovery:
      type: openai-models-list

Run omp models refresh lab to force an online refresh and show only matching models. Ordinary launches use the cached catalog when it is fresh.

discovery.type accepts:

Except for proxy, discovery requires provider-level api. proxy derives a model's API from its advertised endpoint types and uses provider-level api only as a fallback. Optional discovery.timeoutMs must be a positive number of milliseconds.

Do not label a generic OpenAI-compatible server as ollama: Ollama discovery expects Ollama's native endpoints. Use openai-models-list instead.

Override a built-in providerCopy Anchor Link

A provider entry without models changes an existing provider rather than replacing its catalog. This is useful for a company gateway or for correcting metadata:

providers:
  openai:
    baseUrl: https://gateway.internal.example/v1
    apiKey: COMPANY_OPENAI_KEY
    headers:
      X-Team: coding
    modelOverrides:
      gpt-5.4:
        contextWindow: 400000

Provider headers, compat, and remoteCompaction are baselines. Model headers replace provider headers with the same name, and modelOverrides patches one bundled or discovered model. If a custom models entry uses the same provider and id as an existing model, the custom definition replaces that model's transport configuration.

File and provider referenceCopy Anchor Link

The default path is ~/.omp/agent/models.yml; ~/.omp/agent/models.yaml is used when the .yml file is absent. If neither YAML file exists and legacy models.json does, omp migrates it to models.yml. The root object accepts only providers.

Each key below providers becomes the provider portion of the selector.

A provider without models must still do something: set at least one usable override such as baseUrl, headers, apiKey, auth: none, compat, disableStrictTools, guardrailIdentifier, remoteCompaction, modelOverrides, or discovery.

API valuesCopy Anchor Link

These are the transports built into omp. An endpoint with a different wire protocol requires an extension, not another string in models.yml.

Model field referenceCopy Anchor Link

modelOverrides accepts the same metadata fields except id, api, and baseUrl. Its cost object may be partial, while a full model definition's cost must contain all four rates.

Thinking fieldsCopy Anchor Link

A new explicit thinking block uses this shape:

reasoning: true
thinking:
  mode: effort
  efforts: [low, medium, high]
  defaultLevel: medium

Legacy levels, or the pair minLevel and maxLevel, are still accepted and normalized to efforts; prefer efforts in new files.

Remote compaction fieldsCopy Anchor Link

remoteCompaction may appear at provider, model, or override level. It accepts enabled, api, endpoint, model, v2StreamingEnabled, v2Endpoint, and streamingEndpoint. Endpoint and model strings must be non-empty. Leave this block out unless your provider offers a compatible server-side compaction endpoint.

Advanced compatibility fieldsCopy Anchor Link

Most OpenAI-compatible services work without compat. Set these fields only to match documented endpoint behavior or to address a concrete request error. Every field is optional; provider values form the baseline and model values merge over them.

Request and streaming: supportsStore, supportsDeveloperRole, supportsMultipleSystemMessages, maxTokensField (max_completion_tokens or max_tokens), supportsUsageInStreaming, supportsToolChoice, supportsForcedToolChoice, alwaysSendMaxTokens, strictResponsesPairing, streamIdleTimeoutMs, cacheControlFormat (anthropic), supportsLongPromptCacheRetention, supportsImageDetailOriginal, and extraBody.

Reasoning: supportsReasoningEffort, supportsReasoningParams, reasoningEffortMap, thinkingFormat (openai, openrouter, zai, qwen, or qwen-chat-template), qwenTemplateReasoningEffort, reasoningContentField (reasoning_content, reasoning, or reasoning_text), requiresReasoningContentForToolCalls, allowsSyntheticReasoningContentForToolCalls, requiresAssistantContentForToolCalls, disableReasoningOnForcedToolChoice, disableReasoningOnToolChoice, and whenThinking. whenThinking contains another partial compatibility block applied only while thinking is active.

Tools and message history: requiresToolResultName, requiresMistralToolIds, requiresAssistantAfterToolResult, requiresThinkingAsText, supportsStrictMode, toolStrictMode (all_strict or none), and requiresToolResultId.

Streaming and Anthropic-compatible behavior: streamMarkupHealingPattern (kimi, dsml, qwen, or thinking), supportsEagerToolInputStreaming, allowAnthropicHeaderOverrides, and replayUnsignedThinking.

Gateway routing: openRouterRouting and vercelGatewayRouting, each with optional only and order string lists.

Bedrock prompt caching: promptCacheMode (none, automatic, or explicit), promptCacheMinimumTokens, promptCacheMaximumCheckpoints, and supportsLongPromptCacheRetention.

TroubleshootingCopy Anchor Link

RelatedCopy Anchor Link

Source: omp.sh — MIT, Stencil Labs. Initially imported on 2026-10-03; updates via automatic sync.