Providers and models
A provider is a vendor account. A model is one thing that vendor will run for you. Connecting the first does not automatically give you every one of the second, and assuming otherwise is the most common surprise here.
Connecting a provider
Section titled “Connecting a provider”Console, Workspace, LLM providers. You supply a credential, the gateway probes it, and the provider comes up healthy or it does not.
The probe uses one nominated model. A healthy provider means that model answered. It does not certify every model the vendor lists, so treat health as “the door opened”, not “everything inside works”.
The model catalogue
Section titled “The model catalogue”The gateway caches each vendor’s advertised model list so the console has something to show without calling out on every page load. Two consequences worth knowing:
- A model added by the vendor this morning may not appear in the list yet.
- Vendors will usually serve models that were never in the list at all, so an absent name is not proof of an unavailable model.
The catalogue is a convenience, not a contract.
Choosing a model
Section titled “Choosing a model”The temptation is to pick the largest model available and stop thinking about it. Resist it, for two reasons that have nothing to do with cost.
Bigger models are slower, and in a live chat a slow correct answer loses to a fast good one more often than anyone likes to admit. And a model that is excellent at reasoning is not automatically excellent at following a brief, which is most of what an agent actually does.
Change the model, then run your agent’s tests before publishing. Model choice is a configuration change with the blast radius of a code change.
Failure modes
Section titled “Failure modes”| Symptom | Usual cause |
|---|---|
| Bursts of failures that clear on their own | Provider throughput limit, not the model |
| Every call fails immediately | Credential rejected or revoked |
| One model fails, others fine | That model was deprecated or renamed by the vendor |
| Answers arrive but are truncated | An output token limit is set lower than the answer needs |
That last one deserves a word. An unset maximum is not the same as an unlimited one. Leave it unset and you inherit a default, and the default is smaller than you think. Long answers get guillotined mid-sentence, which reads to a customer as the agent losing its train of thought.