← Blog
Person wearing glasses at a desk with code on one monitor and a forest scene on another.
Published on
2025-10-06

A buyer’s guide to OpenAI‑compatible APIs

An OpenAI-compatible API implements a documented subset of OpenAI-style request and response formats, allowing clients such as the OpenAI SDKs to call an alternative model endpoint. Compatibility applies to the interface; it does not guarantee the same model behavior or support for every OpenAI feature.

This buyer’s guide is for developers and platform engineers choosing between managed endpoints, self-hosted inference, and a routing proxy. It covers operating responsibilities, API checks, and connection examples so you can assess migration effort before you replace an existing provider.

Is an OpenAI-compatible API a drop-in replacement?

Only for the routes and features your application has verified. Changing the base URL, API key, and model value may be enough for a basic Chat Completions call. Tools, structured output, streaming, embedding requests, and error handling need separate checks. A shared JSON format does not ensure identical answers, latency, or costs.

Start by deciding who will operate the serving stack. Hivenet Inference Service provides managed, dedicated model endpoints. Compute with Hivenet provides GPU or CPU infrastructure where you install and operate vLLM or another serving stack yourself. If you operate several inference backends, Hivenet Router can provide a shared routing layer on infrastructure you choose.

What portability looks like in existing clients

A compatible interface can reduce dependence on one provider’s client integration, provided your application uses the shared subset of features. Keep provider-specific settings separate and test a second destination before relying on that portability.

For example, Google’s OpenAI compatibility guide shows Gemini access through the OpenAI libraries by changing three values: the API key to a Gemini API key, the base URL to Google’s endpoint, and the model to a supported Gemini model. It also documents streaming responses. Those are Gemini-specific capabilities; they do not establish what a Hivenet endpoint or another provider supports.

Client tools can expose the same configuration pattern. Cline’s OpenAI Compatible provider accepts a base URL, provider API key, and model ID for local or cloud providers. Review its model settings against the chosen backend’s documentation. Configuring a client successfully does not prove that an agent’s tool calls, context requirements, or image inputs will work.

Compare managed APIs, self-hosted servers, and Hivenet Router

The following table summarizes the operating choices. Refer to each product’s documentation before treating a capability as available.

OptionWho operates it?When to choose it
Hivenet Inference ServiceHivenet operates serving infrastructure and runtime. You configure the endpoint and client.A managed, dedicated model endpoint.
Compute with HivenetYou operate the inference runtime on rented GPU or CPU infrastructure.Custom weights, runtime choice, or operating-system access.
Hivenet RouterYou deploy and maintain Router and its agents alongside your inference servers.Shared routing, access controls, or fallback across backends. Router does not include GPUs or run models.

For managed Inference Service, you choose the model, variant, fixed replica count, API keys, and client configuration. The current capacity documentation says autoscaling is not available; check the console for capacity before deployment or resizing.

Do not use the feature list for one path as proof of what the other path supports. A self-managed vLLM server exposes the features of the version and configuration you deploy. The managed Inference Service exposes the interface described in the current Hivenet documentation and console-generated examples.

What to verify before switching OpenAI-compatible APIs

1. Documented routes

Ask for the exact supported endpoints instead of relying on the phrase “OpenAI-compatible.” Hivenet’s managed endpoint documentation covers Chat Completions. Do not assume that Responses, legacy Completions, embeddings, model listing, moderation, or another OpenAI route is available unless the current documentation or the endpoint’s generated example says so.

2. Base URL, API key, and authorization

Confirm whether the supplied endpoint URL already contains /v1. Hivenet Inference Service endpoint URLs do, so use the URL copied from the console without appending a second /v1. Create an API key for Inference Service in the active organization, not a Compute API key. Store it outside source code and use it with endpoints owned by the same organization.

3. Exact model value

Use the model value from the endpoint’s quick-start example. It can differ from the endpoint name or the model family shown in the catalog. Keep the endpoint URL, API key, and model value in configuration so you can change providers or deployments without rewriting application logic.

4. Requests and responses

Start with a small non-streaming Chat Completions request. Verify the status code and the fields your application reads, such as choices[0].message.content for a basic text response. Then test optional behavior such as streaming, tool calling, structured output, stop sequences, or token-usage fields only when the provider documents it for the endpoint you are evaluating.

5. Errors and retries

Record the status code and response body that the service actually returns. Do not build against an example error schema or rate-limit header that the provider has not documented. For retryable failures, use bounded exponential backoff, honor Retry-After when present, and avoid retrying authentication or validation failures without correcting the request.

Account for retries already built into your client. The current Python SDK and JavaScript SDK retry certain connection errors and HTTP failures twice by default. Coordinate SDK, application, and router attempts under one retry budget. After partial streamed output, treat a retry as a new generation and decide how to handle the incomplete answer rather than appending it blindly.

6. Quality, latency, and cost

Build a representative prompt set from your real workload. Compare answer quality, time to first token where available, total latency, throughput, and cost under the same request pattern. Record the test date, SDK and server versions, model, replica count, and enabled features with the results. Test the configuration you intend to use.

7. Location and data handling

Check the current Inference Service location behavior. The selected location applies across your Inference Service models; per-model location selection is not currently available. Choosing Global lets Hivenet assign an available location, which appears in endpoint details. Review that actual location, the console’s data-handling notice, and contractual terms. A region name alone does not establish logging, retention, residency, or compliance guarantees.

How to call an OpenAI-compatible API on Hivenet

Create an endpoint in the Hivenet Compute console, wait until it is running, create an API key for Inference Service, and copy the generated quick-start values. The endpoint must be accessible from the machine running your client. The endpoint URL already includes /v1.

The Python and server-side Node.js examples use the openai package. Set HIVENET_ENDPOINT_URL, HIVENET_INFERENCE_API_KEY, and HIVENET_MODEL before running them, and keep the key on the server. Automatic SDK retries are disabled here so the first compatibility check exposes the initial result. Choose a retry policy for production after testing. The examples were checked against the documented SDK interfaces but were not executed against a live endpoint for this review.

Send a curl HTTPS request

For a direct HTTP check, create a UTF-8 file named request.json with the JSON below. Replace YOUR_MODEL_VALUE with the exact value from the endpoint’s quick-start snippet.

{
  "model": "YOUR_MODEL_VALUE",
  "messages": [{"role": "user", "content": "Say hello in one sentence."}]
}

In a Bash shell, run this curl command after setting the endpoint URL and key environment variables. Use the HTTPS base URL copied from the console; it already ends in /v1. The Authorization header sends the bearer token, and Content-Type identifies the JSON body.

curl "${HIVENET_ENDPOINT_URL:?Set HIVENET_ENDPOINT_URL}/chat/completions" \
  -H "Authorization: Bearer ${HIVENET_INFERENCE_API_KEY:?Set HIVENET_INFERENCE_API_KEY}" \
  -H "Content-Type: application/json" \
  --data-binary @request.json

This follows Hivenet’s documented curl request. It is a non-streaming text check, not evidence that tools or other API routes work. The shell expansion and line continuations shown here are Bash syntax; use the SDK examples or adapt the command for your shell.

Python

import os
from openai import OpenAI

client = OpenAI(
  base_url=os.environ["HIVENET_ENDPOINT_URL"],
  api_key=os.environ["HIVENET_INFERENCE_API_KEY"],
  max_retries=0,
)

response = client.chat.completions.create(
  model=os.environ["HIVENET_MODEL"],
  messages=[
    {"role": "user", "content": "Write a one-sentence status update."}
  ],
)

print(response.choices[0].message.content)

JavaScript

import OpenAI from "openai";

function requiredEnv(name) {
  const value = process.env[name];
  if (!value) throw new Error(`Missing environment variable: ${name}`);
  return value;
}

const baseURL = requiredEnv("HIVENET_ENDPOINT_URL");
const apiKey = requiredEnv("HIVENET_INFERENCE_API_KEY");
const model = requiredEnv("HIVENET_MODEL");

const client = new OpenAI({ baseURL, apiKey, maxRetries: 0 });

const response = await client.chat.completions.create({
  model,
  messages: [
    { role: "user", content: "Give me one key risk for this project." },
  ],
});

console.log(response.choices[0].message.content);

Use the exact endpoint URL and model value shown by your endpoint. If the first request fails, follow the Inference Service troubleshooting guide before changing client behavior.

Run your own OpenAI-compatible server on Compute

Choose Compute when you want to deploy vLLM, TGI, SGLang, llama.cpp, or another runtime yourself. In this path, you are responsible for the runtime version, supported routes, model files, authentication, connectivity, updates, scaling, and monitoring.

Start with the Compute quickstart and the current Compute FAQ. After the server is running, test the interface documented by that runtime and configuration. A route supported by one vLLM release or template should not be assumed to exist on every self-managed deployment.

Use Hivenet Router as an API proxy for self-hosted servers

An OpenAI-compatible server handles inference requests. A router decides which eligible server receives them. With Hivenet Router, you deploy the router and connect agents beside the inference engines you operate. Those servers can run on your own hardware, Compute with Hivenet, or other infrastructure. Hivenet infrastructure is not required.

Router is currently in alpha, so pin the version you evaluate and recheck its documented behavior when upgrading. You remain responsible for deployment, transport security, client authentication, model access, backend capacity, updates, and monitoring. For a single server without shared routing requirements, calling the engine directly may be simpler.

The comparison of Hivenet Router, Inference Service, and Compute explains their operating responsibilities. Our guide to model selection and backend routing covers when the additional layer is useful.

Check the router and the backend together

For ordinary agent-backed chat routing, Hivenet Router forwards the request body to the same API path on the selected backend. The Chat Completions and Messages reference documents the supported paths and requirements. The backend must implement the endpoint the client calls. A Chat Completions server does not gain Anthropic Messages support simply because Router exposes a Messages route.

Tool calling, structured output, vision, audio, sampling controls, and backend-specific parameters depend on the selected engine and model. Test these features on every backend eligible for that workload. A successful text request does not establish that the same deployment can serve a tool call or multimodal input.

Local streaming also requires a streaming-capable backend. Router relays its chunks; it does not turn a complete non-streaming response into incremental output. The current Chat Completions reference documents a terminal stream_truncated error frame when an upstream stream is cut short. Test whether your SDK surfaces this error and handles partial output and final usage fields correctly across the full client-to-backend path.

Do not infer support for the Responses API, legacy Completions, or arbitrary backend endpoints from a working Chat Completions request. Verify each required route separately. Router's model discovery, embeddings, and reranking interfaces also have their own documented behavior and capability requirements.

Treat provider fallback as a separate compatibility test

External provider fallback uses a different path from ordinary backend forwarding. The current adapters target supported non-streaming chat requests and reconstruct selected fields instead of passing the original body through unchanged. They can change the model and data destination, and they do not preserve every tool, structured-output, or multimodal feature.

Test the provider-fallback limits against the exact requests your application sends. A streaming or tool-calling request that works on the primary backend is not proof that it will work after provider fallback. Decide whether a controlled error is preferable when no compatible destination remains. For queue, retry, and recovery acceptance checks, use our LLM fallback guide.

A practical migration checklist

  1. Choose managed Inference Service or a self-managed serving stack. If self-managing, decide whether direct backend access is enough or whether you also need a router.
  2. Copy the exact endpoint URL and model value from the current source. Map each required route and feature to a documented capability and a test result.
  3. Send one non-streaming Chat Completions request.
  4. Test the documented routes and fields your application needs on each eligible backend and any configured provider fallback.
  5. Compare quality, latency, throughput, and cost with representative prompts.
  6. Verify error handling without assuming a particular schema or header.
  7. Confirm location, data-handling, and contractual requirements before production use.
  8. Keep provider-specific values in configuration and retain a rollback path.

Source review: November 11, 2026. Managed-service claims were checked against Hivenet’s current documentation, and Router behavior against source revision f97fec8. This was a documentation review, not an integration or failover test. Confirm compatibility on the SDK, router, engine, and model versions you plan to deploy.

Frequently asked questions

Can I reuse the same model name I use with OpenAI?

Do not assume so. Use the exact model value shown in the endpoint’s generated example and keep it in configuration.

Should I append /v1 to a Hivenet Inference Service URL?

No. The endpoint URL copied from the Hivenet console already includes /v1.

Where should I check current models and locations?

Use the live Hivenet console and the models and variants documentation. Availability can change by model, variant, capacity, and location.

For managed endpoints or rented infrastructure, open the Hivenet Compute console. To evaluate routing across servers you operate, start with the Hivenet Router quickstart.

Your next workload belongs on Hivenet.

Pick one AI, compute, or storage workload and see the difference for yourself. Spin it up in minutes, or let our team map your fastest path to production.