AIVAX
Console

AIVAX / Blog

Notes from the operating layer.

Field notes on AI infrastructure, retrieval, model operations and the systems that keep every request visible.

44 published field notes

Latest blog posts

  1. Watercolor of a tidal flat under an overcast sky, where a faint horizontal waterline crosses the scene and only three dark rocks rise above it while the rest show as blurred shapes beneath the sea-foam surface

    AIVAX Labs

    RAG similarity threshold: let retrieval come back empty

    Top-k vector search returns k chunks even when your documents can't answer the question. How to read your own scores, where the cutoff sits next to a reranker, and what the model should do when nothing passes.

    Read post
  2. Watercolor on dark toned paper in indigo, midnight blue and dim amber: a wide lit bridge crossing a night canal on the right, and on the left a narrow rope footbridge with a single amber lantern that fades into fog halfway across

    AIVAX Labs

    Cheap Claude API: how to vet a discounted provider or relay

    How to vet a discounted Claude or GPT API relay: test whether it serves the model it names, read its data policy, pick the workloads that tolerate a discount, and keep discounted routes opt-in on AIVAX.

    Read post
  3. Vermilion paper strips on an off-white page, some peeling up and tied by thin blue threads to a parallel column of cobalt cut-paper shapes on a slate surface

    AIVAX Labs

    Translate a PDF with an LLM: extract the text first, then translate it in chunks

    Translating a PDF with an LLM is two jobs: keeping the layout, or getting the text into another language. For the text, extract Markdown with OCR, split it at headings, and send the chunks to a translation endpoint in batches.

    Read post
  4. A tall stack of translucent paper sheets dissolving into cold mist that condenses into a single water droplet suspended above an empty glass, on a pale stone table against a glacial horizon

    AIVAX Labs

    Summarize PDFs, images, and text with one API call

    AIVAX's Summarization API condenses text, images, and PDFs into prose shaped by optional instructions — one summary per document, in input order, billed only for input.

    Read post
  5. Matte gouache painting in beige, charcoal, bronze and warm gold: an excavation trench where a rectangular block of stratified earth opens like a drawer, revealing a precise grid of small compartments, each holding one artifact — a coin, a key, a seal — while the surrounding strata stay rough and unsorted

    AIVAX Labs

    Turn PDFs and images into JSON with one API call

    AIVAX's Extraction API returns one JSON object per document, shaped by the schema you send. How the schema drives the search, what comes back when a field is missing, and when to extract instead of summarizing or fetching.

    Read post
  6. A sunlit control tower of pale concrete and ochre steel seen from below, its horizontal window bands lit like lines of a terminal screen against a cerulean sky, with a narrow white stair spiraling up the tower's flank

    AIVAX Labs

    AIVAX CLI: operate your account from the terminal

    The AIVAX CLI (@aivax/cli) covers the same products as the API from one binary — gateways, RAG, Batch, OCR, tests — with agent-optimized responses and keys in the OS credential store.

    Read post
  7. Charcoal and pastel drawing on gray paper: a large dark disc tilted on pale rippled sand, with a ring-shaped hub, orange lines across its face and a cracked patch beside the hub; dark rocks glowing orange at their base, and a low horizon ridge glowing orange under a pale sky

    AIVAX Labs

    Unsupported parameter: 'temperature' is not supported with this model: which GPT-6 models accept it

    GPT-6 rejects temperature and top_p above reasoning effort none, and only GPT-6 Sol, Luna and 6.1 Terra can run at none. What AIVAX does with the parameter.

    Read post
  8. Terraced hillside of dry-stone walls in a risograph print of blue and ochre: upright wheat sheaves tied with blue cloth on the upper terraces, and below the lowest wall three sheaves lying scattered in the dirt with their bands loose

    AIVAX Labs

    Extract structured data from a PDF with an LLM API: JSON from a schema, billed and metered per document

    AIVAX's extract endpoint turns PDFs and images into JSON shaped by your schema. Billing and quota are per document, so send one document per request and retry only the one that failed.

    Read post
  9. Aquatint of a long banquet table in a dark blue hall under arched windows: three cloches lit by an amber lamp on the left, one chair pushed back from the table, and a row of closed cloches running off to the right, with a moon in the far window

    AIVAX Labs

    OpenAI Realtime API interruptions and barge-in: truncate with the audio the user actually heard

    Over WebSocket the client owns playback. Keep a playback ledger per item and send conversation.item.truncate with the milliseconds the user actually heard.

    Read post
  10. Two sandy islands in calm teal water seen from above: the larger holds a loose cluster of terracotta crates, many with blank paper tags, and the smaller holds four crates bound in green cord beside a moored rowboat

    AIVAX Labs

    Expired OpenAI or Gemini batch job: partial results and retrying only what failed

    OpenAI keeps an expired batch's finished responses and lists the rest in an error file. Gemini's guide says an expired job has no results. Retry script included.

    Read post
  11. Seven bronze solids in a row on a pale plaster ledge, most tied with cord to blank paper tags: a sphere, cone, cylinder, cube, half-dome, ring and faceted block, with one larger untagged cube behind them against a dark olive wall in warm side light

    AIVAX Labs

    How to get the cost of each LLM API call: the Consumed-Credits header and pricing streams

    OpenAI's usage object has tokens, not cost. AIVAX sends a Consumed-Credits header on non-streamed calls; for streams, price the final usage chunk.

    Read post
  12. Ink and watercolor of a timber-framed barn taken down for moving: a pale patch of footing stones on the left, oak beams and posts lying in two rows beside a heap of wooden pegs, and one bay already raised on new stones next to a mallet, with green hills and an oak behind

    AIVAX Labs

    OpenAI Prompt Objects and Agent Builder shut down November 30: what you lose and where each piece can live

    OpenAI retires reusable prompt objects (v1/prompts) and Agent Builder on November 30, 2026. Copying the text into code is the easy part; hosted editing, rollback and hosted tools each need a new owner. This post reads the code behind each option and runs the parts that work offline.

    Read post
  13. Three chipped white ceramic beads and four glossy cyan ones in a diagonal cluster, resting on layered navy paper

    AIVAX Labs

    GPT-5 and o3 deprecation date: December 11, 2026. Migrate to GPT-5.6 Sol, Terra, Luna

    OpenAI shuts down GPT-5, mini, nano, Pro, o3 and o3-pro on December 11, 2026. At list prices, GPT-5.6 input costs 2-8x more; retest effort and tools.

    Read post
  14. A linocut of stepping stones crossing a bright blue channel: cracked ochre stones sink on the left, and smooth white stones curve away to a sunlit bank crowded with sunflowers

    AIVAX Labs

    GPT-4o and GPT-4.1 deprecation date (January 14, 2027): how to migrate to gpt-6-luna

    OpenAI retires GPT-4o, GPT-4o mini, GPT-4.1 and GPT-4.1 mini on January 14, 2027. Move to gpt-6-luna at effort none, then re-test tool calls, sampling and cost per task.

    Read post
  15. A black basalt wall with one thin vertical slot and a tall arched opening; a broad violet beam from the left narrows into a thread of light along the slot, while a ribbon of light bends through the arch

    AIVAX Labs

    'Function tools with reasoning_effort are not supported': the fix depends on the model

    Chat Completions rejects function tools combined with reasoning_effort on GPT-5.4 and later. Which models accept none, which need Responses, and a gateway workaround.

    Read post
  16. A narrow beam of cold light passes through a facade of silver and frosted-glass louvers and falls on three pale-blue glass spheres on an icy floor

    AIVAX Labs

    RAG metadata filtering: pre-filter by tags, metadata and date

    Filter RAG results by metadata, tags and date before ranking, or run a filter-only browse. Syntax, why filters return nothing, and a working AIVAX request.

    Read post
  17. Vermilion archways cross a pale tiled courtyard, shifting out of alignment beside a single black tile and casting long diagonal shadows.

    AIVAX Labs

    Low prompt cache hit rate on OpenAI and Anthropic: causes and how to calculate it

    Calculate your prompt cache hit rate with the right input total for each API, then find the changed prefix or expired entry behind the misses. Covers OpenAI, Anthropic and AIVAX.

    Read post
  18. Five blue paper tags in a column, each on its own blue thread, beside one long yellow paper strip folded back and forth, with a blue thread lying along each end of the strip

    AIVAX Labs

    OpenAI Evals shutdown: what to save and where multi-turn tests go

    OpenAI Evals went read-only on 31 October and is due to close 30 November. What to save, recreating dataset evals in Promptfoo, and where multi-turn tests go.

    Read post
  19. A dense cluster of tiny pink glass seedpods narrows toward three larger coral pods on a rippled sage-green surface.

    AIVAX Labs

    Too many MCP tools: measure and cut tool-definition tokens

    Every MCP tool a client loads puts its name, description, and schema into each request. How to measure that overhead and cut it without losing the right tool.

    Read post
  20. Five cut bamboo sections stacked beside curled paper in a misty bamboo grove

    AIVAX Labs

    PDF to Markdown for RAG: extract, chunk, and import JSONL

    Convert a PDF to Markdown with OCR, split it into retrieval-ready chunks, and export JSONL your RAG collection imports. Two API calls and a short script.

    Read post
  21. A row of weathered wooden stakes with pale bands stands in wet tidal sand at dusk, a lantern hanging from the nearest one, while driftwood and a small rowboat drift along curved currents

    AIVAX Labs

    OpenAI GPT-4, o1, o3-mini and o4-mini shutdown on October 23, 2026: what breaks and what to check

    OpenAI shut down GPT-4, o1, o3-mini, o4-mini and other models on October 23, 2026, so requests naming them now fail. Find the IDs left in fallbacks and jobs, then check each replacement's parameters, output and cost.

    Read post
  22. A red fabric ribbon opens into a broad sail and deep pleated folds against a charcoal background

    AIVAX Labs

    Generate images in Claude Code with an MCP server

    Claude Code has no image model of its own. Connect AIVAX's hosted Media Generation MCP to generate images with size and format control, or MP3 speech, and see each image's cost.

    Read post
  23. Two dark teal glacial streams converge into one channel between textured white and pale blue ice banks, seen from directly above

    AIVAX Labs

    Idempotent tool calls: stop an LLM agent retry from creating a duplicate ticket

    Idempotency in AI agents starts outside the model: tool_call_id is not a business key. Mint the operation ID in your backend, claim it with the write, and replay the stored result.

    Read post
  24. Sunlit open-air concrete room with blank paper sheets clipped to slatted walls, diagonal bands of yellow light across them, and a blue ring on a stand in the foreground

    AIVAX Labs

    How to check PDF extraction quality before embedding for RAG

    Before embedding, sample real pages, write the facts you expect, and grade each document for wrong numbers, lost table headers, and missing facts.

    Read post
  25. An open paper folio connected by red thread to folded blue paper pieces on an ivory field

    AIVAX Labs

    How MCP skills carry a workflow across the server boundary

    SEP-2640 lets MCP servers offer discoverable workflow skills as resources. The host still decides what to load, verify, approve, and execute.

    Read post
  26. Four distinct botanical forms grow from one pale seed pod in a sunlit greenhouse

    AIVAX Labs

    Free RAG API: what AIVAX Free includes for embeddings, reranking and OCR

    AIVAX Free includes daily allowances for RAG embeddings, Reflex reranking, Julia-1 decisions and Fetch/OCR. What each covers, the request limits that apply, and how to prototype a RAG pipeline inside them.

    Read post
  27. Rough cracked clay vessels on a wooden bench passing under a glowing brass caliper gauge, emerging as smooth measured vessels on a ledger grid

    AIVAX Labs

    When an LLM returns invalid JSON: truncation, refusals, repair

    Invalid JSON from an LLM usually means truncation, a refusal, or a schema miss. Where AIVAX healing applies, and what your code must still validate.

    Read post
  28. Three branching channels of water pass through separate gates before converging into a single measuring basin

    AIVAX Labs

    Route by complexity, price by the task

    A lower token rate does not tell you what a completed task costs. AIVAX's Complexity Router selects a model and reasoning effort for each request so teams can measure quality and spend at the task level.

    Read post
  29. A ceramic water mill turning a mixed stream of paper sheets, photo prints, and ledger pages into a single clear channel of plain text slips

    AIVAX Labs

    How to extract text from a URL, scanned PDF or image for an LLM pipeline

    Use AIVAX Fetch to extract text from URLs, scanned PDFs and images, with per-item errors, processing-unit usage and optional schema-guided JSON.

    Read post
  30. A brass sorting machine on a wooden desk sliding three blank paper tickets down chutes into wooden trays

    AIVAX Labs

    Decisions API example: triage a support ticket by urgency, department and frustration

    Triage a support ticket in one Decisions API call: urgency (noul), team (choice) and frustration (score), with curl, routing code and model prices.

    Read post
  31. An ice-white evidence table where pale-blue conduits connect many translucent search cards to three open source documents under glass mounts

    AIVAX Labs

    Web search API for LLM agents: why you also need a fetch step

    A web search API gives an LLM URLs and snippets, not evidence. How we chain search and fetch, what we keep as an evidence trail, and what each step costs on AIVAX.

    Read post
  32. A glasshouse conservatory at dawn where labeled seedling trays feed a long irrigation channel that runs past open gardening manuals toward the glasshouse doors

    AIVAX Labs

    Give your coding agent a memory it can read and a world it can fetch

    Coding agents lose context between sessions and guess at the world outside the repo. AIVAX's Collections and Web Utilities MCP servers give them writable semantic memory plus fetch and search — with scoped credentials, bounded retrieval, and explicit write control.

    Read post
  33. A weathered wall of card-catalog drawers with blank brass label plates, one drawer half-open with a slipped card tied by a single red thread to a small brass desk lamp

    AIVAX Labs

    How to protect AI agent memory from poisoning

    Memory poisoning persists across sessions. Scope writes to a verified user, expire on policy, record provenance, and test what a saved claim does later.

    Read post
  34. A stone sluice gate with three channels meters a river of record-cards into calm terraced basins while a gatekeeper works a lever

    AIVAX Labs

    How to run thousands of LLM requests in batch with rate limits and retries

    Use bounded concurrency, balance checks, targeted retries, and resumable jobs to process thousands of LLM requests without replaying completed work.

    Read post
  35. Terraced night gardens crossed by thin diagram paths with node dots, one ember-red path veering away past a small stone watchtower

    AIVAX Labs

    From test to monitor: the judge trajectory as a production signal

    Agentic Tests already scores every turn of a simulated conversation. Applied to real production traffic, the same trajectory signal — score, at-risk state, persistent loss — tells you when a live conversation is drifting before the user gives up.

    Read post
  36. Blue request cards, each carrying a yellow state token, travel independently toward three red server blocks in a textured screen print

    AIVAX Labs

    Migrate an MCP server off Mcp-Session-Id: the stateless 2026-07-28 spec

    MCP 2026-07-28 drops initialize and Mcp-Session-Id. Move state to explicit handles, make retries safe, and keep a legacy path for older clients.

    Read post
  37. Botanical flat lay in cream, pink, and green with pressed leaves, handmade papers, bowls, and a vessel of grains

    AIVAX Labs

    RAG vs vector database: what vector search leaves for you to build

    A vector database indexes and searches embeddings. RAG also needs extraction, chunking, updates, ordering and evaluation. Here is where each job lands.

    Read post
  38. Translucent capsules carried on deep-blue currents pass through gated openings in a porous white boundary into calm terraced channels

    AIVAX Labs

    How to safely connect third-party MCP servers

    Review MCP tools, credentials and imported instructions before connecting. Apply authorization and approval rules, with AIVAX configuration examples.

    Read post
  39. A scaffold bridge carries seed pods, glass vessels and archives from a root-bound grove monolith across a gorge into an open modular garden

    AIVAX Labs

    Migrating from OpenAI Assistants to Responses after the shutdown

    OpenAI shut down the Assistants API on August 26, 2026. Map threads, runs and file search to Responses, recover saved history, and decide who owns state.

    Read post
  40. A precision glass-and-metal translator separating layered signals into visible and opaque output channels

    AIVAX Labs

    How to set reasoning effort across providers: OpenAI, Anthropic and Gemini

    Compare OpenAI reasoning effort, Anthropic thinking and Gemini thinking controls. Configure AIVAX requests and handle summaries, state and token costs.

    Read post
  41. A monumental stone gateway filtering irregular colored fragments before they enter a calm central chamber

    AIVAX Labs

    LLM input moderation: system prompt or separate moderation step?

    Use a separate moderation step to decide whether input may proceed before generation. System prompts guide behavior; they do not enforce permissions.

    Read post
  42. A tactile tabletop evaluation with three parallel conversation tracks converging on a final scored outcome

    AIVAX Labs

    How to test multi-turn AI agent conversations with a simulated user and an LLM judge

    Define an observable goal, simulate a user, and judge the full conversation. Inspect weak turns, recovery, outcomes, and costs with AIVAX Agentic Tests.

    Read post
  43. A broad field of document cards narrowing into a candidate set and then an ordered shortlist

    AIVAX Labs

    Do I Need a Reranker for RAG? Semantic Search vs. Reranking

    Test a reranker when relevant documents rank too low. Fix missing candidates first, then measure answer quality, latency, and cost.

    Read post
  44. Smooth stones arranged in decreasing size on warm sand, representing recurring document retrieval

    AIVAX Labs

    Best reranker for RAG: Reflex vs Cohere, Voyage, Jina, Qwen and NVIDIA

    Reflex reranks RAG candidates with semantic and lexical signals instead of a model pass per pair. Where that is enough, what it costs next to the model rerankers on the same endpoint, and how to test the choice.

    Read post