RAG with embeddings

RAG (Retrieval-Augmented Generation) is an architecture where the model answers from your documents instead of its training memory: the corpus is split into chunks and vectorized, the nearest chunks are found for each question, and the answer is generated from them. The model gets access to your knowledge base without any fine-tuning.

Two Mixen endpoints cover the whole loop:

• POST /v1/embeddings — text into a vector (for both the index and the question);
• POST /v1/chat/completions — the judge model at the reranking stage and the final grounded answer.

Rerank

There is no dedicated rerank endpoint. Instead of a specialized rerank model we use LLM-as-judge: a cheap chat model scores the relevance of retrieved chunks (step 3). Quality barely suffers, and it costs kopecks.

How the pipeline works

1. Index — documents are split into chunks and vectorized once. This is the only heavy pass: it is amortized and never repeated.
2. Retrieval — the question is vectorized too; top 10–20 candidates are taken by cosine similarity.
3. Rerank — a judge model reads the candidates and keeps the top 3–5 truly relevant ones.
4. Answer — the final model answers from the remaining context and cites chunks by number.

Per request you only pay for the question embedding, the judge call and the answer generation.

Requirements

pip install -U openai numpy httpx   # httpx is needed for the async section below

The code targets Python 3.10+.

Step 1. Index

The example corpus is chunks of the Mixen docs; your documents take their place. Vectors are computed in a batch — one network round trip for the whole pack instead of per-item requests.

import hashlib
import json
from pathlib import Path

import numpy as np
from openai import OpenAI

client = OpenAI(
    base_url="https://api.mixen.ai/v1",
    api_key="your-api-key",
)

EMBED_MODEL = "text-embedding-3-small"  # 1536 dimensions, ~2.7 ₽ per 1M input tokens

chunks = [
    "Mixen is a neural network aggregator: text, images, video and audio in one dashboard, billed in roubles.",
    "Top up your balance via SBP, bank card, Telegram Stars or crypto; generation is billed for tokens actually spent.",
    "The public API is OpenAI-compatible: same SDK, only the base URL changes — https://api.mixen.ai/v1.",
    "API keys are created in the dashboard, start with mxn- and are limited to 60 requests per minute.",
    "POST /v1/chat/completions is the main endpoint: message history, SSE streaming, images and files on input.",
    "POST /v1/embeddings returns one vector per input text and preserves the order — convenient for batching.",
]


def embed(texts: list[str]) -> np.ndarray:
    """Batched embeddings: one API call for the whole pack."""
    resp = client.embeddings.create(model=EMBED_MODEL, input=texts)
    return np.array([item.embedding for item in resp.data])


def sha256(text: str) -> str:
    return hashlib.sha256(text.encode("utf-8")).hexdigest()


def build_index(chunks: list[str], cache_file: str = "embedding-cache.json") -> np.ndarray:
    """On-disk vector cache: sha256 of the chunk as the key, the vector as value.

    Editing one document does not re-embed the whole corpus: only chunks
    missing from the cache are sent to the API.
    """
    cache: dict[str, list[float]] = (
        json.loads(Path(cache_file).read_text()) if Path(cache_file).exists() else {}
    )
    missing = [c for c in chunks if sha256(c) not in cache]
    if missing:
        for chunk, vector in zip(missing, embed(missing)):
            cache[sha256(chunk)] = vector.tolist()
        Path(cache_file).write_text(json.dumps(cache, ensure_ascii=False))
    return np.array([cache[sha256(c)] for c in chunks])


matrix = build_index(chunks)  # (n_chunks, 1536)

Embedding models are served by the /v1/embeddings endpoint and are listed in the GET /v1/models catalog — look for the type: "embeddings" field: text-embedding-3-small (1536 dimensions), text-embedding-3-large (3072, more accurate and pricier at ~17.5 ₽ per 1M tokens) and the legacy alias text-embedding-ada-002, which maps to small.

Step 2. Retrieval

Cosine similarity on NumPy — the dot product of normalized vectors. No external vector DB needed to start.

def retrieve(question: str, top_k: int = 10) -> list[tuple[str, float]]:
    """Top-k nearest chunks by cosine; returns text and score."""
    q = embed([question])[0]
    q /= np.linalg.norm(q)                                       # normalize the question
    m = matrix / np.linalg.norm(matrix, axis=1, keepdims=True)    # and the index matrix
    scores = m @ q
    order = np.argsort(scores)[::-1][:top_k]
    return [(chunks[i], float(scores[i])) for i in order]


for text, score in retrieve("How can I top up my balance?", top_k=3):
    print(f"{score:.3f}  {text}")

Retrieve with a margin (10–20 candidates): the judge below narrows the list down to 3–5 chunks.

Step 3. Rerank with LLM-as-judge

Embeddings capture the topic well but confuse chunks that are similar in meaning yet different in substance. The judge is an ordinary cheap chat model that reads the texts and scores their usefulness for the specific question.

import re

JUDGE_MODEL = "z-ai/glm-5.3-flash"  # judge: 8.4 ₽ / 1M input, 27.9 ₽ / 1M output


def _parse_scores(text: str) -> list[float]:
    """JSON from the judge; fallback path pulls the array out with a regex."""
    try:
        return json.loads(text)["scores"]
    except (json.JSONDecodeError, KeyError, TypeError):
        match = re.search(r"\[[^\]]*\]", text)  # JSON may arrive wrapped in markdown
        try:
            return json.loads(match.group()) if match else []
        except json.JSONDecodeError:
            return []


def judge_rerank(question: str, candidates: list[str], top_k: int = 4) -> list[str]:
    """Score candidates with the judge model and keep the best top_k."""
    listing = "\n\n".join(f"[{i + 1}] {c}" for i, c in enumerate(candidates))
    resp = client.chat.completions.create(
        model=JUDGE_MODEL,
        max_tokens=1500,
        messages=[
            {"role": "system", "content":
                "You are a relevance assessor. Rank the chunks by how much "
                "each one helps answer the question. Reply with strict JSON "
                "with a scores key — a list of numbers from 0 to 1, one per "
                "chunk in the original order."},
            {"role": "user", "content": f"Question: {question}\n\nChunks:\n\n{listing}"},
        ],
    )
    scores = _parse_scores(resp.choices[0].message.content or "")
    if len(scores) != len(candidates):
        return candidates[:top_k]  # judge missed the format — cut as is
    ranked = sorted(zip(candidates, scores), key=lambda pair: pair[1], reverse=True)
    return [c for c, _ in ranked[:top_k]]

Reasoning eats the output budget

glm-5.3-flash is a reasoning model: it thinks before emitting the JSON, and the reasoning is billed and consumes max_tokens. A truncated response arrives with finish_reason: "length" and the parser falls back to the regex path. Set max_tokens to at least 1500. Even so, a judge call over ten chunks costs about 0.01–0.03 ₽ — cheap enough to run on every request.

When the judge pays off:

• the result set has many chunks close in topic but different in substance;
• precision is critical: support, legal docs, internal policies — better fewer but correct;
• the corpus is large and top-k has to stay at 15–20.

When you can skip it:

• the corpus is small (dozens of chunks) and cosines already separate the results well;
• latency matters more than precision — every extra call adds delay.

Step 4. Answer

The final model receives only the selected chunks and a strict instruction to cite sources.

ANSWER_MODEL = "anthropic/claude-sonnet-5"  # 213.9 ₽ / 1M input, 1069.5 ₽ / 1M output

SYSTEM_PROMPT = (
    "Answer only from the provided chunks. After every fact, cite the chunk "
    "number in square brackets: [1]. If the chunks do not contain the answer, "
    "reply “The documents do not contain the answer”."
)


def answer(question: str, evidence: list[str]) -> str:
    context = "\n\n".join(f"[{i + 1}] {c}" for i, c in enumerate(evidence))
    resp = client.chat.completions.create(
        model=ANSWER_MODEL,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": f"Document chunks:\n\n{context}\n\nQuestion: {question}"},
        ],
    )
    return resp.choices[0].message.content


question = "How do I top up my balance?"
candidates = [c for c, _ in retrieve(question, top_k=10)]
print(answer(question, judge_rerank(question, candidates)))

Prefer an OpenAI-family model — openai/gpt-5.6-sol at the same price (213.9 and 1069.5 ₽ per 1M input and output).

Keep the prompt prefix stable: the system instruction and the message skeleton do not change between requests, so they hit the prompt cache. Repeat requests read input at 21.4 ₽ per 1M instead of 213.9 ₽ — ten times cheaper; the cache lives for 5 minutes and each hit extends it. Rule of thumb: the constant instruction goes at the start of the prompt, the changing context at the end.

Streaming the answer

Our API returns real deltas: the first text arrives long before generation finishes. For long answers, wrap step 4 in a generator.

def generate_stream(question: str, evidence: list[str]):
    """The answer streams in deltas as it is generated."""
    context = "\n\n".join(f"[{i + 1}] {c}" for i, c in enumerate(evidence))
    stream = client.chat.completions.create(
        model=ANSWER_MODEL,
        stream=True,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": f"Document chunks:\n\n{context}\n\nQuestion: {question}"},
        ],
    )
    for chunk in stream:
        if chunk.choices and chunk.choices[0].delta.content:
            yield chunk.choices[0].delta.content


for piece in generate_stream(question, judge_rerank(question, candidates)):
    print(piece, end="", flush=True)

Async pipeline

In a bot or web server, blocking a worker for three API round trips is a bad idea. Below is the same pipeline on AsyncOpenAI: three sequential awaits — there is nothing to parallelize here, since every stage needs the previous one’s result (what does parallelize is in the multi-query section).

import asyncio

from openai import AsyncOpenAI

aclient = AsyncOpenAI(
    base_url="https://api.mixen.ai/v1",
    api_key="your-api-key",
)


async def rag_pipeline(question: str, top_k: int = 10, final_k: int = 4) -> str:
    """Index → retrieval → judge → answer, fully asynchronous."""
    q = np.array((await aclient.embeddings.create(
        model=EMBED_MODEL, input=question,
    )).data[0].embedding)
    q /= np.linalg.norm(q)
    scores = (matrix / np.linalg.norm(matrix, axis=1, keepdims=True)) @ q
    found = [chunks[i] for i in np.argsort(scores)[::-1][:top_k]]

    listing = "\n\n".join(f"[{i + 1}] {c}" for i, c in enumerate(found))
    judged = await aclient.chat.completions.create(
        model=JUDGE_MODEL,
        max_tokens=1500,
        messages=[
            {"role": "system", "content":
                "You are a relevance assessor. Rank the chunks by how much "
                "each helps answer the question. Reply with strict JSON with "
                "a scores key — a list of numbers from 0 to 1 per chunk."},
            {"role": "user", "content": f"Question: {question}\n\nChunks:\n\n{listing}"},
        ],
    )
    jscores = _parse_scores(judged.choices[0].message.content or "")
    if len(jscores) == len(found):
        found = [c for c, s in sorted(zip(found, jscores),
                                      key=lambda pair: pair[1], reverse=True)][:final_k]

    context = "\n\n".join(f"[{i + 1}] {c}" for i, c in enumerate(found))
    final = await aclient.chat.completions.create(
        model=ANSWER_MODEL,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": f"Document chunks:\n\n{context}\n\nQuestion: {question}"},
        ],
    )
    return final.choices[0].message.content


print(asyncio.run(rag_pipeline("What should I do about error 429?")))

Metadata

Plain chunk text only answers “what was found”. Interfaces also need “where from”: add a dataclass with the source and the section.

from dataclasses import dataclass


@dataclass
class Chunk:
    text: str
    source: str   # file, page or URL of origin
    section: str   # thematic section: "billing", "limits", "models"...


corpus = [
    Chunk("Top up your balance via SBP, bank card, Telegram Stars or crypto.",
          source="docs/pricing.md", section="billing"),
    Chunk("Generation is billed from the balance for tokens actually spent.",
          source="docs/pricing.md", section="billing"),
    Chunk("API keys start with mxn- and are limited to 60 requests per minute.",
          source="docs/rate-limits.md", section="limits"),
    Chunk("On error 429, retry the request with increasing backoff.",
          source="docs/rate-limits.md", section="limits"),
]

# the index is built over .text; the corpus list fixes the order
matrix = build_index([c.text for c in corpus])


def retrieve_filtered(question: str, section: str | None, top_k: int = 10) -> list[Chunk]:
    """Section filter goes BEFORE the judge: fewer candidates, cheaper and sharper scoring."""
    pool = [i for i, c in enumerate(corpus) if section is None or c.section == section]
    q = embed([question])[0]
    q /= np.linalg.norm(q)
    m = matrix / np.linalg.norm(matrix, axis=1, keepdims=True)
    scores = m @ q
    pool.sort(key=lambda i: scores[i], reverse=True)
    return [corpus[i] for i in pool[:top_k]]

The answer can now cite the source — [1] docs/pricing.md instead of an anonymous [1]: users see where a fact came from, and you find stale documents faster.

Multi-query RAG

Users phrase questions in their own words, and a single formulation may never intersect with the document vocabulary. The fix: an LLM reformulates the question several ways, retrieval runs for every variant, and candidates are merged with deduplication.

async def rephrase(question: str) -> list[str]:
    """Three variants of the question: synonyms, broader, more specific."""
    resp = await aclient.chat.completions.create(
        model=JUDGE_MODEL,
        max_tokens=800,
        messages=[{"role": "user", "content":
            "Rephrase the question in three different ways: with synonyms, "
            "more broadly and more specifically. Answer with exactly three "
            "lines, no numbering.\n\n"
            f"Question: {question}"}],
    )
    lines = (resp.choices[0].message.content or "").splitlines()
    cleaned = [l.strip().lstrip("0123456789.-) ") for l in lines if l.strip()]
    return [question, *cleaned[:3]]


async def multi_query_candidates(question: str, per_query: int = 5) -> list[str]:
    """Search per formulation; merge candidates without duplicates."""
    variants = await rephrase(question)
    resp = await aclient.embeddings.create(model=EMBED_MODEL, input=variants)
    m = matrix / np.linalg.norm(matrix, axis=1, keepdims=True)
    merged: dict[str, None] = {}  # dict instead of set — preserves arrival order
    for item in resp.data:  # response order matches the input order
        q = np.array(item.embedding)
        q /= np.linalg.norm(q)
        for i in np.argsort(m @ q)[::-1][:per_query]:
            merged.setdefault(chunks[i], None)
    return list(merged)


question = "how do I pay for the AI services"
found = asyncio.run(multi_query_candidates(question))
evidence = judge_rerank(question, found)

Rephrasing runs on the same cheap model and adds roughly another 0.01 ₽ per request. A side effect is robustness to typos and slang: at least one formulation usually lands in the document.

Full example

The complete script for steps 1–4 — copy it and plug in your corpus.

"""RAG over documents with Mixen: index → retrieval → judge → answer."""
import hashlib
import json
import re
from pathlib import Path

import numpy as np
from openai import OpenAI

client = OpenAI(base_url="https://api.mixen.ai/v1", api_key="your-api-key")

EMBED_MODEL = "text-embedding-3-small"
JUDGE_MODEL = "z-ai/glm-5.3-flash"
ANSWER_MODEL = "anthropic/claude-sonnet-5"

SYSTEM_PROMPT = (
    "Answer only from the provided chunks. After every fact, cite the chunk "
    "number in square brackets: [1]. If the chunks do not contain the answer, "
    "reply “The documents do not contain the answer”."
)

chunks = [
    "Mixen is a neural network aggregator: text, images, video and audio in one dashboard, billed in roubles.",
    "Top up your balance via SBP, bank card, Telegram Stars or crypto; generation is billed for tokens actually spent.",
    "The public API is OpenAI-compatible: same SDK, only the base URL changes — https://api.mixen.ai/v1.",
    "API keys are created in the dashboard, start with mxn- and are limited to 60 requests per minute.",
    "POST /v1/chat/completions is the main endpoint: message history, SSE streaming, images and files on input.",
    "POST /v1/embeddings returns one vector per input text and preserves the order.",
]


def sha256(text: str) -> str:
    return hashlib.sha256(text.encode("utf-8")).hexdigest()


def embed(texts: list[str]) -> np.ndarray:
    resp = client.embeddings.create(model=EMBED_MODEL, input=texts)
    return np.array([item.embedding for item in resp.data])


def build_index(cache_file: str = "embedding-cache.json") -> np.ndarray:
    cache: dict[str, list[float]] = (
        json.loads(Path(cache_file).read_text()) if Path(cache_file).exists() else {}
    )
    missing = [c for c in chunks if sha256(c) not in cache]
    if missing:
        for chunk, vector in zip(missing, embed(missing)):
            cache[sha256(chunk)] = vector.tolist()
        Path(cache_file).write_text(json.dumps(cache, ensure_ascii=False))
    return np.array([cache[sha256(c)] for c in chunks])


def retrieve(question: str, top_k: int = 10) -> list[str]:
    q = embed([question])[0]
    q /= np.linalg.norm(q)
    m = matrix / np.linalg.norm(matrix, axis=1, keepdims=True)
    return [chunks[i] for i in np.argsort(m @ q)[::-1][:top_k]]


def _parse_scores(text: str) -> list[float]:
    try:
        return json.loads(text)["scores"]
    except (json.JSONDecodeError, KeyError, TypeError):
        match = re.search(r"\[[^\]]*\]", text)
        try:
            return json.loads(match.group()) if match else []
        except json.JSONDecodeError:
            return []


def judge_rerank(question: str, candidates: list[str], top_k: int = 4) -> list[str]:
    listing = "\n\n".join(f"[{i + 1}] {c}" for i, c in enumerate(candidates))
    resp = client.chat.completions.create(
        model=JUDGE_MODEL,
        max_tokens=1500,
        messages=[
            {"role": "system", "content":
                "You are a relevance assessor. Rank the chunks by how much "
                "each helps answer the question. Reply with strict JSON with "
                "a scores key — a list of numbers from 0 to 1 per chunk."},
            {"role": "user", "content": f"Question: {question}\n\nChunks:\n\n{listing}"},
        ],
    )
    scores = _parse_scores(resp.choices[0].message.content or "")
    if len(scores) != len(candidates):
        return candidates[:top_k]
    ranked = sorted(zip(candidates, scores), key=lambda pair: pair[1], reverse=True)
    return [c for c, _ in ranked[:top_k]]


def answer(question: str, evidence: list[str]) -> str:
    context = "\n\n".join(f"[{i + 1}] {c}" for i, c in enumerate(evidence))
    resp = client.chat.completions.create(
        model=ANSWER_MODEL,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": f"Document chunks:\n\n{context}\n\nQuestion: {question}"},
        ],
    )
    return resp.choices[0].message.content


matrix = build_index()  # first run embeds, afterwards it comes from the cache

if __name__ == "__main__":
    question = "How can I top up my balance?"
    evidence = judge_rerank(question, retrieve(question, top_k=10))
    print(answer(question, evidence))

The economics of a typical request (10 candidates to the judge, 4 chunks in the answer context):

Stage	Rate	Magnitude
Question embedding	small, ~2.7 ₽ / 1M input	fractions of a kopeck
Judge	glm-5.3-flash, 8.4 / 27.9 ₽ per 1M input / output	0.01–0.03 ₽
Answer	claude-sonnet-5, 213.9 ₽ / 1M input (cache read 21.4 ₽) and 1069.5 ₽ / 1M output	the main item

A request usually costs 0.05–0.2 ₽: the final model’s output is the most expensive part (five times the input price), so short chunks and concise answers save directly. Indexing is amortized — vectors are computed once and sit in the hash-keyed cache, repeat runs are free. The whole pipeline stays under 1 ₽ per request even without caches.

Chunking documents

RAG quality is decided at the chunking stage more than by the choice of answer model:

• Size — 200–500 tokens per chunk. Smaller loses the context of a phrase; larger drags noise into the result and makes context pricier.
• Overlap — 10–15% between neighbouring chunks so a thought at a boundary is not lost. For Russian take 20%: rich morphology scatters related sentences more aggressively.
• Boundaries — cut at headings and paragraphs, not mid-sentence by character count.

A ready-made splitter comes from langchain-text-splitters (a standalone package — the Langchain core is not required):

pip install -U langchain-text-splitters

from langchain_text_splitters import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=1200,     # in characters: roughly 300–400 tokens of Russian text
    chunk_overlap=240,   # 20% — headroom for morphology
    separators=["\n\n", "\n", ". ", " ", ""],  # from paragraphs down to words
)
pieces = splitter.split_text(Path("docs/pricing.md").read_text(encoding="utf-8"))

Production stack

A NumPy matrix is enough up to tens of thousands of chunks. Once the corpus outgrows memory or you need filters and replication, move to a vector store:

Tool	What it is	When to choose
pgvector	a Postgres extension	data already lives in Postgres — vectors go into the same schema, no new DB
Qdrant	self-hosted vector DB	large corpora, metadata filters, hybrid search out of the box
Chroma	embedded database	prototypes and small corpora with persistence
FAISS	an indexing library	maximum search speed, everything in memory
Weaviate / Pinecone	managed services	you do not want to run the infrastructure yourself

Embeddings are not recomputed during the migration: the same text-embedding-3-small and the same vectors load into the store as is.

A BM25 hybrid covers the blind spot of embeddings — exact names, product codes, error codes:

# pip install -U rank_bm25
from rank_bm25 import BM25Okapi

bm25 = BM25Okapi([c.lower().split() for c in chunks])


def hybrid_retrieve(question: str, top_k: int = 10) -> list[str]:
    """Embeddings catch meaning, BM25 catches exact word forms; candidates go to the judge."""
    q = embed([question])[0]
    q /= np.linalg.norm(q)
    m = matrix / np.linalg.norm(matrix, axis=1, keepdims=True)
    by_vector = {chunks[i] for i in np.argsort(m @ q)[::-1][:top_k]}
    by_keyword = {chunks[i] for i in np.argsort(
        bm25.get_scores(question.lower().split()))[::-1][:top_k]}
    return list(by_vector | by_keyword)

And the hash-keyed embedding cache from step 1 moves over unchanged — it is just a layer in front of any store.

Practice

• One model for the index and for queries. The small and large vector spaces are incompatible: mix them and cosines turn to noise.
• Index in batches. One /v1/embeddings call per pack of chunks — faster and cheaper than per-item requests.
• top-k — start with 10 out of retrieval and narrow to 3–5 after the judge.
• Cosine cutoff around 0.3. Below that, chunks are almost certainly irrelevant; an honest “not found” beats context made of noise.
• The judge is not for every request. If the top-10 is already well separated (a visible cosine gap) — go straight to the answer and save the latency.
• A Russian corpus with quality demands — text-embedding-3-large. Honest caveat: we carry no specialized multilingual models (e5, multilingual) — large is the best multilingual option among ours.
• A stable prompt prefix — a system instruction unchanged across requests hits the prompt cache and cuts the answer model’s input price roughly tenfold.
• A hash-keyed embedding cache — reindexing touches only the chunks that changed, not the whole corpus.

Troubleshooting

The embeddings batch fails with 400

The batch’s total input exceeded the request limit. Cut packs down to ~100 chunks, fewer for long chunks:

def embed_safe(texts: list[str], batch_size: int = 100) -> np.ndarray:
    parts = [embed(texts[i:i + batch_size])
             for i in range(0, len(texts), batch_size)]
    return np.vstack(parts)

The judge returns something that is not JSON

Three typical causes. First, the model wrapped the answer in a markdown block labelled json: the regex in _parse_scores pulls out the array and ignores the wrapper. Second, reasoning consumed max_tokens and the response was cut mid-way (finish_reason: "length"): raise the ceiling to 1500 or higher. Third, explanatory text around the array; the regex handles that too. If the judge keeps breaking, make sure the system prompt demands strict JSON.

Cosines are all alike or garbage

Check normalization: both the question and the matrix rows are divided by their norms before the dot product. Then look for duplicate chunks in the index (they bloat top-k) and whether the same embedding model was used for the index and the query. Mixing models produces plausible-looking but meaningless scores.

The model hallucinates on top of the context

Tighten the system prompt: “if the chunks lack the data, say so”, no general knowledge. Reduce chunks to 3 after the judge, enable the cosine cutoff and require a chunk-number citation after every fact — an unsupported claim becomes visible immediately.

The answer takes too long

Stream the answer: the user sees the first text immediately instead of after the full generation. Parallelize the judge and query reformulation with asyncio.gather, take embeddings from the cache, and skip the judge entirely for simple corpora.

Weak quality on Russian

Switch to text-embedding-3-large, make chunks smaller (200–300 tokens) and raise the overlap to 20%. Add the BM25 hybrid — exact word forms are caught better by lexical search than by embeddings.

Next steps

• Integrations — Dify and n8n: a ready-made RAG pipeline without code.
• Chat and streaming — all POST /v1/chat/completions parameters, including reasoning modes and web search.
• Model catalog — models and prices for the answer stage.