A different strategy per document
A lightweight model tells policies, manuals, FAQs and table-heavy documents apart and sets the split unit and size for each. No one-size-fits-all setting.
TECHNOLOGY
We don’t let AI
We don’t let AI
“just read” your documents.
Anyone can feed a model the full text. It’s slow, expensive, and the answers get blurry. In Wisdombase, everything below runs by default, from the moment a document is uploaded to the moment an answer comes back.
Naive reading vs Wisdombase — measured
We asked 28 questions of a real 50-page company brain and measured with the actual tokenizer.
Test conditions: the 50-page WRKS AI user guide (280K characters) · 28 natural-language questions · qwen tokenizer usage · August 2026. Even against knowing the right document in advance, the gain grows with document length — a 13,455-token pricing policy question came out 9.3× cheaper.
Six stages run before every answer
All automatic. You configure nothing.
1
A lightweight model reads the document first
It identifies topic, structure and language and assigns a document type. Whether it’s a policy, a manual or an FAQ changes the strategy for every following stage.
{ "type": "policy", "split": "by clause", "chunk_size": 800 chars }2
Split to fit the document (strategic chunking)
Policies by article, FAQs by Q&A pair, manuals by section. Every chunk carries a “Document title › Section title” breadcrumb, so its source is clear on its own.
HR Policy › LeaveHR Policy › ExpensesFAQ › Q. Refunds?
3
Turn meaning into numbers (embedding)
Chunks become arrays of numbers so that different wording with the same meaning can still be found. The resulting vectors are stored AES-256 encrypted.
"15 days of annual leave…" → [0.021, -0.114, …] → DPV1:a83f… ciphertext4
Fuse two searches (hybrid RRF)
Semantic search finds meaning; full-text search finds exact words — and each misses different things. We merge both by rank (RRF), so mismatched scoring scales can’t distort the result.
Semantic+Full-text=Final ranking
5
Filter near-duplicates (MMR diversification)
If the top results all say the same thing, the answer rests on a narrow base. Candidates that overlap with what’s already chosen are penalized, so distinct evidence rises evenly.
Leave clauseLeave clause (duplicate)Family-event leave clause
6
Read only as much as needed (progressive reading)
When chunks aren’t enough, it widens step by step: outline → one-line section summaries → full text of the needed section. The whole document is the last resort. Tokens read scale with the question, not the document.
Outline→Section summaries→Needed section→Full text (last resort)
One key principle: summaries are used only to narrow what gets read. Search always runs against the full original text, so anything a summary leaves out is never missing from search.
Models chosen by measurement, not gut feeling
We compared 5 embedding models and 3 analysis models on the same question set before choosing. Two test sets: real Korean company documents, and synthetic Korean/English/Chinese documents.
| Embedding model | Accuracy on real documentsTop-1 hit · 372 chunks, 28 questions | KO/EN/ZH cross-lingual | Chosen |
|---|---|---|---|
| Qwen3 4B | 79% | 6/6 | ✓ Chosen |
| Qwen3 8B | 75% | 6/6 | |
| Qwen3 0.6B | 64% | 4/6 | |
| OpenAI small | 61% | 3/6 | |
| OpenAI large | 57% | 5/6 |
Interesting result — the most expensive model (OpenAI large) came last on real Korean documents. Picking the famous model without a benchmark would have cost 22 points of accuracy. That’s why we measure.
Your documents never leave the country
Most AI services send your documents to an overseas API to create embeddings. Wisdombase runs its embedding model on GPU infrastructure that AI3 operates itself, in Korea, under our own control.
Our GPUs, not someone else’s
Not rented cloud — GPU infrastructure AI3 bought and operates. We know exactly whose hardware your documents touch.
Starts in Korea, ends in Korea
Upload, conversion, embedding, storage and search all happen in Korea. No exposure to foreign providers’ terms changes or cross-border transfer issues.
Never used for training
On our own servers, “no training” is architecture, not a promise. The vectors are stored AES-256 encrypted.
Closer means faster
No round trip across the Pacific means lower latency. Upload thousands of documents at once and it all stays on domestic lines.
This matters most to organizations where cross-border transfer reviews are a burden, like public institutions and finance.
Need something to show your security team? See the security policy page.
All of this is simply the default
No option to switch on, no extra charge. It all applies the moment you upload.
Search that doesn’t miss
Ask “how many days off?” and find the document that says “annual leave” (semantic search). For proper nouns and codes where exact characters matter, full-text search backs it up. Both fused with RRF.
Reading that saves tokens
Outlines and section summaries are pre-generated by a lightweight model, so AI tools read exactly the part they need. Your AI bill follows the size of the question, not the document.
Connected by standard
This entire search and reading pipeline is exposed through MCP, the standard protocol. Plug it into ChatGPT, Claude or Gemini and it works the same everywhere.