Who Taught It What to Ask? Why Quaestio's Real Product Is Its Knowledge Base

Anyone can call a frontier model. What makes Quaestio's feedback worth having is what the model is told to look for, and what it is never allowed to say. That lives in a teacher-approved knowledge base, and it is the real product.

In What Does a Question Cost? I priced a review one model call at a time. Tucked into the assumptions was a line about roughly 18,000 tokens of shared material sent with every review: instructions, criterion checklists and knowledge-base entries, cached because they are the same for every student. On the bill, it is one of the cheapest inputs. In this post I want to explain why it is also the most valuable thing Quaestio owns.

Everyone has the same engine

When I tell people I'm building an AI feedback tool for coursework, the first question is usually: which model does it use? It's a fair question, and the answer is a good one. Quaestio uses a frontier model, among the best available. But it's also the least interesting thing about it.

Any school, any student and any competitor can use the same model tomorrow. If the model were the product, there would be no product. I learned this in presales: the follow-up to every good demo was what stops someone else doing this next quarter? For an AI product, "we use a better model" is never the answer, because the model is rented, and everyone can rent it.

What makes a Quaestio review different is what the model is told to look for, and how it is allowed to respond. That lives in the knowledge base.

What the knowledge base actually is

Think about how an experienced supervisor reads a science draft. They don't just read it and form an impression. They carry a mental checklist built over years. Does the research question name both variables? Does the uncertainty reflect how much the repeated readings varied? Does the conclusion answer the question that was actually asked? And for each concern, they know a good question that helps the student see it for themselves.

Quaestio's knowledge base writes that checklist down. For every subject, level and type of assessment, it holds a set of checks, each testing a single expectation. Every check has the same parts:

The rule for content is simple. Every word is original, written by subject specialists, and nothing is published until a qualified teacher or examiner who did not write it has approved it. This is also where two of the four follow-up options I described in Building Quaestio come from: seeing which part of the criteria a question relates to, and finding where to read more, are served straight from the knowledge base, with no model call at all.

One check, start to finish

Take a physics student who writes "±0.1 s" next to every timing, even though her three repeated trials differ by half a second. A general chatbot might tell her the correct uncertainty, or simply rewrite her analysis. Quaestio's check for this has a narrow brief: notice whether the stated uncertainty ignores the spread of the repeats and, if it does, ask something like:

The same check also says, explicitly: do not suggest a value, and do not name a formula. The student still has to do the thinking. She just knows where to look.

Why this is the part that matters

It makes "questions, never rewrites" real

Promising question-only feedback is easy. Guaranteeing it is hard when all you have is an instruction to a chatbot. When questions come from templates that have been tested against the same filters as live feedback, and every check carries its own list of forbidden suggestions, the promise is built into the content rather than hoped for from the model. It is the same principle as the rest of Quaestio: rules the system enforces, not promises the model is asked to keep.

It makes the AI consistent

The AI is only asked whether a specific, visible pattern appears in a specific part of the draft. That is a narrow job, and narrow jobs are done reliably. The same draft gets the same checks every time, whichever student submits it and whichever school they attend. That matters, because part one began with how much the help students get depends on luck.

It makes Quaestio credible as an independent voice

Many students look for a second opinion precisely when they aren't sure their teacher's feedback is fair. A second opinion is only worth having if it is grounded in the criteria rather than in an AI's impression. Every question Quaestio asks traces back to a specific, teacher-approved, versioned check, so if a student or school queries a review, there is something concrete to look at.

It improves without touching student work

Most AI products get better by learning from their users. We have promised never to do that: "never train, never remember" is one of Quaestio's five principles. The knowledge base is how we keep that promise and still improve. Quality comes from expert authors and a test library of drafts we write ourselves, plus simple counts from the buttons every student already sees: how often a question is accepted, and how often it is rejected as not relevant. A check that students keep rejecting goes back to a teacher for review. No essay is ever read to make that happen.

It gets stronger over time

Checks shared across the sciences are written once and reused, so each new subject builds on the last. Every issue we find becomes a new test case. And because each check needs a teacher's careful review, the finished library will represent hundreds of hours of expert work that cannot be replicated overnight. A competitor can rent the same model on the same afternoon. They cannot rent that.

What it takes

None of this comes free, and I'd rather say so plainly.

• Teacher time. Every check and every question is reviewed by a teacher before it is published. That is the largest cost of running Quaestio, far larger than the model cost I worked out in part three.
• Versioning. Assessment criteria change, so content has to be versioned to match, and a review always records which version of each check it used.
• Originality. The knowledge base describes expectations in our own words. It never reproduces official guides or marked exemplars, and guarding that takes care at every step.

That is the hard part. It is also why it is worth doing.

Three things I learned

1. In an AI product, the moat is rarely the model. The model is the part every competitor can buy. The defensible part is the domain knowledge that tells the model what good looks like, and the review process that makes that knowledge trustworthy.
2. Narrow jobs make AI reliable. "Give feedback on this essay" invites the model to improvise. "Does this pattern appear in this passage?" doesn't. Breaking expert judgement into small, checkable steps is what turned a promise into a guarantee.
3. A promise not to learn from users forces you to design how you will learn. Ruling out training on student work felt like a constraint. It became a design: expert authors, synthetic test drafts and simple usage counts, with a teacher in the loop for every change.

Back to the question

When people ask which model Quaestio uses, I'll keep answering honestly. But the better question is: who taught it what to ask?

The answer is teachers. That is exactly how it should be.