What an AI Interviewer Actually Scores You On (Our Real Rubric, Published)
Most candidates guess at the rubric. We are publishing ours: the five competencies, their weights, the pass gates, and the score bar for SDE 2, SDE 3 and Staff.
Most candidates prepare for interviews without ever seeing the rubric. We are publishing ours. Below are the actual competencies, weights, pass gates and score targets used by the AI interviewer on the InterviewLM marketplace, for five live engineering roles across three seniority bands. Not a simplified version — the numbers the system uses.
We are publishing it for a self-interested reason as well as a fair one: an assessment that only works when candidates cannot see the criteria is not measuring skill, it is measuring insider knowledge. If knowing the rubric lets you score higher, you should know the rubric.
How the scoring actually works
Three ideas do most of the work.
1. Competencies, not a single number. Each role is scored on four to six named competencies, each carrying a weight. There is no one score that everything collapses into during evaluation — the composite is derived from the parts, and the parts are what a hiring manager reads.
2. Every competency has a tier. MUST_HAVE competencies carry a hard minimum: fall below it and the composite score cannot rescue you. CORE competencies carry the most weight. SUPPORTING competencies matter but will not sink you alone.
3. The bar moves with the band, the criteria do not. SDE 2, SDE 3 and Staff candidates are scored on identical competencies with identical weights. Only the target level and the gate change. This is deliberate: it means everyone in a pool sat the same interview against the same rubric, so ranking is a like-for-like comparison.
The five scored competencies
| Competency | What it actually measures | How it shows up in the interview |
|---|---|---|
| technical-depth | Whether you understand the systems you claim to have built, below the API surface | Follow-ups that go one layer past your first answer, until you reach your edge |
| problem-solving | Decomposing an underspecified problem and defending trade-offs | Open design prompts with missing requirements you are expected to notice |
| code-quality | Whether what you build can be safely changed by someone else | How you talk about testing, failure modes and change safety |
| communication | Explaining technical reasoning to someone who is not inside your head | Whether the interviewer has to ask you to clarify, and whether you say why not just what |
| ai-collaboration | Directing models deliberately and verifying what comes back | How you use, question and correct AI when it is available in the round |
Real rubrics, three roles
Weights multiply the competency score in the composite. Targets are the score level expected at that band, shown as MID → SENIOR → LEAD (published as SDE 2 → SDE 3 → Staff).
Agentic AI Engineer
| Competency | Tier | Weight | Target: SDE 2 → SDE 3 → Staff |
|---|---|---|---|
| ai-collaboration | MUST_HAVE | 1.5 | 78 → 93 → 95 |
| technical-depth | CORE | 1.3 | 75 → 90 → 95 |
| problem-solving | CORE | 1.2 | 72 → 87 → 95 |
| code-quality | CORE | 1.0 | 65 → 80 → 90 |
| communication | SUPPORTING | 0.8 | 62 → 77 → 87 |
Hard gate: ai-collaboration must reach 55 (SDE 2), 70 (SDE 3) or 80 (Staff). Below that, nothing else matters.
Backend Engineer
| Competency | Tier | Weight | Target: SDE 2 → SDE 3 → Staff |
|---|---|---|---|
| code-quality | MUST_HAVE | 1.3 | 75 → 90 → 95 |
| technical-depth | CORE | 1.3 | 75 → 90 → 95 |
| problem-solving | CORE | 1.2 | 72 → 87 → 95 |
| ai-collaboration | SUPPORTING | 0.9 | 65 → 80 → 90 |
| communication | SUPPORTING | 0.8 | 60 → 75 → 85 |
Hard gate: code-quality must reach 50 (SDE 2), 65 (SDE 3) or 75 (Staff).
Forward Deployed Engineer
| Competency | Tier | Weight | Target: SDE 2 → SDE 3 → Staff |
|---|---|---|---|
| communication | MUST_HAVE | 1.5 | 78 → 93 → 95 |
| problem-solving | CORE | 1.3 | 74 → 89 → 95 |
| customer-empathy | CORE | 1.2 | 72 → 87 → 95 |
| technical-depth | CORE | 1.1 | 68 → 83 → 93 |
| cross-functional | CORE | 1.0 | 68 → 83 → 93 |
| ai-collaboration | SUPPORTING | 0.8 | 62 → 77 → 87 |
Hard gate: communication must reach 55 (SDE 2), 70 (SDE 3) or 80 (Staff).
Notice what changes between roles. The Backend rubric puts code-quality behind the gate; the Forward Deployed rubric puts communication there and adds customer-empathy as a scored competency in its own right. A strong backend engineer applying to a forward-deployed role is not being scored on the same thing at all — which is exactly why applying to the role that matches how you actually work outperforms applying to the one with the better title.
What this means for how you prepare
Find the MUST_HAVE and clear it first. It is a gate, not a weight. Being outstanding at four competencies and below the line on the gated one produces a rejection. If you are targeting an agentic AI role, your AI collaboration has to be visible and deliberate — being a strong Python engineer who happens to use Copilot will not clear an ai-collaboration gate of 70.
Weights tell you where to spend the interview. In the Backend rubric, technical-depth and code-quality together carry 2.6 of about 5.5 total weight. Time spent on failure modes, data modelling and change safety is worth roughly twice time spent on communication polish. In the Forward Deployed rubric the arithmetic inverts.
The band gap is where most rejections live. Look at the Backend jump from SDE 2 to SDE 3 on technical-depth: 75 to 90. That is not "a bit more experience", it is a different kind of answer. A 75 explains how the system works. A 90 explains how it fails, what you would have to see to believe it was failing, and what you would trade away to fix it.
Communication is scored even where it is SUPPORTING. In every rubric we run, the description is some version of "writes down why, not just what". Reasoning you do not say out loud cannot be scored. This is the single most common avoidable loss.
Why publish this at all
The standard argument against publishing a rubric is that candidates will game it. In practice, the things you have to do to "game" this rubric — reach real depth on your claimed systems, notice missing requirements, verify what the model tells you, explain your reasoning — are indistinguishable from being good at the job. That is the test of whether a rubric is any good.
The opposite policy has a worse failure mode. A hidden rubric advantages candidates with insider access: people who know someone at the company, who have seen the internal levelling guide, who have been coached. Publishing it removes an advantage that had nothing to do with skill.
FAQ
What do AI interviewers score you on?
On named competencies with defined weights — typically technical depth, problem-solving, code quality, communication and AI collaboration — rather than on a single overall impression. Each competency has a target level for the seniority band, and at least one is a hard gate with a minimum you must clear regardless of your other scores.
What is a MUST_HAVE competency?
A MUST_HAVE competency carries a hard minimum score. Falling below it is disqualifying on its own, independent of the weighted composite. For a backend role the gate is code quality; for a forward-deployed role it is communication. Identify the gate for the role you are applying to and clear it first.
Do senior candidates get a different interview?
Not a different interview — a different bar. The competencies, weights and questions are the same across SDE 2, SDE 3 and Staff. The target level for each competency and the gate minimum rise with the band. This is what makes candidates in the same pool directly comparable.
Can I see my own score?
On the InterviewLM marketplace, candidates receive strengths and a percentile-style standing rather than a raw number. The full session is replayable, which matters more than a number: a result you can inspect is a result you can argue with.
Does knowing the rubric let me game the interview?
Only in the sense that knowing the syllabus lets you game an exam that tests understanding. Clearing these gates requires real depth on systems you actually built, real trade-off reasoning, and real verification of AI output. If those can be faked for 45 minutes under adaptive follow-ups, the rubric is wrong and we want to know.
Which competency do most candidates fail on?
Depth under follow-up, and the reason is usually not knowledge but narration. Candidates answer the first question well and then treat the second and third "why" as hostility rather than as the actual test. The follow-up chain is where the band is decided.
Sources
- Rubric data: live InterviewLM marketplace cohort configuration, five engineering roles across three bands, as of August 2026
- Structured-assessment consistency: SHRM Talent Acquisition Benchmarking Report
Want to be scored against this rubric? [Open marketplace roles](/jobs/interviewlm/agentic-ai-engineer) — free for candidates, one sitting, and you keep the evidence card whether or not you are hired.