How to Evaluate Prompt Engineering Skills
The difference between a good and great AI-assisted developer often comes down to how they communicate with AI. Here's how to assess this critical skill.
Prompt engineering has emerged as a critical skill for modern developers. But what does "good" prompting actually look like, and how do you evaluate it in a hiring context?
Why Prompt Quality Matters
The quality of AI responses depends heavily on input quality. A developer who can write clear, effective prompts will:
- Get useful responses on the first try (saving time)
- Receive more accurate and relevant code suggestions
- Avoid debugging AI-generated mistakes
- Work more efficiently with any AI tool
In our data, candidates with high prompt quality scores solve problems 40% faster on average.
Anatomy of an Effective Prompt
1. Context Setting
Great prompts establish context upfront:
Poor: ``` write a function to parse dates ```
Better: ``` I'm working on a TypeScript backend with ISO 8601 date strings coming from an API. Write a function to parse these dates into JavaScript Date objects, handling timezone information correctly. ```
The better prompt specifies:
- Language/environment
- Input format
- Specific requirements (timezone handling)
2. Constraints and Requirements
Explicit constraints produce better outputs:
Poor: ``` make it handle errors ```
Better: ``` The function should return a Result type with either the parsed Date or a specific error message. Handle invalid formats, out-of-range values, and null/undefined inputs. ```
3. Output Format Specification
Telling AI what you want the response to look like:
Return only the function implementation without explanation. Include JSDoc comments for the parameters and return type.
4. Examples When Helpful
For complex or ambiguous requirements, examples clarify:
Input examples:
- "2024-01-15T10:30:00Z" → Date object in UTC
- "2024-01-15" → Date object at midnight UTC
- "invalid" → Error result
Red Flags We Watch For
When evaluating candidates, these patterns suggest weak prompting skills:
1. Vague, Single-Word Prompts "help" or "fix this" without context suggests the candidate doesn't understand how to communicate effectively with AI.
2. No Iteration When AI gives a wrong answer, candidates who just say "that's wrong" instead of explaining what's wrong will get the same wrong answer again.
3. Prompt Stuffing Throwing everything at AI in one huge prompt often produces worse results than a focused, iterative approach.
4. Copy-Pasting Error Messages Alone "Error: Cannot read property 'x' of undefined" without any context about what they're trying to do.
Green Flags That Indicate Skill
1. Progressive Refinement Starting with a high-level request, then drilling down based on responses:
- "Implement user authentication for this Express app"
- "Add session management using JWT tokens"
- "Include refresh token rotation for security"
2. Thoughtful Follow-Ups "That looks good, but we also need to handle the case where the token has expired but the user is still on the page."
3. Questioning AI Responses "Are you sure about using synchronous file operations here? That might block the event loop."
4. Providing Feedback "The variable names aren't descriptive enough. Can you use more semantic names like 'userProfile' instead of 'data'?"
How We Score Prompt Quality
Our system evaluates prompts across several dimensions:
| Factor | Weight | Description |
|---|---|---|
| Clarity | 25% | Is the request unambiguous? |
| Context | 20% | Does it include relevant background? |
| Specificity | 20% | Are requirements clearly stated? |
| Iteration Quality | 20% | Do follow-ups effectively refine output? |
| Efficiency | 15% | Are they getting good results without unnecessary back-and-forth? |
Teaching Prompt Engineering
If you hire someone with weak prompting skills, the good news is they can improve. Here are key principles to teach:
1. Be explicit about everything - Don't assume AI knows your context 2. Break complex tasks into steps - Sequential prompts beat one giant prompt 3. Provide examples - Especially for format or style requirements 4. Iterate thoughtfully - Each follow-up should add new information 5. Verify outputs - Never trust AI code without review
The Future of Prompting
As AI models improve, prompting will evolve. Models are getting better at inferring context and handling ambiguity. But the fundamental skill—clear communication—will always matter.
The developers who can clearly articulate what they want will always get better results, whether from AI or from human collaborators.
See prompt evaluation in action with our [AI Scores documentation](/docs/ai-scores), or [try an assessment](/auth/signup) yourself.