How I Built a Fine-Tuned Interview Simulator with Mistral-7B

I'm Shubham Pareek, and I built this project because most interview prep products optimize for comfort instead of signal. I wanted something closer to an actual interviewer loop: adversarial when needed, grounded in resume context, and observable enough that I could debug model behavior like any other production system.

I built Resume Griller to simulate interview pressure in a way that is actually useful for candidates and recruiters. The core requirement was not just "good responses" from the model, but targeted signal extraction:

• detect resume gaps,
• ask escalating follow-ups,
• keep the conversation grounded in context,
• and do all of this with low latency.

System sketch

The system uses a hybrid pipeline:

1. A fine-tuned model for interview behavior and questioning style.
2. A retrieval layer for factual grounding and candidate-context recall.
3. A policy layer for prompt injection checks and bias/unsafe output detection.

def choose_generation_path(question_type: str, risk_score: float) -> str:
    if risk_score > 0.75:
        return "guarded"
    if question_type in {"follow_up", "behavioral_gap"}:
        return "fine_tuned"
    return "general_llm"

What mattered in production

The biggest lesson: model quality alone was not the bottleneck.

Latency, retrieval precision, and failure handling mattered more than squeezing out a few benchmark points. The sections below cover:

• the LoRA/PEFT training setup,
• why I mixed Groq + vLLM execution paths,
• and how I tracked hallucination regressions after prompt changes.

The Problem With Generic Interview Prep

Most interview prep tools are too polite. They generate generic questions, accept shallow answers, and move on quickly. That is useful for confidence-building, but not for revealing actual weak spots.

What I wanted was a system that behaves more like a strong interviewer: one that notices ambiguity, presses on incomplete answers, and adapts the next question based on what the candidate just missed. That difference is what turns a demo chatbot into a practice tool that creates measurable improvement.

Architecture Overview

The system started with Mistral-7B as the base model, then used LoRA fine-tuning to teach interviewer behavior patterns rather than general knowledge. Candidate-specific context (resume details, projects, technologies, experience timeline) is embedded and stored in ChromaDB, then retrieved at inference time to ground the questions.

For serving, I split the inference path. Groq handled low-latency generation for interactive turns, while heavier or specialized tasks could be routed to a separate path for more control. A FastAPI backend on GCP orchestrated request flow, retrieval, validation, and response formatting.

This architecture let me iterate on behavior, retrieval, and infrastructure independently instead of coupling every change into one model-serving path.

The Mistake-Guided Framework

The core idea behind the Mistake-Guided Framework is simple: the system should remember where the candidate struggles and increase pressure on those areas.

Instead of treating each question as independent, the platform tracks error patterns such as incomplete metrics, weak tradeoff reasoning, shallow system explanations, or unclear ownership stories. If a user repeatedly misses one category, the interviewer increases the frequency of that category and generates sharper follow-up prompts.

This creates a feedback loop that feels closer to a real interview, where weak answers attract more scrutiny rather than being ignored.

Latency Optimization

The 52% latency reduction came from avoiding a monolithic "one model does everything" setup. In early versions, every step went through the same generation path, which added unnecessary delay even for lightweight operations.

By splitting responsibilities, I could route fast conversational turns to Groq while keeping other components (retrieval, validation, policy checks, and heavier generation paths) decoupled. I also reduced token usage by tightening prompt templates and only passing the most relevant resume chunks instead of large context dumps.

The result was not just faster responses, but a more stable system under load because bottlenecks became visible and measurable.

What I'd Do Differently

If I were rebuilding it today, I would invest earlier in offline evaluation for interviewer quality, not just latency and correctness. It is easy to measure response speed; it is much harder to measure whether the follow-up questions are usefully difficult.

I would also formalize a clearer annotation schema for mistake categories before collecting more training data. That would make the fine-tuning dataset more consistent and reduce drift in how the system interprets "weak answers."

Finally, I would add stronger longitudinal analytics for users, so the system can show progress over time instead of only scoring a single session. For recruiters and serious candidates, trend lines are more compelling than one-off feedback.

I would also invest earlier in synthetic-but-reviewed evaluation sets for follow-up quality. Latency metrics and answer correctness are straightforward to chart, but the real product value comes from whether the system asks the next question well. Building those evaluation sets sooner would have made iteration faster and made model upgrades safer to ship.