Behavioral Scenario #26
The Ghost in the Machine
“You've inherited a codebase where 40% of the code was written by AI agents six months ago. Nobody knows why it was built the way it was.”
The Situation
You've joined as the new Principal Android engineer for a team that moved fast last year using AI coding agents for a major feature. The engineers who used the agents have since moved to other teams. The feature works, but there have been three unexplained performance regressions and one data corruption bug. You open the code and find no decision records, no comments explaining non-obvious choices, extensive AI-generated boilerplate, and subtle patterns that look correct but have no clear rationale.
Context
- The feature works in normal operation but has produced three unexplained performance regressions
- One data corruption bug has occurred — root cause not yet understood
- The engineers who built the feature with AI agents have moved to other teams and aren't available
- No ADRs, design docs, or decision records exist — rationale for key choices is unknown
- The team is scared to change anything in the codebase for fear of triggering more regressions
- The product team wants new features built on top of this foundation within the next quarter
The Question
“Tell me about a time you had to take ownership of a system you didn't build and didn't fully understand.”
Response Options
One of these is the strongest response. The others reflect common approaches with real trade-offs.
I recommended a full rewrite — you can't confidently build on a foundation you don't understand, and the data corruption bug confirms the codebase has integrity problems that aren't safe to work around.
I took an instrumentation-first approach: before changing anything, I added comprehensive observability — metrics, traces, and structured logging — to understand the actual runtime behavior. Then I wrote characterization tests that document what the system currently does, not what it should do. The characterization tests became the safety net and the implicit decision record. Once I had observability and characterization coverage, I could start mapping the code's implicit contracts and extracting the rationale from the behavior — then write it down as explicit decision records.
I decided to add new features carefully on top of the existing code, staying away from the areas that had caused regressions and hoping the bugs would remain isolated as long as we didn't touch those modules.
I did a full manual audit of every AI-generated file, annotating each one with my best assessment of the intent and the rationale based on code inspection.
The Debrief
Why the Best Response Works
Answer B works because it establishes the safety net before any changes and extracts the ground truth from runtime behavior rather than static code inspection. Characterization tests document what the system actually does — which is the only accurate source of truth when rationale is missing. The instrumentation-first approach is not just cautious — it's the prerequisite for any safe modification of a system whose invariants are unknown.
What to Avoid
Rewriting from scratch discards working code along with unknown bugs — and produces new unknown bugs because the implicit constraints of the original system aren't documented. Manual annotation without observability data produces documentation of what you think the code does, not what it actually does in production. The runtime behavior is the ground truth.
What the Interviewer Is Probing
The interviewer is evaluating whether you can take ownership of a system under conditions of deep uncertainty — without either freezing (doing nothing) or overreaching (rewriting everything). The instrumentation-first approach is the signal that you know how to build understanding incrementally from observable evidence.
SOAR Structure
**Situation:** Inherited an Android codebase where 40% of code was AI-generated with no decision records; three unexplained performance regressions, one data corruption bug; original engineers unavailable; product team wants new features in one quarter. **Obstacle:** Team scared to change anything; no rationale for non-obvious patterns; data corruption bug root cause unknown; new features required on top of an unsafe foundation. **Action:** Instrumentation-first: added comprehensive observability before touching any code; wrote characterization tests documenting current behavior; mapped implicit contracts from runtime behavior; extracted and wrote formal ADRs from observed behavior. **Result:** Data corruption bug root cause identified through observability data (an implicit idempotency assumption violated in one specific edge case); three regression causes identified and fixed; new features built on the now-understood foundation; ADRs established as a team requirement for all future AI-assisted development.
The Learning Arc
"AI-generated code without decision records is the new legacy code problem — except it arrives faster and at higher confidence levels than hand-written legacy code. The fix isn't to rewrite it. It's to first make the implicit explicit: instrument it, characterize it, and write down what you learn. The code knows what it does — you just have to ask it the right questions."
IC Level Calibration
Be cautious about changing the code, add targeted tests for the specific areas that have caused regressions, and document your findings as you investigate.
Take an instrumentation-first approach before any changes, write characterization tests that document current behavior as the safety net, and start mapping the code's implicit contracts from the observability data.
Make the implicit explicit at scale — extract the decision record from the observed behavior, write it down as formal ADRs, and establish that this never happens again by making decision records and observability requirements part of the team's definition of done for any AI-assisted feature development.
Company Calibration
Netflix
Chaos engineering: understand failure modes before trusting stability
Design docs as living artifacts: absence is a risk signal, not a neutral state
Amazon
Operational Readiness Review: you must understand what you're operating
Stripe
Idempotency and correctness: you must understand invariants before touching financial code
Want to pick your response and see the full analysis?
Practice This Scenario Interactively