Technical Interviews are realigning with reality through AI Fri Aug 08 2025 AI Tools will be a part of interviews at large orgs in the future, they make software brownfield interviews possible without taking too much time. -------------------------------------------------------------------------------- Technical Interviews are realigning with reality through AI =========================================================== Published Aug 7, 2025 - 5 min read Since the 2000s, Google introduced a data structures and algorithms [L1] focused interview process [L2]. To them, getting the smartest people together to make a world-scale product needed a rigorous filter to gate-in only the quality candidates they sought. Except, everyone else followed and we got ridiculous prompts like "How many ping pong balls can fit in a 747 [L3] " and "How many man -holes are there in New York City", not withstanding textbook responses like "Implement heap sort on this whiteboard." These challenges aren't for receiving an accurate answer – they're for starting a conversation where the interviewee asks questions to qualify an answer grounded in the conditions set by the interviewer. Questions that involve rote memorization or preparation on leet-code are hardly valuable now that a dated Llama3.2 2GB [L4] model can emit a working copy of heap sort under ten seconds. Interviews are shallow windows to see into a candidate's capability. Given the time expense to both parties, why waste our time on the round-about conversation about ping pong balls? Meta has been dog-fooding Llama [L5] in their development process for a while now. Except, Llama hasn't been at the forefront and that has held them back. After poaching researchers and engineers in this space for ungodly amounts of money [L6] (archived [L7]) and being told how far behind they were, their workforce is getting to touch tools candidates might already have experience with. At Meta, Interviews are changing to involve working on a project with AI tooling [L8] (archived [L9]) where the interviewer either supervises or peer programs with the candidate. I like this approach. I like approaches that involve a fair simulation the employer's environment and involves a since-resolved production issue. /[cendyne: fire-everything-is-fine]--------------------------------------------\ | In fact, my chosen take-home project for candidates distills the issues I | | had to address on friday night outages while I wasn't on-call. | \------------------------------------------------------------------------------/ Real working conditions involve brownfield development [L10], where there's plenty of code with lessons learned over time and with plenty of distractions for a candidate to fall into. Rather than spend hours digesting a few modules and how they interact, the tools we have today are adequate at ingesting a few thousand lines and then emitting an overview of what exists and how it links together in under a minute. This speed up makes brownfield interviewing possible. The tooling we have today is famously good with greenfield [L11] work, where there's little to no code yet. It's what keeps the "AI bros" (including vendors) pumping up the hype over generating one-shot web applications and games. At the same time, these tools struggle with brownfield, where the context window has to contain all the bespoke code, patterns, and expectations instead. While context windows can grow to 1 million in length like Google's Gemini, every current LLM out there struggles to find the needle in the haystack beyond 32,000 tokens. /[cendyne: citation-needed]----------------------------------------------------\ | I don't recall what paper shares this number, only that it's been cited | | several times by the sources I often listen to and read. | \------------------------------------------------------------------------------/ /------------------------------------------------------------------[jacobi: hi]\ | Future editor here, several papers are written on this subject freely | | available on arXiv! | | | | * Lost in the Middle: How Language Models Use Long Contexts [L12] | | * Why Does the Effective Context Length of LLMs Fall Short? [L13] | | * Reasoning on Multiple Needles In A Haystack [L14] | \------------------------------------------------------------------------------/ If a candidate can get these tools to accurately produce good code in a brownfield environment in front of an interviewer, and they understand what is produced, that shows far more candidate value than someone who can calculate ping pong balls in an airplane without costing exceptional amounts of time for interviewer and candidate to understand one another. /[cendyne: glue]---------------------------------------------------------------\ | What about their actual skill though? | \------------------------------------------------------------------------------/ /[cendyne: time-for-work]------------------------------------------------------\ | Is our job to be an artisan carpenter that constructs a sculpture that only | | we can feel pride in, or to build functioning automations that operate the | | business? | \------------------------------------------------------------------------------/ The skill to build what's needed for a business is additive to actual coding-in- some-language-or-environment competency. Most of our time at the senior level is spent reading, thinking, and communicating, not typing code and fumbling with compilers and type checkers. What we interview for should be what we actually do. That isn't to say some new graduate is suddenly a senior developer with LLM tools in hand. I believe they'll find it more difficult to get competent in writing code for the rest of their life while using these tools. But even without that code-competency, appropriate use of agents can make someone effective faster and therefore deliver for the business faster. Think of it another way, a business getting a new feature to market and opening $15,000 in additional revenue thanks to LLM technology won't mind if they spend $50 / day logging to AWS CloudWatch because the LLM left in console.trace in the database query function. If businesses actually cared about optimizing operating costs, we wouldn't have Ruby on Rails. /[cendyne: i-never-asked-for-this]---------------------------------------------\ | This is so true that businesses will shave headcount so deeply they will | | become incapable of optimizing their operating costs. Ask me how I know. | \------------------------------------------------------------------------------/ Businesses need as much time to present and iterate on new products to align their products to the market. How pleasant or well-functioning it is internally is of no concern to revenue-generating departments like sales. Even if the trade off is poor-performing or insecure code, the speed to get to and iterate in the market is worth biting on the AI tool hook. Given the pressure to develop and release quickly, finding candidates that can match the same expectation with the tools on the market today is imperative to their growth. And finally, interviews are changing to match the reality businesses are targeting, developers that can contribute to a brownfield environment. /[cendyne: head-empty]---------------------------------------------------------\ | Though, if you have a "certification in prompting", I'll probably dismiss | | your resume 😉 | | | | I cannot trust vibe coders. | \------------------------------------------------------------------------------/ -------------------------------------------------------------------------------- [L1]: https://techdevguide.withgoogle.com/paths/data-structures-and-algorithms/ [L2]: https://www.geeksforgeeks.org/dsa/commonly-asked-data-structure-interview -questions-set-1/ [L3]: https://www.youtube.com/watch?v=eMcOd4sq2NM [L4]: https://ollama.com/library/llama3.2 [L5]: https://www.llama.com [L6]: https://www.bloomberg.com/news/articles/2025-07-09/meta-poached-apple-s- pang-with-pay-package-over-200-million [L7]: https://archive.is/0IT3H [L8]: https://www.wired.com/story/meta-ai-job-interview-coding/ [L9]: https://archive.ph/VVfCL [L10]: https://en.wikipedia.org/wiki/Brownfield_(software_development) [L11]: https://en.wikipedia.org/wiki/Greenfield_project [L12]: https://arxiv.org/abs/2307.03172 [L13]: https://arxiv.org/html/2410.18745v1 [L14]: https://arxiv.org/html/2504.04150v1