Sample interview transcript: Maria Santos for Senior Python Developer

Hi Maria, thanks for making the time. I'd like to start with something recent instead of walking through your CV line by line. What's the backend work you've spent the most meaningful time on lately?

What made that service more involved than a fairly standard CRUD backend?

Two main things. First, the input was messy. We had webhook bursts from external systems, duplicate deliveries, and recruiters expecting the UI to reflect changes almost immediately even when some downstream work took longer. Second, a few of the downstream steps were expensive or failure-prone, like document extraction and, later on, a tightly constrained summary feature. Early on, too much happened synchronously in the request path, so a slow integration or a temporary provider issue leaked straight into the recruiter experience. We had to split the problem properly: keep the API path fast and durable, and move the variable work into background processing without losing a clear audit trail.

When you say you split it properly, how did that show up in the data model, especially with sensitive candidate data mixed into the workflow?

We were quite deliberate about that. I kept a core applications table for workflow identity and stable references, an application_events table for business-relevant state transitions, integration_runs for inbound processing attempts, and a separate candidate_private table for fields we treated as more sensitive. That separation wasn't just tidy modelling. It made access control, deletion work, and service boundaries much simpler. In SQLAlchemy I avoided turning everything into one very generic event store because we'd found that too abstract for day-to-day changes. Instead, we stored explicit workflow state where the application needed it, and append-only events for actions we needed to explain later to customers or support. We also enforced unique constraints on external source identifiers and idempotency keys so duplicate ingest couldn't quietly fork the state.

Did you design it that way from the start, or did a failure push you there?

A failure definitely sharpened it. Early on, one ATS retried the same webhook several times after a timeout, and one of our integration paths wasn't enforcing the external identifier tightly enough. We accepted more than one copy. On top of that, we were persisting the application row and publishing follow-up work separately, so we had cases where the row existed but the enrichment never ran. That was the point where I introduced a transactional outbox table and stricter idempotency handling. The request transaction wrote the application change and the outbox entry together, and a dispatcher moved the work to Service Bus. From there, consumers treated messages as commands that could be replayed safely.

Once you had that in place, how did you decide what stayed in the request and what became worker-driven?

My rule of thumb was pretty simple: if the recruiter was waiting and we could complete the work reliably within a tight budget, it stayed synchronous. If it involved external variability, fan-out, or something we might reasonably retry, it moved out. So request validation, the durable state change, and the immediate response stayed in FastAPI. Enrichment, notifications, document extraction, and later the summary generation became Service Bus-driven tasks. For retries, we classified failures rather than treating them all the same. A rate limit or network timeout got exponential backoff with jitter. A bad payload or validation error did not get endless retries; it went to dead-letter quickly with enough metadata to inspect. For a couple of high-volume external calls we used asyncio with bounded semaphores in the worker so we could parallelise safely without hammering providers or exhausting our own resources.

Give me a production incident where that design was tested for real.

One memorable one was a Monday morning latency spike just after a customer enabled a broader integration scope. API p95 rose sharply, worker lag followed, and we started seeing more lock renewals on Service Bus messages. The root cause turned out to be connection pool pressure between the API and worker workloads. We'd increased worker concurrency a week earlier after a benchmark, but the benchmark didn't reflect the real production mix of queries. Because we had OpenTelemetry traces joined up across the API, the outbox dispatcher, and the workers, we could see time building around database acquisition and then around a particular enrichment query path. The short-term fix was lowering worker concurrency and scaling the service. The longer-term fix was reducing one expensive join, setting clearer pool limits per process, and separating a read-heavy worker path so it contended less with the write-heavy side of the workflow.

What about your observability setup helped you diagnose that quickly, and what did you realise was missing?

The biggest help was consistent trace propagation and structured logging. We put request IDs and tenant or integration identifiers into logs, but we were careful not to log candidate content or free-text notes. On top of traces we tracked the usual RED metrics for the API, plus queue depth, dead-letter counts, worker success and retry counts, and a small set of business counters like import completion delays. We also had alerts around p95 latency and backlog growth. What we didn't have at that point was good alerting around lock renewal failures on longer-running worker tasks. We could see the symptom in traces afterwards, but we weren't getting a clean early warning. I added that after the incident, along with dashboards that separated CPU saturation from database wait time so we could tell more quickly where the pressure actually was.

How do you test a system like that without ending up with a huge end-to-end suite nobody wants to maintain?

I'm pretty deliberate about test boundaries. I want unit tests where the business rules are genuinely local, like state transition validation or retry classification. I want integration tests with real PostgreSQL for anything query-heavy, migration-heavy, or transaction-heavy, because that's where ORM assumptions tend to break. For the API layer we use httpx-based tests around the main contracts and error handling. For the async flows, I don't try to prove the whole distributed system with giant end-to-end tests. Instead, we test the outbox and consumer logic in process, use contract-style tests around the message envelope, and keep a smaller smoke suite in a shared Azure environment to catch configuration, permissions, and wiring issues. GitHub Actions runs the main suite on every PR. The point is confidence for small, reviewable changes, not chasing a coverage number.

You said earlier that you handle the operational side. How has that translated into helping other engineers, not just solving incidents yourself?

A lot of it is making the risky parts visible before they become incidents. I've paired with engineers on query plans, SQLAlchemy session usage, and idempotent consumer patterns because those are areas where small mistakes create a lot of production work later. I also wrote a lightweight review checklist for our team around transaction boundaries, logging fields, and whether a change introduces a privacy risk. It's not heavy process; it's mainly a way to keep the same questions in view during reviews. Over time that helped newer engineers get more comfortable owning backend changes without relying on someone else to spot every operational footgun.

Can you give me a cross-functional example where that kind of thinking changed the shape of the feature?

A good one was the recruiter timeline feature. The initial ask was for a very detailed user-facing history of everything that had happened to an application. If we'd exposed a fully generic event log, it would have included too much internal noise and the API would have been hard to keep stable. I worked with product and design to define which events were actually meaningful to recruiters, which events should stay internal, and where we needed explicit audit data for support. That gave us a simpler public API, a clearer data contract, and enough internal detail to investigate problems without turning implementation details into part of the user-facing model.

Is there another project that shows how you work when you're not starting with a blank sheet of paper?

Yes, at CasePilot SaaS. That environment was less about designing one service from scratch and more about improving a set of existing Python services that handled case intake, document workflows, and audit trails. I worked first in Flask and later in FastAPI. A big part of that role was tightening up PostgreSQL schemas, tuning slow queries, and standardising how we used SQLAlchemy across services so the persistence layer was more predictable. I also helped push for smaller CI-driven changes with API and integration tests in GitHub Actions. It was useful experience because it forced me to improve systems that already had history and constraints, not just build the version I'd choose on day one.

What did standardising SQLAlchemy usage mean in practice there?

Mostly reducing inconsistency that had crept in as the codebase grew. Different services were handling session scope differently, some queries loaded more than they needed, and a few patterns made it too easy to hide inefficient joins until production traffic exposed them. I introduced clearer conventions around session ownership, relationship loading, and where raw SQL was justified instead of layering more ORM logic on top. I also used review and pairing to explain why a pattern was risky rather than just replacing it. That helped because people understood the trade-off, not just the rule.

Your CV also mentions a summary feature. What did you actually ship there, and what guardrails mattered most?

My production experience with LLMs is real but intentionally narrow. We shipped a recruiter-facing application summary feature built from selected structured inputs and sanitised text, not a free-form assistant over all candidate data. Before prompts were sent, we redacted obvious personal details. We stored prompt and model versions for auditability, logged the generated output in a controlled way, and required human review before the summary became visible to users. More broadly, on the privacy side I try to make the right system decisions early: keep more sensitive candidate fields separate from the wider workflow model, restrict which services can read them, avoid putting content into logs and traces, and build retention and deletion jobs that remove or anonymise data while preserving an auditable record that the action took place. I'm comfortable with the backend integration, privacy, and guardrail side of LLM work. I wouldn't present myself as someone doing model tuning or deep evaluation research.

That's very clear, thank you. I appreciate how specific you were about the architecture, the production trade-offs, and the limits of what you've done with LLMs. That's everything I needed from this conversation, so I'll wrap up here and we'll come back to you with next steps.