The hidden queue

Every engineering team has a version of this story.

Your roadmap looks healthy. Feature velocity looks fine. Sprint retros are tolerable. But your engineers are still losing chunks of every day to things that were never planned: waiting 16 minutes for CI to tell them a typo failed linting. Manually running six database commands because the snapshot process lives in someone’s Slack history. Bumping a dependency because Dependabot fired another alert at 3pm on a Friday.

That is the hidden queue. It is not on your roadmap, it is not in your sprints, and it is costing you more than most feature work.

Most teams do not ignore this queue on purpose. It just keeps losing in prioritization because each item feels small in isolation. “Fix the flaky test” is never going to beat “Ship the dashboard redesign” in a planning meeting. But those small items compound. A 16-minute CI wait happens 20 times a day across 5 engineers, and that adds up to over 26 hours of dead time per week - time where nobody is building anything or even thinking about the problem, just staring at a spinner.

Late last year, we changed one rule: if something keeps burning team time, it is roadmap work - not “tech debt we’ll get to someday,” but prioritized work with owners and metrics.

In the 90-day window from December 1, 2025 through February 28, 2026, that rule turned into 57 major non-product changesets across backend, frontend, and monolith tooling repos: 56 major PRs and 1 major direct commit, touching 1,303 files with 120,929 additions and 31,170 deletions.

The point of those numbers is that a whole class of work moved from “someday” to “done” - and we did it without pausing product delivery.

What we shipped this quarter

Here is the breadth of what moved in those 90 days, all in parallel:

• CI architecture - Rewired how we decide what checks to run, when, and at what cost
• Testing platform - Built E2E and integration test coverage in phases, replaced flaky waits with deterministic ones
• Local development - Made git worktrees reliable so agents and engineers can run multiple branches in parallel without collisions
• Operations safety - Took processes that lived in people’s heads (snapshots, support shells, admin actions) and gave them proper guardrails and audit trails
• Security remediation - Closed CVEs and Dependabot alerts in planned waves instead of random fire-drills
• Debt retirement - Squashed tangled migration graphs, removed dead Slack/Teams integrations
• Open-source skills - Turned repeating engineering patterns into reusable agent skills and open-sourced them

Most teams can point to one thing they improved. “We made CI faster.” “We added more tests.” Seven high-friction systems in parallel while keeping review quality, rollback safety, and product delivery intact is a different kind of claim. So the obvious question is: how?

Why this was possible now and not a year ago

Karpathy put it bluntly: coding agents basically did not work before December 2025, and they basically work now. He described giving an agent a full system setup task - SSH keys, vLLM, web dashboard, systemd services - and it came back 30 minutes later with everything working. He did not touch anything.

That matched what we were seeing. As recently as October 2025, the same tools felt like expensive autocomplete. By December, a new generation of models had shipped and changed that completely. Mitchell Hashimoto went through the same arc - from skeptic to building his entire Ghostty workflow around agents, with the key insight that you have to separate planning from execution and let the agent self-correct. Paul Graham framed the implication: “In the AI age, taste will become even more important. When anyone can make anything, the big differentiator is what you choose to make.” Simon Willison figured out the compounding angle: “Coding agents mean we only ever need to figure out a useful trick once.”

They were all talking about the same shift from different angles, but none of them say loudly enough what actually caused it: the models got qualitatively better.

Claude Opus 4.5, Opus 4.6, Codex 5.2, Codex 5.3 - these are not incremental improvements. They can hold entire systems in context, reason about architectural tradeoffs, catch multi-tenant security patterns, and produce code that is reviewable on the first pass. That kind of jump is what made No Code by Hand our default working mode since December.

Our team learned the differences through daily use, not benchmarks. We got opinionated fast, and our honest assessment does not look like any vendor’s marketing page:

Claude models (Opus 4.5, Opus 4.6, Sonnet) are our workhorse for execution. They are fast - genuinely fast, even at high thinking levels. They are great at quick fixes, running skills that require less deliberation, verifying and formatting large numbers of files, and chewing through well-scoped tasks at speed. The Claude Agent SDK (available in both Python and TypeScript) is mature and well-designed - it gives you the full agent runtime with tools, subagent orchestration, MCP support, and granular permissions out of the box. If you are building custom workflows, that SDK matters a lot.

The tradeoff: Claude models can be eager. They will take shortcuts on medium to large features, skip files, and miss critical business logic if you are not watching. They also consume tokens fast - you can burn through a significant budget in a short session. You get results quickly, but you end up verifying more.

Codex models (Codex 5.2, Codex 5.3) are where we go for depth. Planning, architectural reasoning, thorough code review, anything where you need the model to actually think through the full problem before writing code. Execution is slower, but the output is more satisfying - it will not skip critical business logic or take shortcuts the way Claude sometimes does. It respects the complexity of what you are asking it to do.

The tradeoff: Codex is weaker with tool use compared to Claude models. The SDK story is thinner - there is a Codex SDK (TypeScript only), but it is an orchestration wrapper for controlling a local Codex agent, not a general-purpose agent-building runtime like Claude’s. No Python equivalent. Token limits, on the other hand, are very hard to hit - you can run long, deep sessions without worrying about context exhaustion.

In practice, we mix both. A lot of our best work comes from running Claude and Codex together, or multiple instances of the same model, each with a clean context scoped to a specific piece of the problem. One model plans while another executes, or one reviews what the other wrote. We split large tasks across separate sessions so no single context gets polluted with too much state.

With Codex specifically, we use steer mode and queue mode very tactically depending on the task:

• Steer mode for ambiguous work. Anything that involves architectural decisions, multi-tenant security logic, or unfamiliar parts of the codebase - we steer. Example: when building the admin safety model (the system that prevents admins from locking themselves out), we started Codex in steer mode, pointed it at the permission hierarchy, course-corrected twice when it tried to flatten the role graph, and guided it through the edge cases where a superadmin demotes themselves. The final implementation was clean because we were in the loop at every decision point instead of reviewing a finished mess.
• Queue mode for well-scoped, parallelizable work. When the task is clear and the scope is bounded, queue mode lets us fire off multiple Codex tasks and review them as they land. Example: during the CI migration, we queued separate tasks for each workflow file rewrite - linting pipeline, test matrix, deployment gates - each with its own clean context. Five tasks running in parallel, each one scoped tightly enough that the model did not need steering.
• The tactical switch. We start most sessions in steer mode. If the first few interactions show the model understands the problem and the scope is narrowing, we switch to queue mode and let it run. If it starts drifting, we pull it back to steer. We re-evaluate steer vs queue mode throughout the session based on drift, clarity, and risk.

Picking the right model for the right task - and knowing when to combine them - is itself a skill that takes time to develop, and teams that treat all models as interchangeable are missing most of the value.

The biggest shift is how these models want to be used. Earlier generations worked best when you gave them a prompt and left them alone. The current generation is different - they are built for interactive steering. You set direction, check in as it works, course-correct when it drifts, and guide it through the ambiguous parts rather than hoping it guesses right. OpenAI made this explicit with Codex 5.3: steer mode - where you interact with the model while it works without losing context - shipped as a stable feature, enabled by default. Anthropic’s Claude Opus 4.6 moved in the same direction with adaptive thinking, where the model adjusts its reasoning depth based on what you are asking and responds to mid-turn guidance.

This matters more than it sounds. AmpCode put it well: “the agent is no longer the bottleneck. Our ability to tell it what to do is.” A year ago the workflow was write a prompt, wait, review what comes back. Now it is closer to pair programming - you stay in the loop, steer the model toward the right answer as it works, and course-correct in real time rather than filtering through wrong answers afterward. Teams that still treat agents as black boxes they throw tasks at are working against how these models are actually designed.

Our team has taken this to its logical conclusion: we have pretty much ditched our IDEs entirely. We work strictly in terminals - Claude Code, Codex CLI, git, and shell scripts. The models are better at navigating codebases than any file tree sidebar ever was, and the terminal is where steering actually happens - no GUI layer sitting between you and the agent. Karpathy put it bluntly: “You’re not typing computer code into an editor like the way things were since computers were invented. That era is over.” What replaced it is spinning up agents, giving them tasks in English, and managing their work in parallel - but with what he calls “high-level direction, judgement, taste, oversight, iteration, and hints and ideas.” That is steering, not autopilot.

None of this means the models write perfect code or replace engineering judgment. What changed is how cheap it became to try something, evaluate it, and try again. When a rewrite takes 20 minutes instead of a day, you can try three approaches and pick the best one. You can afford to be thorough instead of cutting corners because iteration itself got cheap.

That context matters for everything that follows. The hidden queue shrank because each iteration got about 10x cheaper, so we could clear items that used to stay deferred. So we started with the one system where we could prove it.

CI: the system we could actually measure

We started with CI because it was the one system where we had clean telemetry and clear before/after boundaries. You can argue about whether a refactor “improved code quality.” You cannot argue about whether a workflow went from 942 seconds to 675 seconds. The numbers are right there.

The real problem was architectural

Before the redesign, our CI pipeline had a fundamental coupling problem. One heavy workflow path tried to answer two completely different questions at the same time:

1. Should expensive checks even run on this change? (A typo in a README does not need database-level integration tests.)
2. Is this change safe to ship? (A migration change absolutely does.)

When those two questions are coupled, you get the worst of both worlds. Low-risk changes overpay - a CSS fix waits 16 minutes for checks it does not need. High-risk changes wait behind noise - a database migration sits in the same queue as a copy change.

What we actually changed

The fix was separating qualification from execution - figuring out which checks should run before actually running them:

• Fast checks moved earlier. Linting, type checking, import validation, and test placement checks all moved into a GitHub Actions workflow that runs in under 45 seconds. If these fail, nothing downstream even starts.
• Scope classification. A file-change classifier now evaluates every PR: does this change touch backend code? Database schemas? Frontend only? Docs only? Changes that are irrelevant to deep checks skip them entirely.
• Content-aware caching. Instead of time-based cache invalidation, caches are now keyed on actual content hashes. Same code = same cache, regardless of when you last ran.
• Runtime image optimization. The CircleCI runtime image was rebuilt to eliminate bootstrap overhead - the time between “job starts” and “your code actually runs.”

The numbers

On the heavy path (CircleCI test-and-check workflow), the broad pre/post window around the runtime-image milestone gave the strongest signal:

• Median workflow wall time: 942.5s → 675.0s, a 28.4% reduction (n=218 before, n=149 after)
• Deepest check lane (Deep DB/RLS checks): 332.0s → 95.5s, a 71.2% reduction

Those are the headline numbers, but they are only half the story - because the first win came with an uncomfortable tradeoff.

Why we kept going after the first win

After the first round of CI improvements, we had an intermediate phase where runtime improved but credit consumption actually got worse. We were running more things in parallel, which made the wall clock faster, but the total compute bill went up.

Most blog posts would stop here, post the runtime chart, and declare victory. We did not, and honestly this next part matters more than the numbers above.

We kept running the loop: hypothesis → implementation → measurement → tradeoff review → tune or rollback → standardize. That loop is where cheap correction cycles made the real difference. When rewriting a CI configuration takes 20 minutes instead of a day, you can afford to try three approaches and pick the best one.

Three phases of tradeoff

The three-phase view shows the full path, including the uncomfortable middle:

• Baseline: 37.91 min critical path, 359 mean credits per paired run
• Parallel phase: 10.63 min critical path (great!), 456 credits (worse - more parallel jobs)
• Current phase: 9.08 min critical path, 232 credits (both better than baseline)

Relative to baseline, the current mode is 76% faster and 35% cheaper. The current sample in this phase comparison is still early (n=6), so we treat it as directional and use the larger pre/post windows for stronger claims.

Modeled time savings

The waiting-time model across this window estimates 21.39 hours of developer waiting time recovered, broken into three categories:

• 11.07h from faster heavy lanes (every CI run finishes sooner)
• 6.38h from early-stop on failed fast checks (if linting fails in 30 seconds, you do not wait 15 minutes for full tests to also fail)
• 3.94h from skipping irrelevant deep runs (a docs-only change skips database checks entirely)

This is a model, not direct causal proof. But 21 hours of waiting time eliminated over a few weeks is a meaningful quality-of-life improvement for a small team.

Every CI job, not just the workflow

The workflow-level numbers tell one story. The job-level numbers tell a more interesting one.

The “Deep DB/RLS checks” job - the single most expensive check in our pipeline, the one that spins up a full database and runs row-level security validations - went from 332s to 95.5s. That is a 71% reduction on the job that matters most for safety confidence.

That improvement came primarily from the runtime image rebuild. The old image was doing expensive setup work (installing system packages, configuring database connections, bootstrapping test fixtures) on every single run. The new image bakes all of that into the image layer, so the job starts with everything already in place.

It sounds boring, but it changes daily life. At 5.5 minutes you context-switch to something else and lose focus. At 1.5 minutes you just wait for it and stay in the flow. And then we looked at the trigger configuration and found something we should have caught months ago.

We were running the same checks twice

Here is a problem that is embarrassingly common and embarrassingly wasteful: GitHub fires both a push event and a pull_request event when you push to a branch with an open PR. If your CI triggers on both, you run the same checks twice on the same code.

Before we fixed this, the ratio was absurd: 399 push-triggered runs vs 22 PR-triggered runs on non-deploy branches. An 18:1 ratio of redundant work.

After dedup (#2713): 9 push runs vs 36 PR runs. A 0.25:1 ratio. Only meaningful work runs.

A few hours of investigation, hundreds of compute-hours saved over a quarter. And it is exactly the kind of thing that never gets prioritized because “CI is working fine” - as long as nobody looks at the bill.

With CI finally fast and honest, we could tackle the thing it depends on most: whether the tests themselves were worth trusting.

The test suite we could not trust

The testing work this quarter was about building a testing system that could be trusted by default and maintained without heroics, not just adding more test files.

Why phased rollout matters

The most common failure mode for test initiatives is the big-bang rewrite: someone decides “we need 80% coverage,” writes hundreds of tests in a week, and three months later half of them are flaky, ignored, or commented out. We did the opposite.

The sequence:

1. Infrastructure first. Test runner configuration, fixture management, database seeding, environment isolation - the kind of work nobody wants to do but everything else depends on.
2. Role-based E2E flows. Tests organized by user role - HR admin, manager, employee - because that is how our permission model works. Testing by role catches the bugs that actually happen in production: “this works for admins but breaks for regular users.”
3. Unit and integration depth. Once the E2E skeleton was stable, we filled in unit and integration tests underneath. The E2E tests act as a safety net while you add lower-level coverage.
4. Deterministic stabilization. The final phase was eliminating flakiness. And that brings us to one of the most universally hated problems in software engineering.

The sleep(2000) problem

If you have ever maintained E2E tests, you know this pain intimately. The test clicks a button, then waits. How long should it wait?

// This is a guess
await page.click('#submit');
await sleep(2000);
expect(page.url()).toBe('/dashboard');

Two seconds works on your laptop. It fails on CI because the server is slower. You bump it to 3 seconds. Now the test passes but the suite takes 40 minutes. Someone adds sleep(5000) “just to be safe.” Now it takes an hour. And it still flakes on Mondays when the CI machines are loaded.

The fix is simple to explain and tedious to actually do: go through every test and replace fixed waits with waits that check whether the thing actually happened.

// This is an assertion about behavior
await page.click('#submit');
await page.waitForURL('/dashboard');

The second version waits exactly as long as needed - 200ms on a fast run, 3 seconds on a slow one - and fails immediately with a clear error if the URL never changes. That one shift improved reliability, review trust, and debugging speed because failures started telling you what actually went wrong instead of “timeout after 5000ms.”

A few of the replaced waits started failing consistently rather than flaking - which turned out to be a gift, because the generous sleeps had been hiding genuine performance issues in the application. Fixing the tests ended up fixing the app.

None of that would have mattered, though, if the environment running those tests kept colliding with itself.

Why two agents could not run at the same time

If you are doing agentic coding seriously, Worktrees are the base layer for agentic coding; without isolated checkouts, parallel execution breaks down.

An AI coding agent needs its own isolated checkout to work in. It cannot share your working directory - it would stomp on your files, your running servers, your local state. Git worktrees solve this cleanly: each worktree is a separate working copy of the repo with its own branch, its own file state, and its own index. You can have an agent working on a refactor in one worktree while you are reviewing a PR in another and a second agent is writing tests in a third.

But worktrees only deliver that promise if your local development setup is worktree-aware. Ours was not. And until we fixed it, every other improvement in this post would have been bottlenecked by a broken local environment.

What was actually breaking

Every worktree tries to spin up its own local dev environment. Without isolation, that means:

• Branch A starts Docker with the default port mapping: localhost:8000 for the API, localhost:3000 for the frontend.
• An agent opens a worktree for Branch B. It also wants localhost:8000 and localhost:3000.
• Port collision. One of them fails silently or grabs a random port. The agent’s frontend is now talking to your backend running different code. Or your server crashes because the port is taken. Either way, someone wastes 20 minutes debugging a port-conflict issue that looks like an app bug and burns 20 minutes of debugging.
• Meanwhile, Branch A’s .env file says API_URL=http://localhost:8000. Branch B’s .env says the same thing. The frontend in one worktree makes API calls to a server that is running code from a completely different branch.

This blocked the entire agentic workflow. If worktrees are unreliable, agents cannot work in parallel, and the speed advantage you are counting on just collapses.

The fix

The primary checkout behaves exactly like before - same ports, same config, no surprises for the human developer. But every non-primary worktree gets deterministic port isolation: ports are derived from the worktree path, so they are stable (the same worktree always gets the same ports) and unique (no two worktrees ever collide).

We added root-level wrappers - just up, just down, just sync-api-env - so that starting, stopping, and syncing environment state became a single predictable command regardless of which worktree you are in. The frontend environment file automatically picks up the correct backend URL for whichever worktree is running. An agent can just up in its worktree and get a fully working, fully isolated local environment without any human intervention.

This unlocked everything else. Once worktrees were reliable, we could run multiple agents in parallel on different tasks, each with its own branch, its own server, its own test environment. Most of the speedup came from running three or four agents in parallel without collisions.

But that speed advantage hits a wall the first time one of those agents needs to run a database snapshot.

Operations: when the process lives in someone’s head

Every engineering team has critical processes that work perfectly as long as the right person is online and remembers the exact sequence of commands - which is really just a single point of failure wearing a human costume.

This quarter we found three of these in our own stack and rebuilt them.

Database snapshots

Before: creating a database snapshot for a customer or for staging required running six commands in exact order. Dump the SQL. Package it. Upload to the right bucket. Restore on the target. Create the snapshot record. Notify the team in Slack. Miss a step and you get a corrupt snapshot. Run them out of order and you overwrite something. The whole process lived in one engineer’s head and a Slack message from eight months ago.

After: ./manage.sh snapshot create. One command. It runs preflight checks (is the source database accessible? Is there enough disk space?), executes the full pipeline with error handling at every step, and posts a Slack notification when it is done or if it fails. Any engineer can run it. No tribal knowledge required.

Admin actions

Before: high-impact admin actions (think: merging two user accounts, resetting organization data, modifying subscription state) were one-click operations with no confirmation step. Click the button, the mutation happens immediately, no undo. One accidental click on the wrong row in a moment of pressure = irreversible data loss.

After: high-impact actions follow a three-step model. First, review the impact - the system shows you exactly what will change and who is affected. Second, confirm the scope - you explicitly acknowledge what you are about to do. Third, execute - only after passing eligibility checks and risk assessment. Going from “click and pray” to “plan, confirm, execute” is how you turn potential incidents into boring Tuesdays.

Support shell workflows

Support shells - the tools your team uses to diagnose and fix customer issues in production - got a significant upgrade. Instead of an engineer manually running queries and eyeballing results, the shell now surfaces relevant context, suggests likely fixes, and handles the common reconciliation patterns automatically. The shell keeps engineering judgment in the loop while shortening the path from incident report to root-cause diagnosis.

There is no product launch here and nobody is going to tweet about it, but this is the work that keeps incidents smaller, response times tighter, and makes sure the process still works when the person who “knows how snapshots work” is on vacation. It also removed the last excuse for not touching the security backlog.

Security: from random interrupts to planned waves

Here is how security and dependency work usually goes: Dependabot opens an alert. Someone sees it, maybe today, maybe next week. They bump the version, run the tests, fix whatever breaks, open a PR. It gets reviewed between other work. Meanwhile three more alerts have fired. The PRs conflict with each other. Merge churn increases. Review quality drops because everyone is context-switching.

We replaced that pattern with a wave-based remediation model:

• Wave 1: Patch-level security updates. Django, Pillow, SimpleJWT, pyasn1. Low-risk, high-confidence bumps.
• Wave 2: Cryptography chain. cryptography and weasyprint updates that cascade through the dependency tree. Medium risk, needs careful testing.
• Wave 3: Deep transitive dependencies. Refreshing the protobuf chain through Google’s dependency graph. Higher complexity, so we isolated it into its own review cycle.

The security work went beyond dependency bumping. In the same window, we also:

• Hardened CI input handling. GitHub Actions workflows can receive untrusted input from PR titles, branch names, and commit messages. If those inputs flow unsanitized into shell commands, that is a command injection vulnerability in your CI pipeline. We audited and hardened those paths.
• Reduced PII exposure in telemetry. Logging and error tracking payloads were carrying more personally identifiable information than they needed to. We scrubbed those payloads down to what is actually useful for debugging without exposing user data.

The practical result is that security and dependency work stopped being an interrupt. We close issues faster, in planned batches with proper review scope, and the rest of the team can actually plan their week without random fire-drills derailing it.

The security waves also forced us to confront what we had been putting off: the migration graph and a pile of dead integration code that had been making every other change harder than it needed to be.

Debt: the art of safe subtraction

Deleting code is easy. Deleting the right code without breaking something in production takes a lot more care.

Migration topology

Our Django migration graph had accumulated cruft over time - the way they always do. Branches, merges, dependencies that used to make sense but now just add confusion. New developers would look at the migration graph and have no idea what was load-bearing and what was historical accident.

We used a two-step squash strategy:

1. Introduce replacement lineage. New squashed migrations replace the old chain, but the old migrations stay around for compatibility. Both paths work.
2. Archive and prune. Once the new lineage is stable and deployed, remove the old chain. The graph is clean.

This approach respects the reality that migrations are one of the scariest parts of a Django application. You do not yolo a migration squash. You prove the new path works while the old path is still available as a fallback.

Dead integration code

We also retired deprecated Slack and Teams integration paths - but only after proving they were genuinely unused or had working replacements. “Dead” integration code is never truly dead. It still shows up in code reviews (“is this still used?”), expands the upgrade surface (“this import broke after the SDK update”), and confuses new contributors (“should I be using this API or the other one?”).

Removing it deliberately - with evidence that nothing was actually using it - means every future engineer who touches that part of the codebase has one less thing to be confused about.

By this point we had noticed a pattern: the cleanup workflows, the review checklists, the remediation sequences - we kept solving similar problems the same way. Why were we still rebuilding them from scratch each time?

The biggest multiplier was not a model

If I had to pick the single biggest multiplier from this quarter, it would be this section, and it connects directly to why model selection matters.

Here is what kept happening: an engineer figures out a really effective way to use Opus for a thorough code review, or they discover that Codex handles Dependabot triage perfectly if you give it the right structure. It works great for them. Then it lives in their head, or in a DM, or in a prompt they copy-paste from a notes app. That is useful for one person, but it does not spread.

We took a different approach: we turned those patterns into reusable agent skills and commands, and open-sourced the whole layer in DiversioTeam/agent-skills-marketplace. Right now that repo includes 15 plugins, 17 skills, and 37 commands covering:

• Code review - Hyper-pedantic Django review skills that check for multi-tenant safety, migration correctness, and security patterns
• Commit and PR hygiene - Atomic commit helpers that enforce pre-commit hooks, lint gates, and clean commit messages
• Dependabot remediation - Wave-based triage and execution for both frontend and backend dependency management
• Release operations - PR creation, version bumping, merge conflict resolution, and release publishing
• Docs generation - Repository documentation, API docs from Bruno files, AGENTS.md canonical formats

This is where Simon Willison’s insight lands hardest: “Coding agents mean we only ever need to figure out a useful trick once.” That is exactly what skills do. Someone figures out the best way to run a security audit with Opus. It becomes a skill. Now every engineer on the team runs security audits at that level, automatically.

Instead of “someone has a strong prompt in a DM,” these became shared tools that anyone on the team can run, inspect, and improve. They live in version control with clear entry points, not in someone’s clipboard.

This matters for two reasons:

1. Reviews stop depending on who is doing them. When every code review uses the same skill with the same guardrails, the floor rises. You stop getting thorough reviews from one person and surface-level reviews from another.
2. Onboarding gets faster. New engineers do not need to discover the team’s workflow through osmosis. They install the skills and inherit months of accumulated process knowledge on day one.

This is why I keep saying agentic coding is an execution multiplier, but also a learning multiplier. The model’s capability matters, but what decides whether gains stick across the team or disappear when one person goes on vacation is whether you have a skill system.

We are not the only ones who figured this out. The ecosystem is moving fast - Anthropic ships official plugins for Claude Code, Vercel published a library of agent skills, and individual developers are releasing genuinely useful tools like visual-explainer that solve problems we would have built ourselves. More and more companies and individuals are open-sourcing skills that slot right into existing agent workflows, and our team is actively pulling the best ones into how we work. The value of a skill system goes up the more people contribute to it - not just inside your team, but across the industry.

We did not see what the skills layer had actually done until we stepped back and looked at the quarter as a whole.

How it all compounded

What makes this quarter more than a list of separate improvements is that the workstreams were not independent. They fed into each other, and each one made the next one easier to pull off.

Here is how the loops worked:

• CI speed → testing velocity. Faster CI feedback meant testing improvements were cheaper to validate. You could iterate on a test three times in an hour instead of once.
• Testing quality → CI signal. Better tests with deterministic waits produced cleaner CI signal. Fewer flaky failures meant the CI results were actually trustworthy, which made scope-aware gating and cost optimization possible.
• Worktree isolation → everything. Reliable worktrees meant agents and engineers could iterate on every other stream in parallel without breaking each other’s environments.
• Skills codification → all of the above. Once patterns were encoded into reusable skills, we stopped relearning the same lessons. New contributors inherited the workflow directly.

We did not just do many things in one quarter - we ended up building a system where the improvements reinforced each other. Better CI feedback made testing more productive, which made CI signal more trustworthy, which made it safe to skip unnecessary checks, which freed up time to codify the patterns, which meant the next improvement started from a higher baseline.

If you are looking at this and thinking “we have the same hidden queue,” here is where to start.

What you can actually copy

If you want to apply this without copying our stack, copy the mechanics:

1. Make the hidden queue visible and measure it. Track every time an engineer waits, context-switches, or does manual repetitive work for one week. Add up the hours. Then pick one loop, instrument it, fix it, and measure the before and after. Start wherever your data is cleanest.

2. Separate qualification from execution. In any expensive system - CI, deployments, data pipelines - cheap checks should decide whether expensive checks run. A 30-second linter should gate a 15-minute integration test, not run alongside it.

3. Ship in phases, not big bangs. Every improvement in this post was shipped incrementally. If Phase 2 breaks, you roll back Phase 2. Phase 1 still works.

4. Make worktrees work. If your team is using AI coding agents, fix your local dev setup first. Deterministic port isolation, environment syncing, single-command startup. Without this, you are bottlenecking every other improvement.

5. Learn the models and mix them. Claude is fast and great for execution but will take shortcuts on complex work. Codex is slower but more thorough. Run them together - one plans, one executes, one reviews.

6. Turn prompts into skills. When someone on your team figures out an effective AI-assisted workflow, do not let it stay in their head. Encode it. Version it. Share it. The gap between “one person is fast” and “the whole team is fast” is a skill system.

These patterns are general-purpose and work whether you are running a monolith, microservices, or something in between. The models make it cheaper to try things and iterate. The patterns are what make the results last.

Attention is still the bottleneck

If you take one thing from this post, do not let it be “they wrote less code.” We wrote and deleted a lot of code - 120,929 lines added, 31,170 deleted.

What changed is where we spent our attention.

The repetitive plumbing - CI configuration, test infrastructure, environment wiring, dependency management, snapshot steps - is now cheaper to do. That frees up the team to focus on boundaries, failure modes, control surfaces, and review quality.

We still write code and still need engineers; we are just spending more of that scarce attention on work that actually needs human judgment.

If your own company is a tenant in your own product, it should be the hardest one to access, not the easiest.

At Diversio, one of the products we build is Optimo, a multi-tenant SaaS that helps HR and people teams work with sensitive employee and organizational data.

Most Optimo organizations belong to customers.

One belongs to us.

That internal Diversio org inside Optimo holds our own employee and company data. It lives in the same production cluster as customer tenants, and our engineers and Client Success team use the same product surfaces to support both, always through per-tenant and role-based guardrails.

Customer orgs already have their own access boundaries; this post is about holding our own org to an even higher bar so it is the hardest one to reach, not the easiest.

Even with those existing guardrails, it is easy for internal tooling to drift toward a de facto “god mode” for your own org if you never make the extra constraints explicit. We wanted to push in the opposite direction. We wanted everyone at Diversio to be comfortable with how their data is handled, while still letting a very small engineering team move quickly for customers.

This post walks through how we got there.

Table of Contents

1. What you’ll learn
2. Why our own org is special
3. Layer 1: An org-scoped admin mixin
4. Layer 2: A TOTP-gated workflow for sensitive access
5. Layer 3: Postgres RLS and the sensitive flag
6. Layer 4: IAM and support shells
7. Automation and a five-person team
8. Gaps and future work
9. The bottom line

What you’ll learn

By the end of this post, you will see how to:

• Treat your own company’s tenant as the hardest one to reach in a multi-tenant SaaS.
• Use an org-scoped admin mixin to get “row-level security” at the application layer.
• Wrap sensitive access behind a small TOTP-gated workflow instead of ad-hoc toggles.
• Lean on Postgres Row-Level Security (RLS) and a single support-shell role to protect production data.
• Use CI to keep your RLS and access model from silently drifting.
• Get all of this working with a five-person engineering team without grinding development to a halt.

If you are building a multi-tenant application and your own company is one of the tenants, this post gives you a concrete set of patterns you can reuse or adapt.

Why our own org is special

When we say “our own org”, we mean the internal Diversio organization that lives alongside customer orgs inside Optimo.

It is special for a few reasons:

• It includes our own employee data and internal company information.
• It exercises almost every feature in Optimo, often before customers see it.
• It is the easiest tenant for us to accidentally treat as “less important”, because we are so close to it.

This pattern is not unique to us. GitHub runs on GitHub, Slack lives inside Slack, and Stripe’s teams build and operate on top of their own payments APIs and tools. Claude Code is another good example: a lot of what makes it feel sharp comes from Anthropic dogfooding it heavily¹, and we lean on it in our own repos too. Using your own tools at full blast is one of the fastest ways to find sharp edges before customers do.

Optimo is the same. Our own org is where we try new workflows, push edge cases and shortcut a lot of “demo tenant” friction. That is great for product quality, but it also means the easiest tenant for us to poke at is the one tenant we should be most careful with.

We wanted to set a very simple rule:

If a control is good enough for a sensitive customer, it should apply to Diversio too.

At the same time, we did not want a giant “break everything” toggle in one admin screen that quietly bypassed all of this for engineers on call.

The result is a layered design:

1. A clear notion of which organizations each admin may see at the application layer.
2. A small, TOTP-gated workflow to grant and remove sensitive access.
3. Postgres RLS that treats sensitive orgs as a first-class concept.
4. IAM and support shells that default engineers into safe roles.
5. Automation that keeps all of these in sync as the system changes.

The rest of this post goes layer by layer.

Layer 1: An org-scoped admin mixin

Most internal users at Diversio interact with data through Django admin and a handful of internal consoles. That is where we started.

We already had the usual multi-tenant pieces:

• An Organization model.
• Internal staff users who can act on behalf of organizations.
• A boolean flag that means “this organization is sensitive”.

The problem was that each Django admin class had to remember to do the right thing. It is easy to get one view right, then forget the filter on another.

So we pulled the logic into a reusable organization-scoped admin mixin (internally, OptimoOrgScopedAdminMixin).

At a high level, every staff user has:

• A global profile (are they active staff, what kind of staff).
• Per-org assignments (which customers or regions they are responsible for).
• A flag indicating whether they are allowed to see any sensitive orgs at all.

A central helper answers a single question:

def get_allowed_org_ids(user) -> set[int]:
    # Look at profile, per-org permissions and the sensitive flag.
    # Return the set of organization IDs this user may see.
    ...

The admin mixin uses this helper to:

• Filter list views down to the organizations you are allowed to see.
• Enforce object-level permissions (“can I view, change or delete this row”).
• Narrow foreign keys and dropdowns so you cannot accidentally link records to orgs you do not have access to.

A simplified version looks like this:

class OrgScopedAdmin(ModelAdmin):
    def get_queryset(self, request):
        qs = super().get_queryset(request)
        allowed = get_allowed_org_ids(request.user)
        return qs.filter(organization_id__in=allowed)

    def has_view_permission(self, request, obj=None):
        base = super().has_view_permission(request, obj)
        if not base or obj is None:
            return base
        return obj.organization_id in get_allowed_org_ids(request.user)

Two important choices here:

• Sensitive orgs (including Diversio) are excluded by default, even for superusers.
• Admin views opt in to the mixin and get the same behaviour, instead of reinventing access rules one screen at a time.

For an engineer with strong customer-facing permissions:

• Django admin is still powerful for the customer orgs they are assigned to.
• The Diversio org simply does not show up unless a separate process has granted that level of access.

This is our first layer of “row-level security”: an application-level filter baked into admin, not an afterthought in each view.

Layer 2: A TOTP-gated workflow for sensitive access

The admin mixin enforces “what you can see” for a given profile.

The next question is: how does someone get a profile that is allowed to see the Diversio org at all?

We knew what we did not want:

• A checkbox in Django admin that anyone with enough permissions could toggle.
• A permanent “god mode” role that people forget they are still holding.

Instead, we built a small, TOTP-gated workflow.

Each staff profile has a boolean that means:

This person may see sensitive orgs like Diversio, in addition to their normal org assignments.

That boolean is visible in admin, but it is read-only. The only way to change it is a CLI command that:

• Takes a target user.
• Takes a TOTP code from an approver (CEO, CTO or a small set of delegates).
• Logs who did what and when.

In pseudocode:

def require_sensitive_approval(code: str) -> None:
    if not verify_totp(code):  # shared helper wired to env secrets
        raise ApprovalError("Invalid TOTP code")


def enable_sensitive_access(staff_user, code: str) -> None:
    require_sensitive_approval(code)
    staff_user.can_see_sensitive_orgs = True
    staff_user.save(update_fields=["can_see_sensitive_orgs"])

The outcome looks like this in practice:

• Sensitive access is rare and is usually time-boxed to a specific investigation.
• It is deliberate: someone with real authority has to approve it with a physical device.
• It is auditable: we can answer “who could see the Diversio org in April” without guessing.

This is the human-side guardrail that complements the org-scoped admin mixin.

Layer 3: Postgres RLS and the sensitive flag

Application-level scoping and TOTP workflows are necessary, but not sufficient. Engineers also work close to the database: SQL shells, diagnostics, ad-hoc queries and migrations.

We wanted Postgres itself to agree that the Diversio org is special.

We do two things here.

3.1 Guard the sensitive flag itself

First, we treat the “this org is sensitive” flag as something you do not get to flip casually.

• Normal application roles and ad-hoc DB connections cannot change it.
• Only a narrow path, tied to the TOTP-gated CLI, can set or clear it.
• The database checks the current role and a session variable before allowing updates to that column.

The goal is simple:

• Nobody “experimenting” in a production shell can quietly downgrade the Diversio org from is_sensitive = TRUE to FALSE.
• Even if a bug slips through at the app layer, the database is still able to say “no” on the one write that really matters.

3.2 Postgres RLS for support shells

The second piece is Postgres Row-Level Security (RLS), which we apply to a dedicated support database role, the role engineers use in our production support shells.

For that support role, RLS policies enforce:

• On org-scoped tables, queries only see rows where the organization is not sensitive.
• Rows belonging to sensitive orgs (including Diversio) are filtered out by the database itself.
• These rules apply to both reads and writes.

In simplified SQL:

ALTER TABLE org_events ENABLE ROW LEVEL SECURITY;

CREATE POLICY support_only_non_sensitive
  ON org_events
  FOR ALL
  TO support_shell_role
  USING (
    organization_id IN (
      SELECT id
      FROM organizations
      WHERE is_sensitive = FALSE
    )
  );

If you connect to production using the support shell and run:

SELECT * FROM org_events
WHERE organization_id = (
  SELECT id FROM organizations WHERE name = 'Diversio'
);

Postgres itself refuses to show those rows to the support_shell_role, regardless of what the application code might have done.

3.3 An RLS registry and CI checks

We have a lot of org-scoped tables in Optimo:

• Some with a direct organization_id column.
• Some where the org needs to be derived via a join (through a user, an employee, a workspace and so on).

Managing RLS for all of them by hand is a great way to make mistakes.

So we maintain a small Python “registry” that lists:

• Which tables are org-scoped.
• How to determine the organization for each table.
• Which tables are intentionally excluded.

On top of that registry, we have a management command that:

• Enables RLS on those tables.
• Ensures policies exist for the support role.
• Creates a helper function like org_is_not_sensitive(org_id uuid) for USING and WITH CHECK clauses.
• Scans the schema for any new org-scoped tables that are not in the registry yet.
• Fails loudly if it finds gaps.

We run this command automatically in CircleCI:

• Apply migrations to a fresh database.
• Run the RLS command in “apply plus verify” mode.
• If someone adds an org-scoped table and forgets to update the registry, the build fails.

Conceptually, the CI step looks like this:

- run:
    name: Validate support-shell RLS
    command: |
      python manage.py migrate --noinput
      python manage.py check_support_shell_rls --apply --verbosity 2

RLS becomes an executable spec, not a fragile manual checklist.

Layer 4: IAM and support shells that default to safe

The final layer is how we actually connect to production.

Like many teams, we used to have engineers with:

• Broad AWS permissions.
• The ability to run ecs exec into production containers.
• Access to secrets wired to powerful database roles.

That is convenient, but it does not align with “the Diversio org should be the hardest tenant to reach”.

We reshaped things so that sensitive power is separated and explicit.

4.1 Sensitive secrets are separated

Anything that directly powers sensitive workflows, for example:

• Special database URLs.
• TOTP secrets.
• Break-glass admin credentials.

lives in its own secret, encrypted under a dedicated KMS key. Only specific permission sets and automation roles can read it.

4.2 Permission sets map to real personas

We distinguish a normal “production engineer” persona from a “break-glass administrator”.

The normal production role:

• Cannot talk directly to the main database role.
• Cannot read master database secrets.
• Cannot read sensitive secrets.
• Can still manage non-sensitive app configuration and run diagnostics.

4.3 Support shells are the default

Engineers connect to production via a dedicated support-shell service that:

• Runs the same Optimo container image as the main app.
• Uses the restricted support database role by default.
• Is wired so that a normal production engineer always lands in a support shell, not on a fully privileged backend container.

A small wrapper script ties it together:

permission_set=$(detect_aws_permission_set)

if [ "$permission_set" = "ProdEngineer" ]; then
  target_service="support-shell"
else
  target_service="backend"  # break-glass roles only
fi

open_ecs_exec_shell "$target_service"

If the support-shell configuration is missing or broken, the script refuses to open a shell instead of silently falling back to something more powerful.

Combined with RLS, this gives us:

• A realistic production shell for debugging customer issues.
• A database that still hides rows for the Diversio org from the support role.
• A clear, auditable path to more power when it is genuinely needed.

Automation and a five-person team

At this point, it is fair to ask:

Is this too much machinery for a five-person engineering team?

It would be, if it were all manual.

What makes it work is that the moving parts share a few patterns:

• One admin mixin for org scoping, reused everywhere.
• One TOTP-gated path for sensitive access, instead of scattered toggles.
• One RLS registry and command that define what the database should enforce.
• One CI step that runs migrations, bootstraps RLS and fails when something drifts.

The extra work happens once, up front. After that:

• New admin views inherit the mixin and automatically respect org scoping.
• New engineers follow the same shell story and get the same protections.
• New org-scoped tables are caught by the RLS command if we forget to register them.

The system enforces the policy for us, instead of us trying to remember every rule in our heads.

Gaps and future work

We are happy with the direction, but this is not “done”.

Some gaps we have deliberately left open for future iterations:

• RLS mostly protects the support role today.
  We would like to extend RLS to more roles and tables where it makes sense, without tanking performance or complicating migrations.

• Many read paths still lean on application logic.
  The admin mixin does a lot, and the database does a lot for support shells, but other read paths still rely on Django-level scoping. We would like more database-side guardrails around the most sensitive slices of the Diversio org.

• Sensitive access observability is basic.
  We log the right events and can reconstruct them, but we would like:

  • A small dashboard that graphs sensitive access over time.
  • Simple alerts when approvals spike or come from unexpected places.

• Approvals are very CLI-shaped.
  The TOTP flow is fine for engineers, less so for non-engineers. A small internal UI for approvers (“who is asking to see Diversio, why, and for how long”) would make this much more approachable.

• We do not “game day” this enough.
  We test the pieces individually. We would like regular drills where we intentionally try to break assumptions about the Diversio org in a safe environment and see how the system responds.

These are good problems to have. They are the kind of problems you get to think about only after you have built the basics.

The bottom line

Letting your own company be a tenant in your own product is a great dogfooding story: your own team runs real workflows through the product every day. It is also a serious security challenge.

Our approach at Diversio has been to:

• Treat the Diversio org as the most demanding customer we have.
• Give the application a clear notion of which orgs each admin may see, via an org-scoped mixin.
• Give the database a clear notion of which rows each role may see, via RLS and a registry.
• Let CI and automation keep those contracts honest as Optimo evolves.
• Build human workflows (approvals, TOTP) into the design from the start.

The key point is not that we use Django or Postgres or AWS specifically. The key point is that we use layers:

• UI and admin behaviour.
• Approval workflows.
• Database policies.
• Cloud IAM and shell tooling.
• Automation to glue it together.

Together, these give us something stronger than “trust us, we will be careful” and they do it in a way that a small, focused team can maintain while still shipping features.

Thanks to Ashish Siwal for reviewing this post and helping solidify the implementation.

The hard part about software design is state. If you’re storing any kind of information for any amount of time, you have a lot of tricky decisions to make about how you save, store and serve it.

I recently read Sean Goedecke’s excellent piece on system design principles, and it crystallized many lessons I’ve learned over 10+ years of building systems. Here are my key takeaways and how they’ve played out in practice.

Table of Contents

1. State: The Root of All Complexity
2. Database Design: Get the Schema Right
3. Query Efficiency: Let the Database Work
4. Background Processing: Fast vs. Slow Operations
5. Caching: Choose Carefully
6. Hot Paths: Zero Margin for Error
7. Logging and Error Handling
8. Graceful Failure: The Hardest Problem
9. The Pattern Recognition

State: The Root of All Complexity

State makes everything tricky. The more state we have, the harder it becomes to reason about our systems and keep track of responsibilities.

On the flip side, statelessness relieves us of worrying about systems getting “stuck” or needing complex guardrails to prevent corruption.

You should try and minimize the amount of stateful components in any system.

What this means in practice is having one service that knows about the state - i.e. it talks to a database - and other services that do stateless things. Avoid having five different services all write to the same table.

I’ll admit this isn’t something I actively think about - I might be doing it unconsciously in some cases and missing it entirely in others. But I want to keep an eye on this pattern in future projects.

Database Design: Get the Schema Right

Schema design should be flexible, because once you have thousands or millions of records, it can be an enormous pain to change the schema. However, if you make it too flexible (e.g. by sticking everything in a “value” JSON column, or using “keys” and “values” tables to track arbitrary data) you load a ton of complexity into the application code (and likely buy some very awkward performance constraints).

I’ve always spent significant time on database design when building new systems, and it’s consistently paid off. The JSON trap is real - while convenient initially, JSON fields create several problems:

• Hard to query effectively
• Difficult to enforce schemas
• Serialization/deserialization overhead
• Network transfer costs for large payloads
• Additional validation complexity

That said, I’ve found JSON fields useful for storing external API payloads and validation errors - places where the structure truly varies.

Index Strategy

If you expect your table to ever be more than a few rows, you should put indexes on it.

I think the author didn’t intend it, but index as you go. If your table isn’t showing slow queries yet, don’t index prematurely. You can always add indices later and optimize them based on actual query patterns.

I’m a big fan of PostgreSQL’s explain analyze - and with AI tools, analyzing query plans has become even more accessible.

Query Efficiency: Let the Database Work

When querying the database, query the database. It’s almost always more efficient to get the database to do the work than to do it yourself. For instance, if you need data from multiple tables, JOIN them instead of making separate queries and stitching them together in-memory. Particularly if you’re using an ORM, beware accidentally making queries in an inner loop. That’s an easy way to turn a select id, name from table to a select id from table and a hundred select name from table where id = ?.

As a longtime Django user who’s been reviewing Django code for 10+ years, I’ve seen the N+1 query problem countless times. It’s crucial to educate your team on ORM fundamentals and maintain documentation of good vs. bad practices.

Having a central document helps during code reviews - it’s hard to repeatedly explain the “why” with the same depth. A shared resource makes linking to best practices much easier.

Background Processing: Fast vs. Slow Operations

The goal is a fast request-response cycle for the best user experience. Operations that can’t be completed quickly should move to background processing, with users notified when complete.

This seems obvious, but determining what belongs in the background requires careful consideration of user expectations and system constraints.

Caching: Choose Carefully

Be selective about what and when to cache. Otherwise, you’ll face the much harder problem of cache invalidation. The best solution is reducing the number of places using cache.

Cache invalidation remains one of the hardest problems in computer science for good reason.

Hot Paths: Zero Margin for Error

I define hot paths as system components where downtime significantly affects our SLA - critical numbers, authentication, or features that directly impact business and users.

These systems require extra care:

• World-class unit tests and coverage
• Attention to unhappy paths
• More conservative change management

Take Instagram: it’s more important for signed-in users to see their feed than for signup to work perfectly.

Logging and Error Handling

Beyond traditional logging, we utilize logs in unit tests to ensure they’re not removed carelessly during updates. This makes log changes visible during code review.

We also return custom error codes with responses - they help identify issues without log diving and assist both customers and developers debugging problems.

For critical errors sent to monitoring tools like Sentry, we use custom exception names to quickly spot issue locations based on error messages.

Graceful Failure: The Hardest Problem

This ties back to state management. Building resilient state machines requires balancing when to fail, retry frequency, state resets, and maintaining manageable state complexity.

It’s one of the trickiest aspects of system design - fighting between reliability and simplicity.

The Pattern Recognition

After years of building systems, these principles become pattern recognition. The key is being intentional about applying them rather than hoping they happen by accident.

Sean’s post does an excellent job distilling complex system design into actionable principles. The challenge is remembering to apply them when you’re deep in implementation details.

What patterns have you noticed in your own system design work? I’d love to hear about principles that have served you well over time.

Picture this: You’re debugging an issue that spans your React frontend, Django backend, and Terraform infrastructure. Three terminal windows. Three different repos. Three different contexts. By the time you trace the bug from UI to API to database, you’ve lost half your morning just to context switching.

We solved this at Diversio. The solution made both humans and AI 10x more effective with our codebase.

Table of Contents

1. What you’ll learn
2. The Problem We Were Facing
3. Our Solution: Git Submodules Done Right
4. The AI Advantage
5. Advanced Implementation
6. MCP Integration and Team Benefits
7. The Technical Details
8. Implementation Strategy
9. The Bottom Line

What you’ll learn

• Structure a monolith repository with git submodules for maximum efficiency
• Master git worktrees for parallel development without context switching
• Automate worktree creation with our Python scripts (links included!)
• Reduce context switching by 90% and watch productivity soar
• Supercharge AI tools with complete codebase context—making them actually useful
• Implement immediately with practical tips and working code

If you’re tired of juggling multiple repos and losing context between features, this guide shows you exactly how we solved it. If you’re already using git submodules, you’ll learn how to level up with worktrees and automation.

The Problem We Were Facing

Every feature we built touched multiple repositories:

• Clone three separate repos
• Keep versions in sync
• Jump between directories constantly
• Lose context when debugging cross-service issues

But here’s the real kicker. AI tools like Claude Code couldn’t see our full system. They’d suggest fixes that made sense for one service but broke two others. They had fragments, not the full picture.

Our Solution: Git Submodules Done Right

We built a monolith using git submodules that gives us the best of both worlds:

diversio-monolith/
├── frontend/              # React application
├── backend/               # Django API
├── design-system/         # Shared UI components
├── optimo-frontend/       # Optimo product app
├── diversio-serverless/   # AWS Lambda functions
├── infrastructure/        # Terraform definitions
├── terraform-modules/     # Reusable infrastructure
└── scripts/               # Development automation

Each directory is its own git repository. But they all live in one workspace. One clone gets you everything.

From Days to Hours

Our product manager Sam now redesigns features in hours, not days. He pulls the latest code, runs migrations, and has AI analyze our Bruno API collections. Remember when we migrated from Postman to Bruno? That decision pays huge dividends in a monolith.

The workflow looks like this:

1. Pull latest commits across all services
2. Run migrations with full database context
3. AI reads organized API documentation
4. Create comprehensive designs with complete system understanding
5. Implement changes knowing exactly what will be affected

What used to take hours of repository hopping now happens in a single flow.

The AI Advantage

Here’s where it gets interesting. By giving AI tools complete system context, we unlocked capabilities that were impossible before:

• Cross-repository analysis: Claude Code traces API calls from frontend to backend to infrastructure
• Better architectural decisions: Suggestions consider the entire stack
• System-wide debugging: Issues that span services become traceable
• Smarter refactoring: Changes account for all dependencies

The AI isn’t guessing anymore. It knows.

Advanced Implementation

Git Worktrees: The Secret Sauce

The real magic happens with git worktrees. Instead of switching branches and losing context, we work on multiple features simultaneously:

# Create a new worktree for a feature
uv run scripts/create_worktree.py

Our automation handles everything:

• Interactive branch selection across all repositories
• Smart filtering for hundreds of branches
• Automatic submodule initialization
• Proper conflict resolution
• Copies .env files (because git worktrees don’t do this by default, and nobody wants to debug missing environment variables)

Check out our automation scripts:

• create_worktree.py - Interactive worktree creator with submodule support
• update_submodules.py - Automated submodule updater with branch configuration

Each developer can have multiple features in progress, each in its own directory, with full system context preserved.

Custom AI Agents: Specialized Tools for Every Task

We built specialized Claude Code agents that understand our codebase deeply:

• Frontend PR Specialist: Analyzes React/TypeScript changes with component architecture visualizations
• Backend PR Specialist: Reviews Django changes with database schema analysis
• Infrastructure PR Specialist: Validates Terraform changes with cost and security impact assessments
• Integration Specialist: Traces data flows across frontend, backend, and infrastructure
• Data Migration Specialist: Handles complex data transformations and migrations
• Testing Automation Specialist: Writes comprehensive test suites following our patterns

Each agent has deep knowledge of its domain. They don’t just suggest generic fixes. They understand our specific patterns, our conventions, our tech stack.

We even built meta agents that manage other agents:

• Subagent Creator: Designs new focused agents with clear boundaries
• Subagent Reviewer: Reviews and optimizes agent definitions to prevent overlaps

These meta agents save hours when adding or updating agents. No more manual agent configuration. The AI helps manage the AI.

The best part? These agents are shared across the entire team through the monolith. When one engineer creates an agent, everyone benefits immediately.

MCP Integration and Team Benefits

We integrated Model Context Protocol (MCP) servers that give Claude Code direct access to:

• CircleCI: Build status, test results, deployment logs
• Context7: Up-to-date documentation for any library
• Gemini AI: Research and code analysis
• Playwright: Browser automation for testing

These integrations work seamlessly across the entire monolith, giving AI unprecedented visibility into our development pipeline.

Benefits Across the Team

Product Managers get complete system visibility for rapid prototyping. API documentation and database schemas always accessible.

New Engineers get a single command setup. Clone one repo, get everything with working examples.

Senior Engineers can do system-wide refactoring. Make changes across repos with full context.

AI Tools get complete codebase analysis. Suggestions that actually work.

DevOps teams see infrastructure definitions alongside the code they deploy.

The Technical Details

Submodule Management: Each submodule points to a specific commit. Changes need commits in both the submodule and the monolith. This preserves atomicity while enabling coordination.

Branch Independence: Work on different branches in each submodule simultaneously. Perfect for features that need coordinated changes. And crucially, pull requests still go to individual repositories, maintaining our existing review processes and CI/CD pipelines.

Automated Tooling: Our Python scripts handle branch management, worktree creation, and submodule updates. No manual coordination needed.

API Documentation as Code: Bruno’s file-based approach means API collections live with the code they test. They’re discoverable, version-controlled, and immediately accessible to both humans and AI. The AI can read, understand, and even suggest API changes based on the actual implementation.

Implementation Strategy

AI tools are becoming primary development partners. But these tools are only as good as the context you give them. By structuring our codebase for maximum AI comprehension, we created an environment where both humans and AI work at their full potential.

The results: 10x faster feature development, 90% fewer integration bugs, and architectural decisions that consider the full system impact.

Getting Started

If you’re working with multiple related repositories, you can implement this:

1. Create a monolith repository
2. Add existing repos as git submodules
3. Build automation scripts for common workflows
4. Configure MCP servers for your tools
5. Train your team on git worktrees
6. Move API documentation to file-based tools like Bruno

The investment pays off immediately. No more context switching, better AI assistance, and a development experience that scales with complexity.

The Bottom Line

This isn’t just about tooling. It’s about creating an environment where high-quality work happens quickly and naturally.

When your product managers can redesign systems confidently, your engineers can refactor safely across services, and your AI tools provide system-aware suggestions, you’ve built more than a development environment.

You’ve built a competitive advantage.

Want to dive deeper into Git worktrees? Check out my comprehensive guide: Git Worktrees: From Zero to Hero. It covers everything from first principles to advanced workflows with submodules.

Thanks to Samuel Bonin and Amal Raj for reviewing this post.

We moved our entire Postman collection to Bruno over a weekend and let Claude Code chew on the new files by Monday morning. By lunchtime, API docs were writing themselves.

Table of Contents

1. What you’ll learn
2. Why we relied on Postman for so long & where it falls short
3. Migration basics
4. Scripts that saved us hours
5. Common Migration Patterns
6. The AI Integration Revolution
7. The Unexpected Benefits
8. Getting Your Team Started
9. What This Means for Engineering Teams

What you’ll learn

• Migrate a Postman collection (requests and environments) to Bruno in a few hours.
• Keep docs in‑lockstep with your codebase; no more stale Postman descriptions.
• Let AI write the boring bits: type /bruno-api path/to/request.bru and get ready‑to‑ship docs, TypeScript types, and React Query hooks.
• Catch breaking changes at review time, not after deploy night.

If you’re exploring AI‑first tooling and looking to streamline your API workflow, this guide walks you through our practical migration step‑by‑step. If you’re already using Bruno, you’ll learn how to help improve your existing workflow further using AI.

Why we relied on Postman for so long & where it falls short

At Diversio, Postman had been our all‑purpose API toolkit since the company’s first endpoint shipped in 2018. Every engineer has owned a collection or three; shared environments lived in the cloud; QA and PMs could fire requests without touching the codebase.

Why Postman Works for Us (So Far)

• Scriptable: Pre‑request scripts spun up test users, refreshed auth tokens, and handled auth with a single click.

• Variable templating: {{base_url}}, {{auth_token}}, and other vars kept requests DRY across local and cloud setups.

• Chained workflows: Scripts set environment variables that downstream requests consumed, so multi‑step API flows ran end‑to‑end without anyone hand‑editing the shared collection.

• Performance‑testing ready: The same collections powered our performance tests, giving us baseline latency numbers without duplicating effort.

• Full‑stack friendly: Frontend and backend engineers could debug and iterate on the same requests without context‑switching or extra tooling.

• Collaborative: Share links meant non‑engineers could poke endpoints in seconds.

Why it started to hurt

As our API surface has increased and now we are working on more and more features at once(thanks to agentic coding), the workflow that once felt effortless began eating hours and hurting our productivity.

• Manual sync tax – Every new endpoint meant combing through multiple collections to wire up scripts, tests, and examples by hand.

• Docs drift – Descriptions hid in Postman’s UI; unless someone remembered to update them, they slipped out of date.

• Invisible breaking changes – Because collections lived in Postman’s cloud, reviewers never saw contract updates during code review.

• Meeting creep – We still ended up on calls to reconcile mismatched examples and edge‑case behaviours. A lot of time spent in huddles and stand-ups discussing APIs that we couldn’t document well inside of Postman.

Postman still works but it has just slowed us down. Any change meant updating code and a JSON export no one liked opening. That lag became the bottleneck.

Bruno’s Git‑friendly plain‑text format, and its ability to embed full Markdown docs, looked like a way out. The best part? AI agents can read .bru files like normal code, so automation suddenly became trivial and APIs are now part of our codebase and included in diffs during code reviews.

Migration basics

1. Script syntax

// Postman
var json = JSON.parse(responseBody);
var token = json["access_token"];
pm.environment.set("auth_token", token);

// Bruno
var json = res.getBody();
var token = json.access_token;
bru.setEnvVar("auth_token", token);

2. Base64 helpers

// Postman
const payload = JSON.parse(atob(token.split('.')[1]));

// Bruno (Buffer.from works in Bruno's Node.js environment)
const payload = JSON.parse(Buffer.from(token.split('.')[1], 'base64').toString());

3. Environment files

vars {
  base_url: http://localhost:8000
  api_key: {{process.env.API_KEY}}
}

4. In‑line Markdown docs

Bruno lets every request double as a mini README:

docs {
  # User Authentication
  
  `POST` `/api/v2/auth/login`

  ## Overview

  Returns JWT tokens.

  ⚡ **Rate Limit**: 5/min per IP

  ...
}

Rich tables, code fences, emojis etc can be included. Because it’s Markdown, Claude Code and other AI tools parse it effortlessly.

5. Organizing your Bruno collections

After migration, we organized our Bruno files by feature rather than by API version. Here’s our structure:

bruno/
├── .env                  # Local secrets (git-ignored)
├── .env.example          # Template for team members
├── .gitignore           # Ensures .env stays local
├── environments/
│   ├── local.bru
│   ├── staging.bru
│   └── production.bru
├── auth/
│   ├── login.bru
│   ├── refresh_token.bru
│   └── logout.bru
├── users/
│   ├── get_profile.bru
│   ├── update_profile.bru
│   └── list_users.bru
├── analytics/
│   ├── dashboard_metrics.bru
│   └── export_reports.bru
└── integrations/
    ├── stripe/
    │   └── create_payment.bru
    └── webhooks/
        └── incoming_webhooks.bru

Security tip: Always add .env to your .gitignore. Create a .env.example with dummy values so team members know what environment variables to set:

# .env.example
API_BASE_URL=http://localhost:8000
API_KEY=your-api-key-here
JWT_TOKEN=will-be-set-by-login-script
DEFAULT_COMPANY_ID=1234
TEST_USERNAME=testuser
TEST_PASSWORD=testpass

Each .bru file can include documentation, pre/post scripts, and environment variable references. This structure makes it easy to:

• Find related endpoints quickly
• Review API changes in PRs
• Generate documentation by feature area
• Manage permissions at the folder level
• Keep sensitive data out of version control

Scripts that saved us hours

Environment converter (Python)

The script is available in this GitHub gist: migrate_postman_envs.py

Command

$ uv run migrate_postman_envs.py ./postman_environments/ ./bruno_environments/

Output

🔄 Processing 10 file(s)...

✅ Converted webhook_env.json → ./bruno_environments/webhook_env.bru
✅ Converted local_env.json → ./bruno_environments/local_env.bru
✅ Converted production.json → ./bruno_environments/production.bru
✅ Converted staging.json → ./bruno_environments/staging.bru
✅ Converted development.json → ./bruno_environments/development.bru
✅ Converted test_env.json → ./bruno_environments/test_env.bru
✅ Converted qa_env.json → ./bruno_environments/qa_env.bru
✅ Converted sandbox.json → ./bruno_environments/sandbox.bru
✅ Converted integration.json → ./bruno_environments/integration.bru
✅ Converted demo_env.json → ./bruno_environments/demo_env.bru

✨ Done! Converted 10/10 file(s)

Validator Command

The script can be found here: validate_bruno_files.py

Command

$ uv run validate_bruno_files.py ./bruno_environments/

Output

🔍 Validating 10 Bruno file(s)...

✅ demo_env.bru
✅ development.bru
✅ integration.bru
✅ local_dev.bru
✅ performance_test.bru
✅ production.bru
✅ qa_testing.bru
✅ sandbox.bru
✅ staging.bru
✅ user_acceptance.bru

📊 Summary: 10/10 file(s) valid

Request Migration Script

For the actual requests, here’s a simple bash script:

#!/bin/bash
# convert_postman_scripts.sh

# Common replacements
sed -i 's/JSON\.parse(responseBody)/res.getBody()/g' *.bru
sed -i 's/atob(/Buffer.from(/g' *.bru
sed -i 's/pm\.environment\.set(/bru.setEnvVar(/g' *.bru

# Fix dictionary access patterns
sed -i 's/jsonData\["\([^"]*\)"\]/jsonData.\1/g' *.bru

Pro tip: Run this on a copy first. Some replacements might need manual review, especially if you have complex string patterns.

Common Migration Patterns

Response Validation

Add defensive checks when migrating:

// Old Postman way (often broke with null responses)
var id = JSON.parse(responseBody)["data"]["id"];

// Better Bruno pattern
var response = res.getBody();
if (response && response.data && response.data.id) {
  bru.setEnvVar("resource_id", response.data.id);
} else {
  console.error("Unexpected response structure:", response);
}

Test Migration

If you have Postman tests:

// Postman test
pm.test("Status code is 200", function () {
    pm.response.to.have.status(200);
});

// Bruno test
test("Status code is 200", function () {
    expect(res.getStatus()).to.equal(200);
});

The AI Integration Revolution

Here’s where things get exciting. I created a custom Claude slash command that analyzes our Bruno files and generates comprehensive documentation by inspecting our Django codebase.

The /bruno-api Command

Instead of maintaining documentation scripts, we taught Claude to understand our codebase. Here’s how the actual command works:

# Custom Claude Command: /bruno-api

When the user types `/bruno-api [bruno-file-path]`, you will:

1. **Parse the Bruno File**
   - Extract the HTTP method, endpoint URL, headers, and body structure
   - Identify authentication requirements (Bearer token, API key, etc.)
   - Note any pre/post-request scripts for context

2. **Reverse Engineer the Backend**
   - Use the endpoint URL to find the Django URL pattern:
     path('api/v2/users/', UserViewSet.as_view())
     re_path(r'^api/v1/reports/(?P<pk>\d+)/$', ReportDetailView.as_view())
   - Locate the corresponding view/viewset class
   - For Django Ninja endpoints, find the router and operation functions

3. **Deep Code Analysis**
   - Extract serializer fields, types, validation rules
   - Identify permission classes and authentication requirements
   - Trace through the view method to understand:
     - Query parameters and filtering
     - Data transformations
     - External service calls
     - Error conditions

4. **Generate Comprehensive Documentation**
   Including:
   - Full API endpoint documentation
   - TypeScript interfaces for request/response
   - React Query hooks with error handling
   - Authentication requirements
   - Business logic notes (caching, rate limits, etc.)
   - Common error scenarios

Note: For brevity, we have excluded a lot of details like allowed-tools, Context etc from the command above. But these and other internal project specific details are present in our /bruno-api command.

How It Analyzes Your Code

The sophistication comes from how Claude connects all the pieces:

# Claude's Analysis Process:

1. Bruno file says: GET /api/v2/analytics/inclusion-scores/
2. Find in urls.py: path('api/v2/analytics/inclusion-scores/', InclusionScoresView.as_view())
3. Find InclusionScoresView class
4. Analyze the get() method:
   - What serializer? InclusionScoresSerializer
   - What permissions? IsAuthenticated + HasAnalyticsAccess
   - What does it do? Aggregates survey data with demographic breakdowns
5. Check serializer fields and validation
6. Find related models and business logic
7. Generate complete, accurate documentation

The magic: This is more than just brittle script parsing AST. Claude Code understands our code semantically, follows imports, and comprehends business logic and can inspect multiple aspects of an API.

Real Example Output

Input: Simple Bruno file from Postman migration - see this basic file (just endpoint + auth)

Command: /bruno-api bruno/analytics/user_metrics.bru

Output: Claude analyzes the Django codebase and generates comprehensive documentation. Here’s a small sample:

// TypeScript Interfaces (auto-generated from Django serializers)
interface UserMetricsResponse {
  count: number;
  next: string | null;
  results: Array<{
    user_id: string;
    email: string;
    last_active: string;
    total_sessions: number;
    sessions_this_month: number;
    avg_session_duration: string;
    status: 'active' | 'inactive';
    role: string;
    department: string | null;
  }>;
}

// React Query Hook (with error handling derived from Django views)
export const useUserMetrics = (companyId: string, params?: UserMetricsParams) => {
  return useQuery({
    queryKey: ['user-metrics', companyId, params],
    queryFn: async () => {
      const response = await apiClient.get(
        `/api/v2/companies/${companyId}/user-metrics/`,
        { params }
      );
      return response.data;
    },
    enabled: !!companyId,
    retry: (failureCount, error: any) => {
      // Smart retry logic based on Django view error handling
      if (error?.response?.status === 401 || error?.response?.status === 403) {
        return false;
      }
      return failureCount < 3;
    }
  });
};

This is just a fraction of the output. See the complete generated documentation which includes:

• Complete API documentation with request/response examples
• Comprehensive error handling for all status codes
• Authentication and permission requirements
• TypeScript interfaces for all data structures
• React Query hooks with infinite scrolling support
• Testing examples and integration patterns
• Business logic notes and performance considerations
• Implementation details and database optimization notes

The key insight: Claude reads the actual implementation, so the documentation is always accurate. It can still make mistakes, so it’s always critical to review the files and nudge it in right direction.

Adapting for Your Framework

This approach works for any framework:

• Django: Find views, serializers, permissions
• Express: Parse routes, middleware, validators
• Rails: Analyze controllers, strong params
• FastAPI: Extract Pydantic models, dependencies

The Unexpected Benefits

1. Code Reviews for API Changes

When someone changes an API, reviewers can see it in the PR:

git diff api/users/create.bru

+ body:json {
+   {
+     "email": "{{email}}",
+     "role": "{{role}}",
+     "department": "{{department}}"  // New field added
+   }
+ }

Breaking changes are caught before deployment, not after.

2. AI-Powered API Discovery

New and existing team members can ask:

• “Show me all endpoints that return user data”
• “How do I paginate through results?”
• “Generate TypeScript types for the profile endpoint”

Claude reads your Bruno collections and provides accurate answers and vice-versa.

3. Documentation That Stays Fresh

Since docs live with code, they’re more likely to stay updated. We are working on a pre-commit hook that is going to remind developers to update Bruno files when an API related change is made.

Getting Your Team Started

Start Small, Move Fast

We did our entire migration over a weekend, and you can too. Here’s what worked for us:

1. Pick your proof of concept - Choose one well-used Postman collection
2. Run the migration scripts - Use the Python converters linked above
3. Set up your first AI command - Start with our /bruno-api template
4. Show, don’t tell - Generate docs for one endpoint and share with the team

The Aha Moment

The real buy-in happens when developers see:

• Their API changes appearing in PR diffs
• Claude Code generating accurate TypeScript interfaces
• Documentation that actually matches the code
• No more “update Postman” tickets in the backlog

Practical Next Steps

• Today: Export one Postman collection, import to Bruno
• Tomorrow: Create your first custom Claude command
• This Week: Add .bru files to your repo and update PR templates
• Next Sprint: Deprecate Postman licenses and celebrate the cost savings

Common Questions We Heard

• “Can QA still use it?” → Yes, Bruno has a UI too (and it’s free)
• “What if we need to go back?” → Keep Postman exports for 30 days, but we never looked back

Creating Your Own AI Commands

Here’s a complete example you can adapt:

# .claude/commands/bruno-api.md

You are an API documentation expert for our [Framework] application.

When user types /bruno-api [file-path]:

1. Read the Bruno file at the specified path
2. Extract: method, URL, headers, body structure
3. Find the implementation:
   - For Express: Find app.get/post/put in routes/
   - For Django: Find path() in urls.py, then view
   - For Rails: Find route in config/routes.rb
4. Analyze the handler/controller to determine:
   - Required parameters and validation
   - Authentication/authorization 
   - Response structure
   - Error cases
5. Generate documentation including:
   - Clear description of what the endpoint does
   - Request/response examples with real data
   - [Your frontend framework] integration code
   - Common errors and how to handle them

Use our conventions:
- TypeScript for all interfaces
- Include data validation rules
- Show rate limits if applicable
- Note any side effects (emails, webhooks, etc.)

Implementation tip: Start with one endpoint type (e.g., CRUD operations) and expand from there.

What This Means for Engineering Teams

The shift from Postman to Bruno in 2025 is really a shift in how we think about API documentation. It’s no longer a separate artifact that gets out of sync. It’s part of your codebase, reviewed like code, and enhanced by AI.

Immediate Benefits We Measured

• API documentation time: Reduced from days to hours
• Documentation quality: Rich Markdown docs with examples, tables, and diagrams (vs. limited formatting options and inconsistent AI-generated docs in Postman)
• Breaking changes caught: Significant improvement during code review (from rarely caught to consistently visible in PRs)
• Team participation: Entire team can now contribute without needing a Postman seat
• Onboarding time: New engineers integrate APIs faster with self-documenting collections and AI-generated examples

What is pghistory.context?

pghistory.context is a decorator and a function that is part of the fantastic django-pghistory project. I would cover how we use django-pghistory in a separate post.

When django-pghistory is keeping track of changes to Django models, there are several instances where we want to include additional information with the tracked changes. This is where pghistory.context comes into picture to easily capture such metadata.

We are making use of pghistory.middleware.HistoryMiddleware to collect basic fields that we want to store with each change. This includes IP address, URL and the User who made the request. Some of this is done by the library itself (comments removed for brevity) as seen in the code below.

class HistoryMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response

    def get_context(self, request) -> Dict[str, Any]:
        user = (
            request.user._meta.pk.get_db_prep_value(request.user.pk, connection)
            if hasattr(request, "user") and hasattr(request.user, "_meta")
            else None
        )
        return {"user": user, "url": request.path}

    def __call__(self, request):
        if request.method in config.middleware_methods():
            with pghistory.context(**self.get_context(request)):
                if isinstance(request, DjangoWSGIRequest):
                    request.__class__ = WSGIRequest
                elif isinstance(request, DjangoASGIRequest):
                    request.__class__ = ASGIRequest

                return self.get_response(request)
        else:
            return self.get_response(request)

An interesting thing happening here is that the request-response is wrapped in the pghistory.context context manager. That means all call to pghistory.context anywhere in views would keep on accumulating the metadata that pghistory.context is getting. This is a powerful pattern for audit tracking without having to manually pass context through your application code. Every database change automatically gets tagged with “who” (user) and “where” (URL) the change happened.

But this also has an unintended effect, let’s use an example to explain it(gist)

In [1]: import pghistory
   ...:
   ...: with pghistory.context(url="/foo/bar", user="bugs-bunny") as pg:
   ...:     print(f"Level 1-1: {pg.metadata}")
   ...:
   ...:     with pghistory.context(foo="bar"):
   ...:         print(f"Level 2-1: {pg.metadata}")
   ...:         with pghistory.context(spam="eggs"):
   ...:             print(f"Level 3-1: {pg.metadata}")
   ...:
   ...:         with pghistory.context(hannah="montana"):
   ...:             print(f"Level 3-2: {pg.metadata}")
   ...:             with pghistory.context(timon="pumba"):
   ...:                 print(f"Level 4-1: {pg.metadata}")
   ...:
   ...:     print(f"Level 1-2: {pg.metadata}")
   ...:     with pghistory.context(monty="python"):
   ...:         print(f"Level 2-2: {pg.metadata}")
   ...:
   ...:
   ...: with pghistory.context(guido="bdfl") as pg_2:
   ...:     print(f"Level 1-4: {pg_2.metadata}")
   ...:
   ...: print(f"Level 1-5: {pg.metadata}")
   ...: print(f"Level 1-6: {pg_2.metadata}")
   ...:

Level 1-1: {'url': '/foo/bar', 'user': 'bugs-bunny'}
Level 2-1: {'url': '/foo/bar', 'user': 'bugs-bunny', 'foo': 'bar'}
Level 3-1: {'url': '/foo/bar', 'user': 'bugs-bunny', 'foo': 'bar', 'spam': 'eggs'}
Level 3-2: {'url': '/foo/bar', 'user': 'bugs-bunny', 'foo': 'bar', 'spam': 'eggs', 'hannah': 'montana'}
Level 4-1: {'url': '/foo/bar', 'user': 'bugs-bunny', 'foo': 'bar', 'spam': 'eggs', 'hannah': 'montana', 'timon': 'pumba'}
Level 1-2: {'url': '/foo/bar', 'user': 'bugs-bunny', 'foo': 'bar', 'spam': 'eggs', 'hannah': 'montana', 'timon': 'pumba'}
Level 2-2: {'url': '/foo/bar', 'user': 'bugs-bunny', 'foo': 'bar', 'spam': 'eggs', 'hannah': 'montana', 'timon': 'pumba', 'monty': 'python'}
Level 1-4: {'url': '/foo/bar', 'user': 'bugs-bunny', 'foo': 'bar', 'spam': 'eggs', 'hannah': 'montana', 'timon': 'pumba', 'monty': 'python', 'guido': 'bdfl'}
Level 1-5: {'url': '/foo/bar', 'user': 'bugs-bunny', 'foo': 'bar', 'spam': 'eggs', 'hannah': 'montana', 'timon': 'pumba', 'monty': 'python', 'guido': 'bdfl'}
Level 1-6: {'url': '/foo/bar', 'user': 'bugs-bunny', 'foo': 'bar', 'spam': 'eggs', 'hannah': 'montana', 'timon': 'pumba', 'monty': 'python', 'guido': 'bdfl'}

Here as we can see, regardless of which level of context manager we are in, or even if it’s a completely new context manager, the keys keep on accumulating.

This is not something we want because there are multiple places in our business logic where we want to track different changes with different metadata instead of using the metadata from previous calls to pghistory.context.

def my_view(request):
    if request.method == "POST":
        with pghistory.context(type="POST"):
            # Perform some action
            # save audit log
            pass

    elif request.method == "GET":
        with pghistory.context(type="GET"):
            # Perform some action
            # save audit log
            pass

    # External API call
    # Update an internal state
    with pghistory.context(type="EXTERNAL"):
        # Perform some action
        # save audit log
        pass

    # Another action that updates the internal state
    with pghistory.context(type="INTERNAL"):
        # Perform some action
        # save audit log
        pass

Now for the above, the ideal behaviour we want is that every audit log should get the keys that were included in its context, but not the ones added by other contexts, apart from the common keys coming from the middleware.

Solution

We want a solution where we want the keys added by the middleware to persist, but we do not want the keys added inside the views to persist, and every time we exit from a context the keys should be removed from the metadata.

For this the solution was to override the default implementation of pghistory.context. Let’s say the custom implementation is called custom_pg_context, then we want the output of the program we had earlier on to be:

The key change we made was introducing a new parameter called persist. When set to True (the default), metadata keys will persist across nested contexts and remain available to other contexts. When set to False, metadata keys added in the current context will automatically be removed upon exiting the context, ensuring they don’t leak into subsequent operations.

Here only want the user and url to persist (later on timon as well)

Level 1-1: {'url': '/foo/bar', 'user': 'bugs-bunny'}
Level 2-1: {'url': '/foo/bar', 'user': 'bugs-bunny', 'foo': 'bar'}
Level 3-1: {'url': '/foo/bar', 'user': 'bugs-bunny', 'foo': 'bar', 'spam': 'eggs'}
# Notice the lack of 'spam': 'eggs' in 3-2
Level 3-2: {'url': '/foo/bar', 'user': 'bugs-bunny', 'foo': 'bar', 'hannah': 'montana'}
Level 4-1: {'url': '/foo/bar', 'user': 'bugs-bunny', 'foo': 'bar', 'hannah': 'montana', 'timon': 'pumba'}
# Everything has been removed when we hit 1-2 except for url and user
# and timon, because these we want to persist.
Level 1-2: {'url': '/foo/bar', 'user': 'bugs-bunny', 'timon': 'pumba'}
Level 2-2: {'url': '/foo/bar', 'user': 'bugs-bunny', 'timon': 'pumba', 'monty': 'python'}
Level 1-4: {'url': '/foo/bar', 'user': 'bugs-bunny', 'timon': 'pumba', 'guido': 'bdfl'}
Level 1-5: {'url': '/foo/bar', 'user': 'bugs-bunny', 'timon': 'pumba', 'guido': 'bdfl'}
Level 1-6: {'url': '/foo/bar', 'user': 'bugs-bunny', 'timon': 'pumba', 'guido': 'bdfl'}

Now this is what the code looks like now(gist):

from __future__ import annotations

from contextvars import ContextVar
from copy import deepcopy

import pghistory

context_stack: ContextVar[list[custom_pg_context]] = ContextVar(
    "context_stack", default=[]
)


class custom_pg_context(pghistory.context):
    def __init__(self, *, persist: bool = True, **metadata):
        self._persist = persist
        self._local_metadata = deepcopy(metadata)
        self.token = context_stack.set(context_stack.get() + [self])
        super().__init__(**metadata)

    def __enter__(self):
        return_value = super().__enter__()
        self._tracker_value = return_value
        return return_value

    def __exit__(self, exc_type, exc_val, exc_tb):
        try:
            if not self._persist:
                # Removing keys added in this context from parent contexts
                stack = context_stack.get()[:-1]
                for parent in stack:
                    if parent._tracker_value:
                        for key in self._local_metadata:
                            parent._tracker_value.metadata.pop(key, None)

            return super().__exit__(exc_type, exc_val, exc_tb)
        finally:
            current_stack = context_stack.get()
            if current_stack and current_stack[-1] is self:
                context_stack.set(current_stack[:-1])

    @staticmethod
    def get_parent_context() -> custom_pg_context | None:
        stack = context_stack.get()
        return stack[-2] if len(stack) > 1 else None


with custom_pg_context(url="/foo/bar", user="bugs-bunny", persist=True) as pg:
    print(f"Level 1-1: {pg.metadata}")

    with custom_pg_context(foo="bar", persist=False):
        print(f"Level 2-1: {pg.metadata}")
        with custom_pg_context(spam="eggs", persist=False):
            print(f"Level 3-1: {pg.metadata}")

        with custom_pg_context(hannah="montana", persist=False):
            print(f"Level 3-2: {pg.metadata}")
            with custom_pg_context(timon="pumba", persist=True):
                print(f"Level 4-1: {pg.metadata}")

    print(f"Level 1-2: {pg.metadata}")
    with custom_pg_context(monty="python", persist=False):
        print(f"Level 2-2: {pg.metadata}")


with custom_pg_context(guido="bdfl") as pg_2:
    print(f"Level 1-4: {pg_2.metadata}")

print(f"Level 1-5: {pg.metadata}")
print(f"Level 1-6: {pg_2.metadata}")

What’s happening here?

1. We are making use of ContextVar from the contextvars module to maintain a stack of all contexts and identify parents and children of any particular context we are in.

2. We create a custom class custom_pg_context that inherits from pghistory.context, with an additional persist parameter that defaults to True.

3. In the constructor, we keep a copy of the local metadata and add ourselves to the context stack.

4. During __enter__, we store the tracker object returned by the parent class for later use.

5. The magic happens in __exit__:

   • If persist=False, we look through all parent contexts in the stack and remove any keys from our local metadata.
   • This ensures that temporary metadata isn’t carried forward to other contexts.
   • Keys with persist=True (like our user, url, and timon) remain in the metadata across all contexts.

6. We also maintain proper stack management by removing our context from the stack when exiting.

7. The get_parent_context() static method provides a way to access the parent context if needed.

This implementation solves our problem by allowing us to control which metadata persists across different parts of our application. We can use persist=True for global tracking (like user and URL from middleware) and persist=False for local, context-specific metadata that shouldn’t leak into other operations.

Using the Custom Context in Django

To implement this in a Django project, you would:

1. Create a new module (e.g., core/context.py) with the custom_pg_context implementation.
2. Update your middleware to use this custom context manager instead of the default one:

from core.context import custom_pg_context

class CustomHistoryMiddleware(HistoryMiddleware):
    def __call__(self, request):
        if request.method in config.middleware_methods():
            # Use our custom context manager with persist=True
            with custom_pg_context(persist=True, **self.get_context(request)):
                if isinstance(request, DjangoWSGIRequest):
                    request.__class__ = WSGIRequest
                elif isinstance(request, DjangoASGIRequest):
                    request.__class__ = ASGIRequest

                return self.get_response(request)
        else:
            return self.get_response(request)

3. In your views and services, use the custom context manager with persist=False for operation-specific metadata:

from core.context import custom_pg_context

def my_view(request):
    # Use persist=False for view-specific metadata
    with custom_pg_context(operation="view_details", persist=False):
        # Perform database operations
        item.save()  # This will include the operation metadata
        
    # Start a new context without leaking previous metadata
    with custom_pg_context(operation="update_related", persist=False):
        # Perform more operations
        related_item.save()  # Only includes this operation's metadata

Performance Considerations

One thing to note is that this approach does add some overhead, especially with deeply nested contexts. For most applications, this overhead is negligible compared to database operations, but it’s something to keep in mind for performance-critical code paths.

If you’re concerned about performance, you could optimize the implementation further:

1. Use a more efficient data structure for the context stack
2. Limit the depth of context nesting
3. Consider using a profiler to identify bottlenecks in your specific use case

Conclusion

By extending pghistory.context with our custom implementation, we’ve solved the metadata leakage problem while maintaining the convenience of the context-based tracking system. This approach gives us fine-grained control over which metadata persists across different operations, making our audit logs more accurate and meaningful.

In a real-world Django application, this means we can add specific tracking metadata to different operations without polluting the audit trail of unrelated operations, while still maintaining the global context from middleware.

He wasn’t just an inventor; he created institutions and actively built himself. And he did it in a way that it feels surprisingly relevant today, without losing his sense of humour.

Some things that stuck with me.

Benjamin Franklin: The Original Self-Made Person

He’s the most literal example of self-made people. He didn’t just bootstrap a career, he designed who he wanted to be and iterated toward it.

At one point in his life, he wrote out a list of 13 virtues. Things like temperance, resolution, industry, humility. And then created a tracking system to improve one per week. Like a habit tracker, but from the 1700s. He had a grid and he reviewed it nightly. He built an internal feedback loop before we had that language.

Did it make him perfect? Not even close. But he said something that stuck with me:

“But on the whole, though I never arrived at the perfection I had been so ambitious of obtaining, but fell far short of it, yet I was, by the endeavour, a better and happier man than I otherwise should have been had I not attempted it; as those who aim at perfect writing by imitating the engraved copies, their hand is mended by the endeavour, and is tolerable while it continues fair and legible”

That’s the point.

Learning by reverse-engineering

Franklin taught himself how to write by reading essays in The Spectator. He used to summarize the key arguments, and then he would try to rewrite the entire piece from scratch a few days later. Then he’d compare his version with the original and note where he went wrong.

This is a simple yet brilliant way to learn. He built muscle memory towards it with this approach.

Curious, Playful, and Unafraid to Experiment

This is my favorite part from the episode. Franklin did what interested him, even if it didn’t have any obvious ROI.

Someone once asked him why he was experimenting with hot air balloons, and he responded:

“What is the use of a newborn baby?”

He just liked it and trusted that something good would come out of following that instinct. Which is actually how most breakthroughs happen.

Most productive people aren’t just focussed on productivity, they are the ones who still let them chase just because it’s fun or weird or fascinating.

Peer-driven growth

Franklin started something called the Junto. It was a group of ambitious, curious people who met weekly to debate ideas, share essays, and challenge each other to improve.

This wasn’t about ego or point-scoring. The rule was: no arguing, no personal attacks, no pretending you’re the smartest person in the room. Just honest enquiry and curiosity.

Junto lasted for 40 years.

And it wasn’t just talk — real things came out of it. The idea for America’s first subscription library was born in the Junto and became the Library Company of Philadelphia.

Serious About Humour

Franklin was funny, like actually funny. But he didn’t use it just to entertain — it was how he disarmed people, made ideas stick, and built a public persona people trusted.

When he was 15, he wrote letters under the name Silence Dogood and dropped them anonymously on the doorstep of his brother’s print shop. They were supposedly written by a widowed woman who was “handsome and sometimes witty.” That “sometimes” kills me. It’s the perfect character detail, funny, self-aware, instantly charming.

Later, as the fake author of Poor Richard’s Almanack, he’d write snarky fake feuds with imaginary critics. One of my favorites: he predicted another writer’s death, then when the guy wrote in angry that he was very much alive, Franklin insisted the letter must be a forgery — because the real man would never write so poorly.

Franklin didn’t take himself too seriously. And that was part of his power.

His charm made people want to read him. It made political enemies underestimate him. And it made allies feel like they were in on the joke.

In a world full of Very Serious Men trying to win arguments, Franklin laughed — and then won anyway.

Lessons

Franklin wasn’t just a scientist, or a writer, or a politician, or a businessman. He was all of those things. But more than anything, he was a systems thinker for human life.

He designed how he lived.
He chased what made him curious.
He got better in public.
He didn’t pretend to be perfect.
And he never let ambition overshadow joy.

It’s a good model:

• Create structures that support your growth.
  Build your own systems—track habits, design feedback loops, give your goals shape.
• Form communities that push you forward.
  Don’t just network. Find people who challenge, teach, and grow with you.
• Chase ideas without obsessing over immediate ROI.
  Some of the most useful things start as playful experiments.
• Keep improving, even if you’ll never be perfect.
  Progress matters more than perfection. The act of trying changes you.
• Don’t wait for permission to reinvent yourself.
  Franklin didn’t. You don’t need anyone’s blessing to evolve.
• Use humor as a strategic advantage.
  Be sharp, but approachable. Laugh. It makes people listen — and keeps you human.

Benjamin Franklin didn’t just build a remarkable life — he left behind a blueprint we can all still follow.

The 13 virtues

• Temperance – Eat not to dullness; drink not to elevation.
• Silence – Speak not but what may benefit others or yourself; avoid trifling conversation.
• Order – Let all your things have their places; let each part of your business have its time.
• Resolution – Resolve to perform what you ought; perform without fail what you resolve.
• Frugality – Make no expense but to do good to others or yourself; waste nothing.
• Industry – Lose no time; be always employed in something useful; cut off all unnecessary actions.
• Sincerity – Use no hurtful deceit; think innocently and justly, and, if you speak, speak accordingly.
• Justice – Wrong none by doing injuries or omitting the benefits that are your duty.
• Moderation – Avoid extremes; forbear resenting injuries so much as you think they deserve.
• Cleanliness – Tolerate no uncleanliness in body, clothes, or habitation.
• Tranquillity – Be not disturbed at trifles or at accidents common or unavoidable.
• Chastity – Rarely use venery (sexual indulgence) but for health or offspring, never to dullness, weakness, or the injury of your own or another’s peace or reputation.
• Humility – Imitate Jesus and Socrates.

Risk is often overestimated; opportunity is often underestimated.

Podcast: https://www.founderspodcast.com/episodes/374-rare-jeff-bezos-interview

YouTube: https://youtu.be/s71nJQqzYRQ

Build something that outlasts you

Since early days of Amazon Bezos was thinking long-term and was quite clear that he wanted to build a company that would outlast him.

Spotify’s Daniel Ek shares a similar sentiment: Companies grow like children — initially they are a reflection of their creator, but over time they develops its own traits and characteristics.

Steve Jobs said something similar: You build a company that will stand for something a generation or two from now.

Follow your curiosity

One of his driving principles is “I wake up and follow my curiosity”. Paul Graham calls curiosity the best guide for doing great work.

Instead of over-analyzing your next move, follow your curiosity and let that be your compass.

AI is the new electricity

He describes AI as just an horizontal enabling layer, just like electricity. When electricity came out, people only wanted light bulbs; everything else came after. People weren’t putting electricity in their homes; they were putting light bulbs in their homes. At that point they weren’t even thinking of appliances, switchboard or anything, and there wasn’t even a concept of off switch.

Similarly, AI, like electricity before, will be embedded into everything, transforming industries in unpredictable ways.

If AI is an enabling layer, ask: How can AI supercharge my business, my workflows, my products?_ The most valuable companies of the next decade will answer this question well.

Link to his 2003 TED talk: link

The AWS Insight: Computing as a Utility

His inspiration for AWS came from a 300 year old Luxembourg brewery that had its own power generator — because at that time, there were no power grids. Similarly, before AWS companies had their own data center and he realized that it’s not going to last.

That meant companies had to focus on something that wasn’t their core business, true for anything in the past that had to worry about generating their own electricity.

The Power of Being Misunderstood

He has long accepted that he will be misunderstood. He prioritizes doing what he believes is right over chasing a good PR. He said, if even your closest loved ones do not fully know you, how can you expect others to understand you?

Warren Buffett and Charlie Munger have similar philosophy. They keep an inner scorecard — measuring success by their own principles, not external validation.

If no one skeptical about what you’re doing, you’re probably not pushing hard enough.

Meetings Should Be Messy

He insists that meetings should wander, and people should not come into meetings with rehearsed responses. He also always speaks last and is always asking if anyone has any dissenting opinions.

Meetings should be raw, unfiltered and people should feel comfortable challenging ideas.

Optimism is a Moral Duty

Edwin Land, Steve Jobs’ hero, believed that “optimism is a moral duty”. Bezos embodies this mindset, he truly believes there has never been a better time to be alive.

As a leader, you need to exude optimism. Ambitious projects require an unshakeable belief in a better future.

Every Decision is an Opportunity Cost

He explains why he does so few interview. “Every minute I’m doing this, I’m not spending that time doing something else”.

Time is your most valuable resource — guard it fiercely. Choose only what moves the needle for you.

Ferocious Intelligence

Charlie Munger once said, “He is ferociously intelligent.”

The relentless curiosity, optimism, and long-term thinking are what makes Bezos worth studying.

• Assign a buddy or mentor but avoid using a direct manager for this, as the person won’t be that open and comfortable with their manager.

• Less emphasis on paperwork, focus more on mission, impact, values and culture.

• Set up an interactive onboarding task to keep them engaged from day 1.

• Encourage them to ask questions, voice concerns, and share ideas openly. It’s very important to teach them to not remain limited to DMs and become comfortable in public channels with the whole team.

• Help them set up 1:1s with different members of the team

First month

• Make sure their local set up is up and running.

• Assign small tasks to them. This can be small bugfixes, features and any documentation update. Documentation update is a must, especially the docs they are going through during onboarding.

  Always have tasks in your backlog that are tagged as good-first-task.

• Very important for them to have something live on prod as soon as possible, as that builds up confidence. That means the initial tasks should not involve a lot of back and forth. The goal is to move fast and deploy on prod and get a quick understanding of the system.

• Ask the senior folks with whom they will be directly working to set up regular pair-programming sessions with them. Very important in early days to ensure they feel comfortable and have someone who actually shows them how things are being done.

• Encourage them to dogfood the product. We want their own fresh perspective the way a new user interacts with the product.

• Set up weekly 1:1 with them as manager.

• Have them identify gaps in documentation or processes for a fresh perspective.

Second month

• Set up slightly bigger tasks for them. Something that would require critical thinking and creativity from their end. Do not spoon feed the task, instead let them go through it themselves and come back to you with their own understanding.

  The goal is to help build confidence and see if they are able to fill in the gaps for the things that are not mentioned explicitly.

• Keep an eye on whether they are asking for help or struggling silently.

• Check if they are actively participating in Slack discussions and in meetings.

• Ensure they are learning from mistakes and are avoiding them in the future. It’s also important to see if they pass on the same knowledge to others in the team.

Third month

• By the end of this month they should be actively contributing to technical or non-technical discussions.

• Delivering at least one meaningful project really helps overcome imposter syndrome.

• They should be able to work on their own to an extent.

• They should have strong opinions on things. They should challenge, ask questions, and propose solutions. If they are too passive, they are likely not a good fit.

• If they are repeating the same mistakes, either coding or process related, then try to understand what’s causing it.

• Check if they appear busy but are not taking any ownership. Do they help others but aren’t finishing their own work? These are some clear signs of procrastination.

• Keep an eye on their code reviews. Are they mostly just accepting everything and not really providing much feedback? How are they interacting in code reviews?

I have recently started using uv a lot, specially for quick standalone scripts, most of these scripts are generated by various LLMs for different tasks. The biggest pain point I had with these scripts was to setup a virtual env, installing the packages and asking ChatGPT and other LLMs to give me requirements file/list.

uv changed that by utilizing PEP 723 and embedding these requirement inline to the script.

At Diversio, I frequently use GitHub Copilot a lot, and there I’ve added a small instruction to make sure it’s embedding the requirements in the script itself.

When asked to generate a uv based Python script, at the very top of the file, insert an inline metadata block listing all external packages in this format:

# /// script
# requires-python = ">=3.11"
# dependencies = [
#   "requests<3",
#   "rich",
# ]
# ///

Now, I gave it the prompt:

Give me a uv based Python script that pings google.com and prints the output in colorful tabular format.

You can see its output here

➜ uv run uv_pep_723_example.py
Installed 9 packages in 207ms
Pinging google.com 5 times...
Ping 1: Success - 221.01 ms
Ping 2: Success - 461.52 ms
Ping 3: Success - 288.88 ms
Ping 4: Success - 216.29 ms
Ping 5: Success - 205.57 ms

                 HTTP Ping Results for google.com
┏━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ # ┃ Timestamp           ┃ Host       ┃ Status  ┃ Response Time ┃
┡━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ 1 │ 2025-03-05 00:25:16 │ google.com │ Success │     221.01 ms │
│ 2 │ 2025-03-05 00:25:18 │ google.com │ Success │     461.52 ms │
│ 3 │ 2025-03-05 00:25:19 │ google.com │ Success │     288.88 ms │
│ 4 │ 2025-03-05 00:25:20 │ google.com │ Success │     216.29 ms │
│ 5 │ 2025-03-05 00:25:22 │ google.com │ Success │     205.57 ms │
└───┴─────────────────────┴────────────┴─────────┴───────────────┘

Summary Statistics:
  Successful requests: 5/5
  Min/Avg/Max: 205.57/278.65/461.52 ms
  Standard Deviation: 107.35 ms

Where is uv storing these packages?

➜ ls -l ~/.cache/uv/environments-v2/uv-pep-723-example-af4300630d2aced4/lib/python3.12/site-packages
total 12
drwxr-xr-x  3 monty staff   96 Mar  5 00:25 __pycache__
drwxr-xr-x  8 monty staff  256 Mar  5 00:25 certifi
drwxr-xr-x  9 monty staff  288 Mar  5 00:25 certifi-2025.1.31.dist-info
drwxr-xr-x 17 monty staff  544 Mar  5 00:25 charset_normalizer
drwxr-xr-x 10 monty staff  320 Mar  5 00:25 charset_normalizer-3.4.1.dist-info
drwxr-xr-x 12 monty staff  384 Mar  5 00:25 idna
drwxr-xr-x  8 monty staff  256 Mar  5 00:25 idna-3.10.dist-info
drwxr-xr-x 23 monty staff  736 Mar  5 00:25 markdown_it
drwxr-xr-x 10 monty staff  320 Mar  5 00:25 markdown_it_py-3.0.0.dist-info
drwxr-xr-x  9 monty staff  288 Mar  5 00:25 mdurl
drwxr-xr-x  8 monty staff  256 Mar  5 00:25 mdurl-0.1.2.dist-info
drwxr-xr-x 22 monty staff  704 Mar  5 00:25 pygments
drwxr-xr-x  9 monty staff  288 Mar  5 00:25 pygments-2.19.1.dist-info
drwxr-xr-x 21 monty staff  672 Mar  5 00:25 requests
drwxr-xr-x  9 monty staff  288 Mar  5 00:25 requests-2.32.3.dist-info
drwxr-xr-x 82 monty staff 2624 Mar  5 00:25 rich
drwxr-xr-x  8 monty staff  256 Mar  5 00:25 rich-13.9.4.dist-info
drwxr-xr-x 19 monty staff  608 Mar  5 00:25 urllib3
drwxr-xr-x  8 monty staff  256 Mar  5 00:25 urllib3-2.3.0.dist-info
-rw-r--r--  1 monty staff   18 Mar  5 00:25 _virtualenv.pth
-rw-r--r--  1 monty staff 4342 Mar  5 00:25 _virtualenv.py

The cache can be cleaned using the uv cache clean command as specified here.

References

• https://peps.python.org/pep-0723/
• https://docs.astral.sh/uv/guides/scripts/#declaring-script-dependencies
• https://docs.astral.sh/uv/concepts/cache/#clearing-the-cache

1. Introduction & Evolution at Diversio
2. How We Used to Do It (Pre-ChatGPT)
   • Resume Screening
   • Take Home Exercises
   • Zoom Interview
3. Interviewing in 2025
   • Posting the Job Ad
   • Filtering Resumes
   • Screening Round
   • Coding Round
   • Final Round

Introduction & Evolution at Diversio

At Diversio, our hiring process has evolved significantly over the years. In our early days, we relied heavily on detailed resume screenings and take-home exercises. Today, we’ve adapted to a new landscape where remote hiring, AI-assisted filtering, and a streamlined interview process help us manage the increased applicant volume while ensuring we find the best long-term fits.

Key Changes at a Glance:

• Transition from take-home exercises to real-time, remote interviews.
• Adoption of AI tools (like our custom GPT) to efficiently filter resumes.
• Increased focus on cultural fit and long-term alignment.
• Emphasis on transparent, remote hiring practices.

How We Used to Do It (Pre-ChatGPT)

We were very lucky in terms of hiring at Diversio until 2023—we had an excellent attrition rate and were fortunate to hire very talented interns from the University of Toronto, University of Waterloo, and Queen’s University. The full-time hires were mostly referrals.

1. Resume Screening

A resume screening was conducted by myself and the PM, including checking candidates’ past co-op remarks from other companies.

2. Take Home Exercises

We have a strong belief in asking questions that are directly relevant to what a candidate will do on a day-to-day basis at Diversio.

Depending on the position, we provided a different set of questions. Our exercises included:

• Programming Puzzle:
  Based on an internal algorithm and a problem we had solved in the past, this puzzle was reworded to sound engaging. The code was expected to be well documented and pass the test cases mentioned in the Python script.

  Goals here were:

  • Are they able to follow a complex problem statement (including the provided examples)?
  • Can they fill in the gaps? (i.e., how would they approach tasks at Diversio when some information is not mentioned explicitly, requiring them to figure it out on their own or use their best judgment?)
  • Can they identify edge cases on their own?
  • Can they write readable, well-documented code?

• Command Line Script:
  A task to parse our status page and provide details on how our apps have been performing over the past six months, including identifying the minimum and maximum response times of a service.

  Goals here were:

  • Are they able to parse HTML source code?
  • Are they inspecting elements to see if there are any internal APIs they can utilize?
  • Do they know how to build command-line utilities? (This is a skill we use frequently at Diversio.)
  • How is their code structure, documentation, and testing?

Candidates were expected to submit their solutions on GitHub, which we then reviewed (by myself and other engineers) and provided feedback on.

3. Zoom Interview

Walkthrough and Discussing Their Solutions

Candidates whose solutions passed our internal review were invited to a Zoom interview. During this round, we discussed their take-home exercises in depth—asking them to explain their solutions and thought processes, and throwing in some edge cases or tweaking the requirements to see if they could adapt their solution on the fly.

Real-Time Code Review

At Diversio, we invest a significant amount of time in code reviews because we believe that the effort not only benefits the company but also provides an opportunity to teach and learn from each other.

For this, we used a Python script deliberately filled with poorly written code—ignoring best practices, using inefficient data structures, and exhibiting problematic file I/O, networking, and error handling. The candidate was asked to explain what each section of the script did and to propose better solutions for each section or function, along with justifications.

Goals here were:

• To determine if they focus on solving the underlying problem rather than just critiquing the code.
• To assess their ability to read and understand someone else’s code.
• To gauge their knowledge of best practices and their ability to identify poor coding practices.

We allowed candidates to use Google during the interviews to look up anything they didn’t know. Since their screen was shared with us, we could observe how they searched for information and selected among the different results.

Interviewing in 2025

A lot has changed since 2023:

• We are a much bigger company now, so when we post a job, we receive many more applicants.
• Many layoffs in big tech companies and AI’s impact on dev jobs are resulting in significantly more applicants.
• Take-home interviews have been phased out in favor of real-time evaluations.
• Candidates are using resume tools to trick ATS auto-screening into thinking their resumes are the best.
• Remote hiring is the norm.

Posting the Job Ad

We use Collage internally, which automatically posts the job on Indeed. In addition, we use Instahyre and Hasjob. I also post about it on my LinkedIn to increase its reach.

When to Post?

We have found that posting on Friday works best for us. Within the first two days, we received around 400 applicants, and since it was the weekend, I had enough time to review them and filter out those who appeared promising. This built a pipeline for the next 1–2 weeks that was within our team’s capacity.

Filtering Resumes

• Size of Resume: Resumes longer than 1–2 pages are a direct reject, including those where candidates list every 10–20 bullet points under every role.

• Tenures:
  Short tenures at multiple places are a red flag, as they increase the likelihood of job-hopping. At Diversio, we want to invest in people for the long term.

• Work History: We prefer candidates with experience at product startups over service-based companies. We also assess how they explain the features they have worked on—whether they are concise and to the point or merely rely on buzzwords.
  Resumes that boast metrics like “x% improvement in signups/deployment” or “y% reduction in bugs” without proper context are directly rejected. Such figures often result from attempts to game ATS auto-screening and, frankly, are often meaningless—even after 12 years as a software engineer I don’t have such stats myself.

• GitHub and Portfolio (if any):
  We look at candidates’ pinned repositories. Boilerplate projects, past interview tasks, or college projects aren’t meaningful to us—we value significant contributions, a robust commit history, and open-source work.

• LinkedIn:
  We verify that the candidate’s LinkedIn profile matches their resume, that the listed companies are credible, and we review their post history.

ChatGPT

To speed up the filtering process, we use a custom GPT tool that evaluates most of the above criteria (except for the last two points), assigning points to each resume so that we can sort them efficiently.

Screening Round

Candidates selected after resume screening go through a screening round with a senior engineer at Diversio. We require screen sharing and that the camera be turned on to ensure there is no cheating. Candidates are allowed to use Google—but only on the shared screen.

The goal of this 30–45 minute call is to:

• Check their communication skills.
• Explain Diversio, the job, and its requirements.
• Walk through their resume, asking counter-questions to verify their project experience.
• Check basic coding skills that shouldn’t require any preparation.
• Discuss their notice period and salary expectations.

Candidates are then ranked based on feedback, and top performers move forward.

Coding Round

This round is conducted with the senior engineer leading the respective department (backend or frontend). It is a 1h 15min to 1h 30min call during which the candidate is required to share their screen and have their camera turned on.

This call includes:

• In-depth technical discussions around backend or frontend topics.
• Solving two coding questions and explaining their approach.

Final Round

The final round is a non-technical interview with me, reserved for candidates who performed exceptionally in the screening and coding rounds and met our criteria.

This interview is designed to provide a holistic view of the candidate by:

• Creating a Comfortable Environment: Beginning with a relaxed introduction that encourages open conversation.
• Assessing Collaboration & Adaptability: Exploring how candidates navigate teamwork challenges and manage unexpected changes within dynamic environments.
• Evaluating Cultural & Career Fit: Discussing work styles, long-term aspirations, and feedback experiences to ensure alignment with Diversio’s team values.
• Setting Clear Role Expectations: Clarifying work dynamics such as flexible hours and communication preferences, while highlighting Diversio’s vision and engineering culture.

Starting with Catalina the default shell was changed to zsh. You can verify your current shell using:

➜ echo $SHELL
/bin/zsh

If it’s not zsh for you, you can install it using Homebrew

brew install zsh

Once installed we need to make zsh the default shell using:

chsh -s /bin/zsh

Prezto installation

Prezto is a configuration framework for zsh. Another well known alternative is Oh My Zsh.

I was using Oh My Zsh previously but it had performance issues and switching to Prezto has helped with the performance issues. Oh My Zsh has a much bigger community and receives regular updates and is bit more beginner friendly.

Before installing make sure to make a back of your .zshrc file if you had one already as Prezto can ve problematic with existing .zshrc files.

Clone the Prezto repo

git clone --recursive https://github.com/sorin-ionescu/prezto.git "${ZDOTDIR:-$HOME}/.zprezto"

Link the new zsh config files provided by Prezto to your home directory

setopt EXTENDED_GLOB
for rcfile in "${ZDOTDIR:-$HOME}"/.zprezto/runcoms/^README.md(.N); do
  ln -s "$rcfile" "${ZDOTDIR:-$HOME}/.${rcfile:t}"
done

Once installed you’ll find the Prezto configuration in .zpreztorc file in your home directory. The default file looks like this: https://github.com/sorin-ionescu/prezto/blob/master/runcoms/zpreztorc

The list of modules are available here: https://github.com/sorin-ionescu/prezto/blob/master/runcoms/zpreztorc

zstyle ':prezto:load' pmodule \
 'lazy-load' \
 'environment' \
 'terminal' \
 'editor' \
 'history' \
 'directory' \
 'spectrum' \
 'utility' \
 'completion' \
 'osx' \
 'ssh' \
 'git' \
 'python' \
 'node' \
 'syntax-highlighting' \
 'history-substring-search' \
 'prompt' \
 'autosuggestions'

Here lazy-load is a 3rd party module to lazily load functions that are time consuming: https://github.com/xcv58/zsh-lazy-load

To install this module if you want, do the following:

cd $HOME/.zprezto
git submodule add https://github.com/xcv58/zsh-lazy-load.git modules/lazy-load

Once that is done 'lazy-load' should be the first item in your modules list as shown above.

In my current .zshrc file I’m using it to lazily load nvm and virtualenvwrapper:

# .zshrc

func load_virtualenv() {
  source path_to_virtualenvwrapper.sh
}

func load_nvm () {
    source path_to_nvm.sh
}


# At the end of .zshrc
lazy_load load_virtualenv load_nvm

Selecting theme

Prezto comes with a bunch of themes. To list the themes use prompt -l and to preview one use prompt -p name. Once you’ve picked a theme open your .zpreztorc file and add it zstyle ':prezto:module:prompt' theme 'pure' (here 'pure' is the prompt theme’s name).

Fonts

Currently, I’m using GitHub’s Monaspace font, its Krypton variation. This font supports ligatures.

Other fonts I highly recommend:

• Nerd font — great for terminal prompt
• Fira Code (supports ligatures)
• Monaco (ships with MacOS and great for code but lacks ligatures). A ligature version is also available but I’ve not tried it personally. The Powerline mentioned here is a prompt theme.

Font setup in iTerm 2:

Font setup in VS Code:

Prompt Customization

I’m using Starship for customizing the prompt. Note that by default it requires Nerd Font.

To install it Homebrew can be used:

brew install starship

Then open your .zshrc file and add the following at the end:

eval "$(starship init zsh)"

To configure Starship you need to create a config file first:

mkdir -p ~/.config && touch ~/.config/starship.toml

Then define the items you’d want:

format = """
$custom\
$username\
$hostname\
$shlvl\
$directory\
$git_branch\
$git_commit\
$git_state\
$git_status\
$package\
$elixir\
$golang\
$nodejs\
$python\
$ruby\
$rust\
$terraform\
$nix_shell\
$aws\
$env_var\
$line_break\
$status\n\
$cmd_duration\
$character"""
add_newline = true

Then for each item you can further make your own modification, for example here’s how username, directory and git status looks like for me:

[username]
disabled = false
show_always = true
style_user = "white bold"
style_root = "white bold"
format = "[xonix_$user]($style) "

[directory]
truncation_length = 100  # no truncation
truncate_to_repo = true
format = "[$path]($style) "

## Git settings
[git_branch]
style = "bold purple"
truncation_length = 100  # no truncation
truncation_symbol = "..."

[custom.xonix]
command = "echo -n '🍺 '"
when = "true"

The resulting prompt looks like:

The setting available under any such action is in the docs: username, directory, git_branch and custom

Find detailed documentation here: https://starship.rs/config/#prompt

Checking performance

If your shell is taking a while to load then it can be profiled by adding zmodload zsh/zprof at the start of .zshrc file and zprof at the end of the file. In addition individual modules can be timed using time (pmodload '<module_name>').

This article was originally published on Medium

Table of Contents

1. What’s pdb?
2. Why pdb?
3. How to start pdb?
   • Starting program under debugger control
   • Running code under debugger control
   • Set a hardcoded breakpoint
   • Post-mortem debugging
4. Basic pdb commands
   • Stepping through code
   • Jumping between stacks
   • Breakpoints
   • Tips and tricks
   • What’s new in Python 3

What’s pdb?

pdb is a module from Python’s standard library that allows us to do things like:

• Stepping through source code
• Setting conditional breakpoints
• Inspecting stack trace
• Viewing source code
• Running Python code in a context
• Post-mortem debugging

Why pdb?

It is not necessary to use pdb all the time, sometimes we can get away with simple print statements or logging.

But these other approaches are most of the time not good enough and don’t give us enough control while debugging. Plus after debugging we also have to take care of removing the `print“ statements that we had added to our program just for debugging purpose, this isn’t true for logging though as we can filter out logs easily. But at the end both of these approaches clutter our code and don’t give us enough debugging power either.

How to start pdb?

There are multiple ways to start pdb depending on your use case.

1. Starting program under debugger control

We can start a script itself under debugger’s control by executing the script using -m pdb argument. Let’s run script.py:

$ python -m pdb script.py
 > /pdb-mupy/script.py(1)<module>()
 -> """I am the first script in this demo"""
 (Pdb)

In this mode Python will stop the program on first line and you are going to be inside debugger((Pdb) is pdb’s prompt). At this point you can either set breakpoints or continue executing your program.

Another special thing about it is that after the program completion if no exception occurred then your program will restart in same mode otherwise it will start in post-mortem mode. After post-mortem mode you can restart the program again.

2. Running code under debugger control

Instead of running the whole code under debugger control we can run particular code under using pdb.run, pdb.runeval and pdb.runcall.

>>> import pdb
>>> import script
>>> pdb.run('script.divide(10, 5)')
> <string>(1)<module>()->None
(Pdb) s   # we can run any pdb command here
--Call--
> /pdb-mupy/script.py(6)divide()
-> def divide(numerator, denominator):
(Pdb) n
> /pdb-mupy/script.py(7)divide()
-> return numerator / denominator
(Pdb) p numerator, denominator
(10, 5)
(Pdb) c
>>>

Here <string>(1)<module>() means that we are at the start of string passed to run() and no code has executed yet. In the above example we stepped into the divide function using s(don’t worry about s, n, c etc, we will be covering them in detail).

runeval() does the same thing as run() except that it also returns the value of executed code.

runcall() allows us to pass a Python callable itself instead of a string.

>>> pdb.runcall(script.divide, 10, 5)
> /pdb-mupy/script.py(7)divide()
-> return numerator / denominator
(Pdb)

3. Set a hardcoded breakpoint

This is the most common way to debug programs, it basically involves adding the line pdb.set_trace() in the source code wherever we want our program to stop.

4. Post-mortem debugging

Post-mortem debugging allows us to debug a dead program using its traceback object. In post-mortem debugging we can inspect the state of the program at the time it died. But apart from inspecting the state we can’t do much here(like stepping through the code) because like the name suggests we are performing post-mortem of a dead program.

By default the -m pdb we had discussed earlier puts us in post-mortem mode if an exception occurs. Other ways are using: pdb.pm() and pdm.post_mortem().

pdb.pm() will take us to the post-mortem mode for the exception found in sys.last_traceback.

On the other hand pdb.post_mortem() excepts an optional traceback object otherwise will try to handle the exception currently being handled.

>>> import pdb
>>> import script
>>> script.divide(10, 0)
Traceback (most recent call last):
  File "<ipython-input-8-fe270324adad>", line 1, in <module>
    script.divide(10, 0)
  File "script.py", line 7, in divide
    return numerator / denominator
ZeroDivisionError: integer division or modulo by zero

Now to inspect the state at the time this above exception occurred using pdb.pm().

>>> pdb.pm()
> /Users/ashwini/work/instamojo/pdb-mupy/script.py(7)divide()
-> return numerator / denominator
(Pdb) args  # Arguments passed to the function at that time
numerator = 10
denominator = 0

We could have done something similar using pdb.post_mortem() with the traceback object:

>>> import sys
>>> pdb.post_mortem(sys.last_traceback)
> /Users/ashwini/work/instamojo/pdb-mupy/script.py(7)divide()
-> return numerator / denominator
(Pdb)

Similarly we can handle the current exception being handled using pdb.post_mortem() without any argument:

>>> try:
...     script.divide(10, 0)
... except Exception:
...     pdb.post_mortem()
...
> /Users/ashwini/work/instamojo/pdb-mupy/script.py(7)divide()
-> return numerator / denominator
(Pdb)

Basic pdb commands

(Pdb) prompt we have seen so far is pdb’s own shell and it has its own set of commands that makes debugging even easier. In this section we will go through some of the basic commands.

Before starting with the commands it is important to understand the notation we use for commands, for example a command like c(ont(inue)) means we can either use c, cont or continue for this command. The square brackets([]) followed by a command are its optional arguments, without square brackets it is a compulsory argument.

• h(elp) [command]

• help or simply h provides help related to a pdb command. Without arguments it lists all of the pdb commands available.

• (Pdb) help

Documented commands (type help <topic>):
========================================
EOF    c          d        h         list      q        rv       undisplay
a      cl         debug    help      ll        quit     s        unt
alias  clear      disable  ignore    longlist  r        source   until
args   commands   display  interact  n         restart  step     up
b      condition  down     j         next      return   tbreak   w
break  cont       enable   jump      p         retval   u        whatis
bt     continue   exit     l         pp        run      unalias  where

Get help related to args command:

• (Pdb) help args

(Pdb) help args
a(rgs)
        Print the argument list of the current function.

This command can save your time related to visiting Python documentation in case you forgot about a command.

Note: ! command is the only exception here as help only works with valid Python identifiers. Alternative is to use help exec.

• p or pp

To print variable inside debugger we can use p for normal printing and pp for pretty-printing. We can use simple Python print as well but it is not a pdb command.

• a(rgs)

args prints the arguments with their values of the current function.

>>> pdb.runcall(script.divide, 10 , 15)
> /pdb-mupy/script.py(7)divide()
-> return numerator / denominator
(Pdb) args
numerator = 10
denominator = 15
q(uit)

To exit the debugger we can use q or quit.

• ! statement

To run Python code in debugger we can use ! followed by the code we want to run. Without ! the code can fail if it collides with any pdb command, hence it is recommended to always use ! to run Python code.

$ python -m pdb script.py
> /pdb-mupy/script.py(1)<module>()
-> """I am the first script in this demo"""
(Pdb) !c = 2  # Define a variable named c
(Pdb) p c
2

Without ! it fails because pdb thinks we are trying to run pdb’s c command.

(Pdb) c = 2
2
The program finished and will be restarted:
> /pdb-mupy/script.py(1)<module>()
-> """I am the first script in this demo"""
(Pdb)

• run [args ...]

run allows us to restart a program. This is helpful if we want to restart the programs with different argument without exiting the debugger.

$ python -m pdb script.py 10 5
> /pdb-mupy/script.py(1)<module>()
-> """I am the first script in this demo"""
(Pdb) !import sys
(Pdb) p sys.argv
['script.py', '10', '5']
Let's restart this program with different arguments:
(Pdb) run 30 40
Restarting script.py with arguments:
    30 40
> /pdb-mupy/script.py(1)<module>()
-> """I am the first script in this demo"""
(Pdb) !import sys
(Pdb) p sys.argv
['script.py', '30', '40']

• l(ist) [first[, last]]

l or list command can be used to list the source code.

Without any argument it lists the 11 lines around the current line. With one argument 11 lines around the specified line number. With two argument it lists the lines in that range, if second argument is less that first then it is taken as count.

$ python -m pdb script.py
> /pdb-mupy/script.py(1)<module>()
-> """I am the first script in this demo"""
List 11 lines around the current line:
(Pdb) list
  1  -> """I am the first script in this demo"""
  2
  3     import sys
  4
  5
  6     def divide(numerator, denominator):
  7         return numerator / denominator
  8
  9
 10     if __name__ == '__main__':
 11         numerator = int(sys.argv[1])
List lines 5 to 8:
(Pdb) list 5, 8
  5
  6     def divide(numerator, denominator):
  7         return numerator / denominator
  8

• alias [name [command]] or unalias

alias command can be used to set aliases for commands in debugger, similarly unalias can be used to unset an already existing alias.

Let’s say we want to create an alias that returns a list of squares.

(Pdb) alias squares [i**2 for i in xrange(%1)]
(Pdb) squares 5
[0, 1, 4, 9, 16]

Here %1 is the argument that our alias expects(5 in the above example), if it expects more then we can use %2, %3 etc

We can also create aliases using existing aliases:

(Pdb) alias squares_7 squares 7

Now squares_7 is equivalent to running squares 7

(Pdb) squares_7
[0, 1, 4, 9, 16, 25, 36]

To remove an alias use unalias command followed by the command name.

Stepping through code

One of the strongest feature of pdb is that we can move through our code in various ways: 

• Line by line 
• Jumping inside a function 
• Skip a loop 
• Skip function

In this section we will learn about the commands that allow us to step through the code. The code that we will be use in this section is next_and_step_until.py. These are the commands that you will be using the most, hence it is important to have a clear understanding here.

• n(ext)

n or simply next command runs the code on current line at full-speed and takes us to the next line in the current function.

• s(tep)

s or step is similar to next but they vary when a callable(function etc) is involved. If a callable is there then it will step us inside that callable instead of taking us to next line in the current function. If no callable is involved then it is same as next.

• unt(il)

until command tells the debugger to continue executing until we have reached a line number greater than the current line number. This command is helpful in exiting a loop.

• r(eturn)

r or return takes us to the end of the current function. At global level it takes us to the last line in the module. This command is helpful you want to step through the whole function body at once.

• c(ont(inue))

c or cont or continue command lets us run the whole code at full-speed when we are done with our debugging. If there’s another breakpoint in your program then it will stop at that next breakpoint.

Let’s debug through our script next_and_step_until.py while making use of the above commands. We have set a breakpoint on line #19 in that script and debugger will stop on next valid line: #21.

$ python next_and_step_until.py
> /Users/ashwini/work/instamojo/pdb-mupy/next_and_step_until.py(21)<module>()
-> knights()  # Just want to run this function and move on to next line? Use n(ext).
(Pdb)

Now to run the knights() function at full-speed use n(ext). As you can see it printed the statements we have inside the knights() function and stopped on the next line in the current function.

(Pdb) n
We are the Knights who say ni!
Get us a shrubbery.
> /Users/ashwini/work/instamojo/pdb-mupy/next_and_step_until.py(22)<module>()
-> credits()  # Want to step inside this function? Use s(tep).
(Pdb)

Now let’s say we want to debug something inside credits() call, for that we can use s(tep).

(Pdb) s
--Call--
> /pdb-mupy/next_and_step_until.py(11)credits()
-> def credits():
(Pdb) n
> /pdb-mupy/next_and_step_until.py(12)credits()
-> print "A Møøse once bit my sister... No realli!"
(Pdb) n
A Møøse once bit my sister... No realli!
> /pdb-mupy/next_and_step_until.py(13)credits()
-> print "We apologise for the fault in the print statements."
(Pdb)

Now that we are done with this function we can use r(eturn) to go to its end and then use n to exit:

(Pdb) r
We apologise for the fault in the print statements.
Those responsible have been sacked.
--Return--
> /Users/ashwini/work/instamojo/pdb-mupy/next_and_step_until.py(15)credits()->'M\xc3\xb8\xc...pretti nasti.'
-> return "Møøse bites Kan be pretti nasti."
(Pdb) n
> /Users/ashwini/work/instamojo/pdb-mupy/next_and_step_until.py(24)<module>()
-> for i in range(1, 5):

Now we are in a loop and both n(ext) and s(tep) can’t be used to complete it in a single step. To skip the loop we can go its last line and use unt(il).

-> for i in range(1, 5):
(Pdb) n
> /pdb-mupy/next_and_step_until.py(26)<module>()
-> print "Shrubbery #{}".format(i)
(Pdb) until
Shrubbery #1
Shrubbery #2
Shrubbery #3
Shrubbery #4
> /pdb-mupy/next_and_step_until.py(28)<module>()
-> print "We have found the Holy Grail."
(Pdb)

Now we can continue executing the program using c(ont(inue)).

(Pdb) cont
We have found the Holy Grail.

Jumping between stacks

So far we have only seen how to move forward in code by moving line by line and jumping inside a function call. But pdb also provides us functionality to jump up and down in the current stack.

Best way to demonstrate this is to use a recursive function as an example, its code can be found at recursive.py.

The three commands we will be going through in this section are u(p), d(own) and w(here).

• w(here)

w or where prints the whole trace till the most recent frame and current frame is represented using an arrow.

Let’s run our program and when it stops at the breakpoint we will use w(here) to view the whole stack trace

$ python recursive.py
> /pdb-mupy/recursive.py(10)func()
-> return 0
(Pdb) where
  /pdb-mupy/recursive.py(14)<module>()
-> print (func(4))
  /pdb-mupy/recursive.py(7)func()
-> return func(n - 1)
  /pdb-mupy/recursive.py(7)func()
-> return func(n - 1)
  /pdb-mupy/recursive.py(7)func()
-> return func(n - 1)
  /pdb-mupy/recursive.py(7)func()
-> return func(n - 1)
> /pdb-mupy/recursive.py(10)func()  # Current frame
-> return 0
(Pdb)

Here > represents the current frame.

Now we can go up and down in the stack using u(p) and d(own). Let’s move up twice and then check the argument value using a(rgs). The breakpoint was set at n = 0, now when we moved up twice we have n = 2.

(Pdb) u
> /pdb-mupy/recursive.py(7)func()
-> return func(n - 1)
(Pdb) u
> /pdb-mupy/recursive.py(7)func()
-> return func(n - 1)
(Pdb) args
n = 2
(Pdb)

Similarly we can go back down using d(own):

(Pdb) d
> /pdb-mupy/recursive.py(7)func()
-> return func(n - 1)
(Pdb) args
n = 1

u(p) is also useful when you stepped(s(tep))inside a function accidentally and want to go back.

Breakpoints

So far we have only seen how to set a break-point by updating the code and adding pdb.set_trace() wherever we want to stop our program. But, pdb also provides us a way to set dynamic conditional breakpoints without updating the source code. In this section we will be using code from breakpoints.py file.

• b(reak) [[filename:]lineno | function[, condition]]:

We can set a breakpoint in the current file by specifying the line number or function name in the current file. Or we can also set breakpoint in some other file(this file should be present in module search path) by specifying the file name followed by a line number. Each number is assigned a number and this number can be used later with other commands to access the breakpoint.

condition is a Python statement that should be True to stop at the breakpoint. This condition is executed in the scope at which we have set the breakpoint.

Few examples of setting breakpoints.

• break 13 # Set breakpoint on line number 13
• break divide # Set breakpoint on divide function
• break divide, denominator == 0 # Set breakpoint in divide function only if denominator is 0

$ python -m pdb breakpoints.py 10 0
> /pdb-mupy/breakpoints.py(2)<module>()
-> import sys
(Pdb) break divide, denominator == 0
Breakpoint 1 at /pdb-mupy/breakpoints.py:5

Get list of breakpoints using b(break). Here Num = 1 is the number assigned to the breakpoint. Disp == keep means it is a permanent breakpoint and End = yes means this breakpoint is right now enabled.

(Pdb) break
    Num Type         Disp Enb   Where
    1   breakpoint   keep yes   at /pdb-mupy/breakpoints.py:5
        stop only if denominator == 0
    (Pdb) c

As denominator is program our program stopped under debugger control:

> /pdb-mupy/breakpoints.py(8)divide()
    -> print "Calculating {}/{}".format(numerator, denominator)
    (Pdb) args
    numerator = 10.0
    denominator = 0.0

Let’s restart the program using different arguments:

(Pdb) run 10 5
Restarting breakpoints.py with arguments:
    10 5
> /pdb-mupy/breakpoints.py(2)<module>()
-> import sys
(Pdb) c
Calculating 10.0/5.0
2.0
The program finished and will be restarted
> /pdb-mupy/breakpoints.py(2)<module>()
-> import sys

As the denominator wasn’t zero this time the program didn’t stop at the breakpoint.

(Pdb) break
Num Type         Disp Enb   Where
1   breakpoint   keep yes   at /pdb-mupy/breakpoints.py:5
    stop only if denominator == 0
    breakpoint already hit 2 times

Note: Breakpoints persist even after the auto restart or forced restart(using run) in -m pdb mode.

• tbreak [[filename:]lineno | function[, condition]]:

tbreak allows us to set a temporary breakpoint. This breakpoint goes away as soon as it is hit once. Can be pretty useful if you want to set a breakpoint only once, say inside a loop or for the first time a function is invoked.

$ python -m pdb breakpoints.py 10 5
> /pdb-mupy/breakpoints.py(2)<module>()
-> import sys
(Pdb) tbreak divide, denominator == 0
Breakpoint 1 at/pdb-mupy/breakpoints.py:5
(Pdb) break
Num Type         Disp Enb   Where
1   breakpoint   del  yes   at /pdb-mupy/breakpoints.py:5
    stop only if denominator == 0

Notice the value of Disp this time. It is del instead of keep, means it is a temporary breakpoint.

(Pdb) c
Calculating 10.0/5.0
2.0
The program finished and will be restarted
> /Users/ashwini/work/instamojo/pdb-mupy/breakpoints.py(2)<module>()
-> import sys

Programs simply restarted because the breakpoint condition wasn’t true, now let’s make it true by restarting it with different arguments.

(Pdb) run 10 0
Restarting breakpoints.py with arguments:
    10 0
> /pdb-mupy/breakpoints.py(2)<module>()
-> import sys
(Pdb) c
Deleted breakpoint 1
> /Users/ashwini/work/instamojo/pdb-mupy/breakpoints.py(8)divide()
-> print "Calculating {}/{}".format(numerator, denominator)
(Pdb) break
(Pdb)

This time it did hit the breakpoint and was deleted as well.

• cl(ear) [filename:lineno | bpnumber [bpnumber ...]]:

We can permanently remove breakpoints using cl(ear) command.

• disable [bpnumber [bpnumber ...]] or enable [bpnumber [bpnumber ...]]:

To temporary disable a breakpoint use disable and to re-enable a breakpoint use enable. Unlike clear breakpoints are not removed permanently in this case.

• ignore bpnumber [count]:

We can also ignore a breakpoint count number of times using ignore command. Breakpoint is re-activated when the count becomes 0.

• condition bpnumber [condition]:

To update or add condition to a breakpoint we can use the condition command.

Note: If the condition is an invalid Python code then it will be evaluated as True but in case the breakpoint was a temporary one then it won’t be deleted and if the breakpoint had an ignore count then it won’t be decremented. This is done to notify the user that something’s wrong.

• commands [bpnumber]:

commands is a pretty useful command related to breakpoints. If used in a certain way then it can be equivalent to adding print statements in our code.

This command allows us to run multiple commands when a breakpoint is hit.

Let’s add a breakpoint on divide function and now we will print some stuff as well using commands.

$ python -m pdb breakpoints.py 10 5
> /pdb-mupy/breakpoints.py(2)<module>()
-> import sys
(Pdb) break divide
Breakpoint 1 at /Users/ashwini/work/instamojo/pdb-mupy/breakpoints.py:5

In commands mode the prompt is (com). To end the commands use end.

(Pdb) commands 1
(com) args
(com) p "Inside divide()"
(com) end
(Pdb) c
numerator = 10.0
denominator = 5.0
'Inside divide()'
> /pdb-mupy/breakpoints.py(8)divide()
-> print "Calculating {}/{}".format(numerator, denominator)

Now as you can see our program printed few things on hitting the breakpoint.

We can also use commands like cont, next etc. But these commands will also act as end because these commands can lead us to next breakpoint which may have its own set of commands and then debugger will be confused about whose commands to run next.

Another commands is silent, when this command is part of commands list then you won’t see the message we get at a breakpoint.

$ python -m pdb breakpoints.py 10 5
> /pdb-mupy/breakpoints.py(2)<module>()
-> import sys
(Pdb) break divide
Breakpoint 1 at /pdb-mupy/breakpoints.py:5
(Pdb) commands 1
(com) args
(com) p "Inside divide()"
(com) silent
(com) cont
(Pdb) c
numerator = 10.0
denominator = 5.0
'Inside divide()'
Calculating 10.0/5.0
2.0
The program finished and will be restarted
> /pdb-mupy/breakpoints.py(2)<module>()
-> import sys
(Pdb)

As you can see above the program didn’t stop at the breakpoint this time due to cont command and we didn’t see the lines(shown below) related to breakpoint either due to silent command:

> /pdb-mupy/breakpoints.py(8)divide()
-> print "Calculating {}/{}".format(numerator, denominator)

Tips and tricks

• .pdbrc file: If present the commands present in this file are ran at the start of debugger session. This file can be added to your home directory and/or current directory.

• Plain enter repeats the last command(list command is an exception).

• To enter multiple commands on a single line use ;; as separator.

What’s new in Python 3

Python 3.3+

• Tab-completion via the readline module is available for commands and command arguments.

Python 3.2+

• pdb.py now accepts a -c option that executes commands as if given in a “.pdbrc file`

• A new command ll can be used to see the source related to the current function or frame.

• A new command source can be used to see the source code of an expression: source expression.

• A new command interact can be used to start interactive shell in debugger using the globals() and locals() in the current frame. This can be done in Python 2 using !import code; code.interact(local=vars()).

class Default(dict):
    def __missing__(self, key):
        return key

print ('{name} was born in {country}'.format_map(Default(name='Guido'))) 
# Guido was born in country

But this fails:

>>> print ('{name} was born in {country.state}'.format_map(Default(name='Guido')))
Traceback (most recent call last):
  File "<ipython-input-324-1012aa68ba8d>", line 1, in <module>
    print ('{name} was born in {country.state}'.format_map(Default(name='Guido')))
AttributeError: 'str' object has no attribute 'state'

That is obvious because we are returning a string from __missing__ and that string doesn’t have any attribute of the name state.

Note that the above way is also possible in Python 2 and Python 3.0-3.1 using the Formatter class’s vformat method.

from string import Formatter
f = Formatter()
print f.vformat('{name} was born in {country}', (), Default(name='Guido'))
# Guido was born in country

Dealing with dot notations, conversions(!s or !r) and format specs(^, > etc)

The solution is, for missing keys instead of returning simple string, return an instance of a class that can handle these attribute calls along with creating full string back:

class MissingAttrHandler(object):
    def __init__(self, format):
        self.format = format

    def __getattr__(self, attr):
        return type(self)('{}.{}'.format(self.format, attr))

    def __repr__(self):
        return self.format + '}'


class Default(dict):
    def __missing__(self, key):
        return MissingAttrHandler('{{{}'.format(key))

Now let’s test this:

>>> print('{name} was born in {country.state} and his last '
          'name is {Person.full_name.last_name}'.format_map(Default(name='Guido')))
Guido was born in {country.state} and his last name is {Person.full_name.last_name}

Some of you may have already noticed, this solution has one issue though, it will fail if other formatting details like ^, 10d etc are present:

>>> print('{name} was born in {country.state} and his last '
      'name is {Person.full_name.last_name:*^30}'.format_map(Default(name='Guido')))
Traceback (most recent call last):
  File "<ipython-input-94-b375bfa3e06c>", line 2, in <module>
    'name is {Person.full_name.last_name:*^30}'.format_map(Default(name='Guido')))
TypeError: non-empty format string passed to object.__format__

This is because MissingAttrHandler has no __format__`` method of its own, hence the format lookup goes to its base class object(object.format`)

>>> MissingAttrHandler.__format__ is object.__format__
True
>>> object.__format__(MissingAttrHandler(''), '^*30s')
Traceback (most recent call last):
  File "<ipython-input-129-c4e00a46bd28>", line 1, in <module>
    object.__format__(MissingAttrHandler(''), '^*30s')
TypeError: non-empty format string passed to object.__format__

So, let’s define a `format“ method in our class that takes care of this:

def __format__(self, format):
        return '{}:{}}}'.format(self.format, format)

Let’s test it:

>>> print('{name} was born in {country.state} and dict has '
      '{dict.get:*^30} method.'.format_map(Default(name='Guido')))
Guido was born in {country.state:} and dict has {dict.get:*^30} method.
>>> print('{name} was born in {country.state} and dict has '
      '{dict.get:>30d} method.'.format_map(Default(name='Guido')))
Guido was born in {country.state:} and dict has {dict.get:>30d} method.

Seems to be working fine, let’s try one more thing:

>>> print('{name} was born in {country.state} and dict has '
...       '{dict.get!s:*^30} method.'.format_map(Default(name='Guido')))
Guido was born in {country.state:} and dict has **********{dict.get}********** method.

Well this was quite unexpected, what exactly happened there?

Well due to the !s present in the format string after getting the value of these fields using either str() or repr()(which is a string object), Python will now call __format__ on it with *^30 as an argument. But as we returned a string object and not a MissingAttrHandler object the format call goes to that str.

>>> '{dict.get}'.__format__('*^30')
'**********{dict.get}**********'

We can try to return an instance of MissingAttrHandler rather than a string from its __repr__ method. But to return MissingAttrHandle instance from __str__ or __repr__ we will have to inherit from str as well because Python expects us to return an instance of type str. Now __repr__ will look like:

def __repr__(self):
        return MissingAttrHandler(self.format + '!r}')

Note that now we need to define __str__ as well because our class does not inherit from str which provides a __str__ method, hence calling __str__ on it won’t fallback to __repr__ anymore.

And one cool thing about __format__ is that once defined, it is the function that is by default called during string formatting unless we provide !r or !s explicitly. If !r or !s are present on the format string then __repr__ and __str__ are called respectively and then __format__ is called on the resulting object.

Ah! ha that’s exactly what we needed right? Using this we can also add !r or !s in our format strings and later complete it with the __format__ method.

So, in the end our class will look like:

class MissingAttrHandler(str):
    def __init__(self, format):
        self.format = format

    def __getattr__(self, attr):
        return type(self)('{}.{}'.format(self.format, attr))

    def __repr__(self):
        return MissingAttrHandler(self.format + '!r}')

    def __str__(self):
        return MissingAttrHandler(self.format + '!s}')

    def __format__(self, format):
        if self.format.endswith('}'):
            self.format = self.format[:-1]
        return '{}:{}}}'.format(self.format, format)


class Default(dict):
    def __missing__(self, key):
        return MissingAttrHandler('{{{}'.format(key))

Let’s try it:

>>> print('{name} was born in {country.state} and dict has '
      '{dict.get!r:*^30} method.'.format_map(Default(name='Guido', dct=dict)))
Guido was born in {country.state:} and dict has {dict.get!r:*^30} method.
>>> print('{name} was born in {country.state!r:=20s} and dict has '
      '{dict.get!s:*^30} method.'.format_map(Default(name='Guido', dct=dict)))
Guido was born in {country.state!r:=20s} and dict has {dict.get!s:*^30} method.

Works! ;-)

I hope you must have learned something about string formatting in Python with the aforementioned method.

But is there any other way to do this?

Yes!

Second way:

We can achieve the same thing as above using Formatter class from string module, the parse() method of this class can be used to parse the format string. It returns an iterable that yields a tuple containing (literal_text, field_name, format_spec, conversion). We can use these fields to re-create our string.

from functools import reduce
from operator import attrgetter
from string import Formatter


def get_field_value(field_name, mapping):
    try:
        if '.' not in field_name:
            return mapping[field_name], True
        else:
            obj, attrs = field_name.split('.', 1)
            return attrgetter(attrs)(mapping[obj]), True
    except Exception as e:
        return field_name, False



def str_format_map(format_string, mapping):
    f = Formatter()
    parsed = f.parse(format_string)
    output = []
    for literal_text, field_name, format_spec, conversion in parsed:
        conversion = '!' + conversion if conversion is not None else ''
        format_spec = ':' + format_spec if format_spec else ''
        if field_name is not None:
            field_value, found = get_field_value(field_name, mapping)
            if not found:
                text = '{{{}{}{}}}'.format(field_value,
                                           conversion,
                                           format_spec)
            else:
                format_string = '{{{}{}}}'.format(conversion, format_spec)
                text = format_string.format(field_value)
        output.append(literal_text + text)
        text = ''
    return ''.join(output)