Case study · AI product
A job tracker that learns
How a static list of job leads became a self-improving, two-agent daily system — the problem, the architecture, the product decisions, and what broke along the way.
Type
Personal project
Cadence
Daily, unattended
Agents
2 (worker + critic)
Pattern
Stateless + external state
Executive summary
Knowing how little to build
A scheduled AI system curates about 15 fitting job openings every morning, never re-suggests a role already handled, learns taste from what gets applied to versus skipped, and has a second, independent agent review its work each day. The engineering was simple. The value was in the architecture: a stateless worker agent plus an independent critic agent operating over a shared external memory — chosen deliberately over heavier options like fine-tuning a model or running a fleet of parallel agents.
The problem
A daily search with a memory problem
Every morning means re-running the same queries across a dozen sites, re-reading roles already seen, and mentally filtering for the two things that actually gate the search: experience level and, on a STEM-OPT visa, whether the employer can even hire the candidate — E-Verify enrolment matters more than H-1B sponsorship for the first few years.
The real requirement was never "a list of jobs". It was a system that shows up daily, remembers what it already showed, learns what the user actually pursues, and checks its own work — with the user doing nothing but tagging what they apply to.
The daily choreography
Two runs, twenty minutes apart
11:00
Worker runs (~8 min)
Reads reflections, critique and applied history, searches, excludes handled roles, ranks to taste, publishes ~15 roles, then overwrites its reflection note — which clears the old critique.
11:20
Critic runs
A separate, independent run reviews the fresh board and writes its critique back into the shared store.
Next day
Worker reads both
The reflection and the critic's critique are read first thing, whatever was flagged gets fixed, and the cycle repeats.
How it works
Five ideas
Stateless compute
Each run is a brand-new session with no memory of the last — cheap, disposable, and impossible to corrupt with stale in-process state. The model can be swapped or upgraded any time, because no history lives inside the agent.
External state
Continuity is delegated to a cloud database beside the agents: roles applied to or skipped, the worker's reflections, the critic's reviews. Stateless workers behave as if they remember because the memory is in the data — and it's inspectable and fixable by hand.
Preference learning
Every run contrasts what the user applies to against what they skip, and biases the day's ranking accordingly — a recommender-style loop with no model training, just the user's own signals read back at runtime.
Reflection loop
At the end of each run the worker records what it learned and what to adjust; the next run reads that first. The agent doesn't self-modify — the system around it improves because its inputs get richer each day.
Evaluator / critic pattern
An agent grading its own work tends to rationalise its choices. So a separate critic — fresh context, not invested in the output — audits each board and feeds its critique back in. Worker proposes, critic disposes, in sequence rather than in parallel.
The data model
Two collections
All shared state lives in the dashboard's cloud database — not in the page's code, not in the browser.
applications
One document per role the user has tagged. Fields: company, role, location, salary, url, status (Applied / Skip / Interested), updatedAt. This is what powers exclusion and preference learning.
reflections
A 'latest' document plus dated history. Fields: prefs (the worker's inferred read of taste), notes (adjustments for next run), critique (the critic's findings), plus timestamps. The dashboard surfaces all three in a 'What it has learned' panel.
Key product decisions
Choosing the lightest tool at each fork
Fine-tune a model on my preferences?
No.Retrieval plus a feedback loop delivers the same behavioural outcome at a fraction of the cost — no labelled dataset, no training pipeline, and it never goes stale. Fine-tuning earns its place for format-at-scale or latency, neither of which applied here.
How many agents, and in what shape?
Two, in sequence.A 'lead plus parallel searchers' design would widen coverage but adds cost and coordination-failure risk for little gain at this scale. The pattern that did earn its place was evaluator/critic: worker then critic, sequentially.
Store state in the page, or a real database?
Database.Browser storage is per-device and invisible to the agents. Only a shared, server-side store lets the user's clicks and both agents read the same source of truth — the entire reason the system stops re-suggesting handled jobs and can carry a critique from one agent to another.
What broke
And what I learned
Silent write drops
Rapid tagging tripped a rate limit and some saves failed quietly. Fix: a queue that spaces and retries writes, plus a visible 'synced' state. Observability matters more for agents than for apps, because they fail where no one is watching.
Identity drift
Keying records on company + role text created duplicates when names shifted between runs ('Tint' versus 'Tint AI'). A stable identity key matters as much here as in any data system.
A silently open loop
After adding the critic, a verification pass caught that the worker's instructions never told it to read the critique — the critic was reviewing into a channel no one consumed. It looked done and was not. 'It ran' is not 'it worked.'
Freshness versus volume
A strict 7-day window plus a growing exclusion list can leave fewer than 15 fresh roles on a slow day. The honest fix is to fill with the freshest available and say so, not to pad with stale or off-target listings.
What's next
Where it goes from here
- Outcome tracking: record what happens after applying — call, reject, silence — and weight the sources and role types that actually convert.
- Parallel searchers: promote to a lead-plus-searchers design if source coverage becomes the bottleneck.
- Self-editing reflections: let the loop propose its own instruction tweaks for human review — a step toward genuine self-improvement, with a person in the loop.
The one-line version
The intelligence is a fixed, shared model; everything that makes this feel smart, personal and self-correcting lives in the design around it — stateless runs, external memory, a preference loop, and an independent critic. The scarce skill was not building the agent. It was choosing how little to build.
Read the full case studyView the interactive artefact