Building a Code Evolution Analyzer in a Weekend

Thursday afternoon I thought I’d write a quick script. It took until Friday evening.

---

I wanted to watch a repo grow. Not the snapshot—how many lines now—but the journey: when did TypeScript overtake JavaScript, when did the test suite explode, which commit introduced the generated code that still haunts the language breakdown. The plan seemed simple enough: run a code counter on every commit, collect the numbers, draw a chart. (I’ve thought this about a lot of projects. It’s rarely true.)

The loop itself was honest enough:

// analyze.mjs (v0.1, the naive version)
import { execSync } from 'child_process';

const commits = execSync('git log --format="%H" --reverse')
  .toString().trim().split('\n');

for (const commit of commits) {
  execSync(`git checkout ${commit}`);
  const result = execSync('cloc . --json');
  // ... collect data
}

For a small repo, this runs fine. For a repo with ~10,000 files and ~2,000 commits, it runs for 20 minutes, and somewhere in minute twelve I worked out that I had written about 20 million file reads into a for loop. That number killed the design. If collection takes that long, nothing else matters: the rendering won’t ship, the web service won’t scale, and I’ll never actually use the tool.

Finding scc

I’m a bit embarrassed about how long I stared at that loop before thinking to search “fast cloc alternative.” The answer was scc (Sloc Cloc and Code, written in Go). I swapped one line, ran it again, and the same repo finished in about 15 seconds — roughly 80× faster for a one-word change.

The difference is mechanical: cloc is Perl and walks the tree serially, while scc is Go and parallelizes across cores. For historical analysis, where you run the same counter thousands of times, that gap is what separates a tool you use from a tool you start and walk away from.

With collection survivable, everything else could exist. Lines per language, files per language, bytes per language, with commit index as the clock. The visualization, the queuing, the WebSockets all hang off the loop being fast enough to sample every commit instead of every Nth and pretending.

The Chart That Wouldn’t Draw

I pushed the visualization into a single HTML file: one artifact, no build step, portable output. At 30fps with ~2,000 commits and ~15 languages, that meant asking the browser to redraw tens of thousands of segments every frame, and Chart.js really doesn’t like that.

Chart.js is excellent for static charts. This is a streaming animation where the dataset expands as the playhead advances, and Chart.js redraws the full chart on every update, so even with animations disabled and points hidden the frame drops were predictable. Late Thursday evening I noticed the graph lines weren’t showing up at all. The commit message says “fix: optimize chart updates to O(n) with incremental data points,” which really means “the chart was completely broken and I finally figured out why.”

I tracked the last rendered index and only appended new points instead of rebuilding everything. If someone scrubs backward, the whole chart resets, which isn’t ideal, but I’m honestly not sure how else to handle it without getting much more complicated.

Later I wrote a custom Canvas renderer that decimates data as it draws, bounding the number of segments regardless of commit count. Whether that was the right approach, I don’t know; I wrote it up separately, and I’m still not happy with how it handles zooming.

The Audio Rabbit Hole

I wanted to hear the repo evolve: each language becomes a voice, its line count becomes gain, the chord shifts as commits advance. The first version used triangle waves and sounded terrible. Harsh, buzzy, nothing like what I had in mind.

What followed was about eight hours of audio debugging spread across Thursday night and Friday morning, and the git log is an unflattering record of it. Continuous oscillators, reverb, stopping on completion. Sine waves with detuning, and a volume initialization fix. A major scale starting at C2, per-commit proportions, and a volume variation that started at a flat 20% and ended up spanning 20–100%.

Then I went to bed, woke up, and discovered the voices weren’t staying assigned to the same languages. Each frame re-sorted and reassigned frequencies, so JavaScript might be the bass note one moment and a higher pitch the next. What I wanted was the THX thing, a slow-building chord where each tone swells independently; what I had was noise that changed shape every frame. Stable voice assignment per language fixed the reassignment, though I’m not sure I ever got the swell right.

The worst bug came later that Friday. The pulsing, bursting sound that had been bothering me all along turned out to be updateAudio() silencing all voices first and then setting the active ones. When frames update faster than the 50 ms ramp time, each voice briefly dips toward zero before recovering. The fix was to build a Set of active voice indices first and only silence the inactive ones. Hours to find, one line to change.

Service Shape (If You Want to Share It)

At some point I wanted to share this—“paste a Git URL, get a visualization”—which meant the collection loop couldn’t run on a synchronous HTTP request. It’s CPU-bound and takes minutes for large repos, so I needed durable jobs, progress reporting, and enough isolation that one large repo doesn’t melt everything else.

The shape I landed on has an Express API talking to workers through NATS JetStream, with PostgreSQL for job tracking and Dragonfly (Redis-compatible) for caching and rate limits. It looks like overkill because it probably is, but it maps to the actual problems: jobs peg CPU cores so workers need their own scaling lane, jobs take minutes so the queue needs to survive restarts, and duplicate repos need detecting.

One surprise: I tried Server-Sent Events first for progress updates, and behind Cloudflare the progress arrived in bursts instead of streaming. Some buffering behaviour in the proxy layer, I think. WebSockets fixed it, and the pipeline became simple: worker publishes to NATS, API forwards to the socket, browser updates a progress bar. That isn’t an argument that WebSockets are better; it’s one proxy-layer edge case dominating the whole experience. (I spent longer than I should have trying to make SSE work before giving up.)

Guardrails

Accepting arbitrary Git URLs turns every request into an input validation exercise. If you’ve done any web security work this is obvious, and I still managed to forget it until I tried some test URLs.

Path traversal in the result server came first: resolve to an absolute path, reject anything outside the results root. Command injection via the URL came second, so the security module canonicalizes URLs, enforces HTTPS-only, blocks private IPs, and rejects shell metacharacters before anything reaches git clone. The clone itself runs with GIT_TEMPLATE_DIR empty, core.hooksPath=/dev/null, and the protocol restricted to HTTPS.

None of this was a planned “hardening phase.” It was the system reminding me that string concatenation with user input is a liability even when you know better.

---

Thursday afternoon to Friday evening. It started as a for loop and ended with distributed queues, a custom renderer, and audio synthesis I’m still not happy with. I never sat down to design any of it; I kept hitting walls and building around them.

Whether the architecture makes sense or whether I over-engineered a script, I honestly can’t tell yet. But it works, and I can watch repos grow now, which is what I wanted. (The audio still sounds a bit muddy on repos with many languages. I might revisit that.)

---

See also: Deep Dive: Audio Sonification — the Web Audio experiment
See also: Deep Dive: Canvas 2D Chart Rendering — replacing Chart.js for animation