Skip to content
View NickAme03's full-sized avatar

Block or report NickAme03

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
NickAme03/README.md

Niccolò · @NickAme03

I look for the places where tools for AI coding agents report confidently and measure the wrong thing: tokens counted twice, a model priced at zero, a test suite that has never run on Windows. Then I measure it on real data and take it to the maintainer.

Other people's repositories

Tools that read agent transcripts mostly run their tests on Linux. The paths inside those transcripts are the one thing that changes shape on Windows, so I went looking there. Fork, add a workflow with an ubuntu and windows matrix, run the suite, compare. The ubuntu leg is the control: without it a Windows failure tells you nothing, because you cannot separate a broken platform from a broken suite.

junhoyeo/tokscale is the one worth citing. 947 tests passed and 12 failed on Windows, against 2461 passed and none failed on Ubuntu, with the suite stopping at the first binary and never reaching the end. Three of the twelve were in the program itself, not in its tests.

All three shipped fixed in v4.9.0. The maintainer then put the windows leg in himself and ran it with --no-fail-fast, which reached the nine binaries that had never executed on Windows at all: 34 failures, not 12, and 33 of them predated the report, spanning six root-cause groups. Two of those groups looked like Windows product defects and turned out not to be: the fixtures built their JSON by interpolating a filesystem path, which parses on Linux and not on Windows, and the parse error was swallowed into an empty result. I verified the product side separately against a real Windows install.

The suite is green there now, and the windows leg is a hard gate: a Windows failure blocks a merge, in a project that had never executed a single test on Windows before the report.

That same false claim was still sitting in the project's own CI workflow, where it is what a future reader consults before touching the leg. I opened a PR against it; it conflicted with a change that landed hours later, so the maintainer rebased my correction onto his own tree and merged it as 20e2932, co-authored. The run, the report, what it uncovered.

Before that, five merged PRs on splitrail. Streaming snapshots were summed instead of deduplicated, so subagent transcripts came out 50 to 60% heavier in input and cache tokens, dollar cost included (#220). Claude Opus 5 was missing from the price table, so all of its usage cost $0 (#221). And the checks workflow never ran cargo test while the release workflow shipped a signed Windows binary; it now runs on Linux and Windows (#231).

claude-night-market #656, merged: a report generator where a swallowed error produced the same empty output as a clean pass.

Claude-Code-Usage-Monitor #238, where six of the nine failures were tests asserting on a POSIX API that the program had already worked around.

phuryn/claude-usage came back clean, eight green jobs out of eight. That one is in the record too, with the same detail as the others.

Evaluations

eval-diff asks whether a code-review finding describes a defect the diff introduced or one that was already there. The gold comes from running a check on base and head, never from labelling by hand, and the cases nobody can decide are dropped. A positional baseline (0.40) was written down before any model ran: a result counts only if it beats it.

Client work

autoscuolaspagnoletto.it, a driving school in Milan. I audited the old WordPress site against six local competitors, then replaced it with a static site of nine pages, live 15 days after the audit. I am now writing its Python enrolment service. The code is private.

Elsewhere

path-of-gods, a 2D action-platformer in LÖVE2D/Lua, graded 29/30 as a university exam. Different problem, same habit of building the system before the content.

Earlier I built two things around agent permissions. claude-second-brain is a set of Claude Code skills where every skill declares what comes after it. PlutoGate treats permissions as contract clauses; today the repository is the design and a full statement of what does not work yet.

Pinned Loading

  1. claude-second-brain claude-second-brain Public

    A connected skill ecosystem for Claude Code — skills that know what comes next, anti-sycophancy as architecture, and a graph maintained by one of its own nodes.

  2. PlutoGate PlutoGate Public

    Permissions as contract clauses, not machine settings, an orchestrator with a memory gate for AI agents

  3. ccledger ccledger Public

    Measures how Claude Code usage counting rules differ on a real corpus: deduplication, usage.iterations aggregation, and models present

    Python

  4. istat-ipca-revisioni istat-ipca-revisioni Public

    La serie IPCA di ISTAT conservata con git: la cronologia mostra le revisioni dei dati, non le esecuzioni dello scraper.

    Python

  5. path-of-gods path-of-gods Public

    2D action-platformer built with LÖVE2D/Lua. Fight through the trials of five divine bosses, guided by Hermes, ending with Kronos, God of Time. University exam project (29/30): state machine, compon…

    Lua

  6. eval-diff eval-diff Public

    Una prova sul distinguere, in un diff, un difetto introdotto dal cambiamento da uno preesistente. La gold si deriva eseguendo, non si scrive a mano.

    Python