A progressively enhanced build orchestrator in Rust. neomake reads a
TOML (or TOMLX) configuration file, builds a dependency DAG, runs tasks
concurrently on a tokio executor, and skips unchanged work via a
SHA-256 content-addressable cache.
- TOML in, TOML out. Your build file starts as plain TOML. When you
need more, opt in to TOMLX — a strict superset adding variables,
value-level conditionals, and a small library of built-ins
(
env,glob,exec). - Parallel by default. Tasks with no dependency run concurrently.
Concurrency is capped with
--concurrency N(default: number of CPUs). - Content-addressable cache. Each task's cache key is a SHA-256 of its command, env, sorted input file hashes, and the cache keys of its upstream dependencies, so a change anywhere in the transitive input closure invalidates every dependent.
- Clear diagnostics. Cycle detection reports the concrete cycle path; every config error carries a file name and line number.
neomake/ CLI binary
neomake-core/ task model, DAG builder, cache, tokio executor
neomake-tomlx/ TOMLX lexer, parser, evaluator
neomake-machine/ QEMU machines for machine tests
examples/ basic.toml, advanced.tomlx
docs/tomlx-spec.md formal TOMLX spec
docs/testing.md tests, services and machines
Requires a recent stable Rust toolchain (1.75+).
cargo install --path neomakeA cargo build --release at the workspace root produces
target/release/neomake.
neomake is available from the AUR. With yay or paru:
yay -S neomake
# or
paru -S neomakeOther repositories can call the reusable workflow to install neomake
from this repo and run it against the caller’s checkout:
jobs:
ci:
uses: theJoeLinuxFoundation/neomake/.github/workflows/reusable-neomake.yml@main
with:
neomake-ref: main
config-path: neomake.tomlSee .github/workflows/reusable-neomake.yml
for all inputs (working-directory, tasks, concurrency, extra-args, …).
neomake runs on Linux, macOS, and Windows. Task commands are passed
to a shell whose choice depends on the host:
- Unix-likes —
sh -c <command>. - Windows —
pwsh -NoProfile -NoLogo -Command <command>if PowerShell 7+ is onPATH, otherwisepowershell.exe(Windows PowerShell 5.1), otherwisecmd.exe /S /Cas a last-resort fallback.
Set NEOMAKE_SHELL to override; its value is whitespace-tokenized as
argv and the user's command is appended as the final argument:
NEOMAKE_SHELL="bash -c" neomake run # force bash
NEOMAKE_SHELL="cmd /S /C" neomake run # force cmd.exe on WindowsCache compatibility. The cache format is
neomake-cache-v3: the key now covers a task's name, shell, working directory and output patterns as well. Entries written by an older neomake miss once after upgrading;neomake cache cleanreclaims their space.
Windows PowerShell 5.1 redirection.
>writes UTF-16 + BOM by default in 5.1 (this is fixed in pwsh / PowerShell 7+). For deterministic byte-level output, preferpwshor useSet-Content -Encoding ascii/-Encoding utf8NoBOMinstead.
Create neomake.toml:
[tasks.build]
command = "cargo build --release"
inputs = ["src/**/*.rs", "Cargo.toml"]
outputs = ["target/release/myapp"]
[tasks.test]
command = "cargo test"
deps = ["build"]Then:
neomake run # runs every task (test → build)
neomake run build # builds just `build` and its deps
neomake list # shows the DAG in topological order
neomake cache status # prints cache entry count and size
neomake cache clean # empties the cache directory| Field | Meaning |
|---|---|
command |
The shell command (required). |
deps |
Tasks that must succeed first. |
inputs, outputs |
Glob patterns; inputs are hashed into the cache key, outputs checked on a hit. |
env, env_clear |
Variables to set; start from an empty environment. |
cwd |
Working directory, relative to the project root. |
shell |
"bash -euo pipefail -c" or an argv array; the command is appended. |
timeout, retries |
"10m", "1h30m", "500ms" or seconds; repeat a failed attempt this often. |
cache |
false to always run (the default for tests). |
allow_failure |
A failure does not fail the run. |
requires |
{ commands = [...], files = [...], env = [...] }: when unmet, the task is skipped with the reason instead of failing. |
skip |
A reason to skip the task (or true). |
exclusive |
Named locks: tasks naming the same one never overlap. |
interactive |
Run alone, attached to the terminal. |
description, tags |
For people, and for selecting tasks. |
services |
Services the task uses, started before it runs. |
[settings] in the top-level file sets state_dir (where the cache,
runs and logs go; .neomake by default), default_timeout, shell and
keep_runs.
Every run gets a directory <state_dir>/runs/<time>/ holding one log
per task under logs/ and report.json, a machine-readable account of
the run; runs/latest names the newest. Every command runs in its own
process group, which is stopped as a whole at a timeout or on Ctrl-C.
[tests.*] are tasks that check something: they always run (unless
cache = true), and may be written as steps that check each command's
exit status and output, capture values for later steps, answer a
program's questions on a pseudo-terminal, and come in matrix variants.
A failure names the step and the line it is written on. See
docs/testing.md.
[tests.cli]
deps = ["dave"]
steps = [
{ section = "help" },
{ run = 'dave --help', contains = "Usage:" },
{ run = 'dave isntall', exit = "nonzero", stream = "combined", contains = "did you mean 'install'?" },
][services.*] are processes tests need while they run, started before
their first user and stopped after their last: a command with a
readiness check (a port, a file, a log line, a URL), or neomake's own
static file server over HTTP or HTTPS. A taken port is reported as such
rather than discovered later. See docs/testing.md.
[services.repo]
serve = { dir = "build/repo", listen = [{ addr = "127.0.0.1:8643" }] }
[tests.install]
services = ["repo"]
steps = [{ run = 'dave --repo {{services.repo.url}} install hello' }][machines.*] are QEMU virtual machines that tests run their steps in,
over the serial console: commands and their checks, waiting for output,
reboots, power cuts, file writes, screenshots and media changes, on a
fresh copy of the machine's disks each run. A host without KVM, QEMU or
UEFI firmware skips such tests with the reason. See
docs/testing.md.
[machines.scaffold]
disk = "build/images/scaffold.img"
console = { prompt = "joe# ", banner = "BdsDxe: starting " }
[tests.smoke]
machine = "scaffold"
steps = [
{ run = 'findmnt -rno FSTYPE /', equals = "btrfs" },
{ power_cut = {} },
{ run = 'cat /var/lib/staged', equals = "7" },
]neomake [OPTIONS] <SUBCOMMAND>
Subcommands:
run [PATTERN]... Run tasks and tests (defaults to everything)
test [PATTERN]... Run tests (defaults to every test)
list [--tests] Print tasks and tests in the order they run
check Check the configuration without running anything
log [TASK] [--run ID] Print a task's log from the last run
cache clean Remove every cache entry
cache status Print cache size and entry count
machine list Print the machines
machine run NAME Boot a machine on this terminal
Options:
-c, --config <FILE> Path to config; auto-detects neomake.toml or neomake.tomlx
--concurrency <N> Max parallel commands (default: num_cpus)
--no-cache Treat every task as a cache miss
-q, --quiet Show only the output of what fails
-v, --verbose Show every step as it passes
--no-kvm Treat KVM as missing
run and test options:
--tag <TAG> Only what has this tag (repeatable)
--exclude-tag <TAG> Leave out what has this tag (repeatable)
--dry-run Show what would run, be up to date or be skipped
--fail-fast Stop at the first failure
--timeout-scale <F> Multiply every timeout by F
--output <MODE> stream, grouped or failures
--junit <FILE> Write JUnit XML of the tests to FILE
--json <FILE> Also write the run's report to FILE
--tail <N> Lines of a failed task's output to show (default 20)
--machines <N> Machine tests that may run at once
Exit status: 0 when every task succeeded, was cached or was skipped; 1 when one failed or timed out; 130 when interrupted.
Set NEOMAKE_LOG=debug to turn on tracing diagnostics.
| Feature | Syntax |
|---|---|
| Variable declaration | $profile = "release" |
| Variable reference | ${profile} (in strings) / $profile (in expressions) |
| String interpolation | "target/${profile}/app" |
| String concatenation | "cargo build" + " --release" |
| Conditional value | if ${ci} { "make ci" } else { "make" } |
| Environment access | env("PROFILE", "debug") |
| File globbing | glob("src/**/*.rs") |
| Command execution | exec("git rev-parse HEAD") |
| Git clone | git("github:owner/repo", "v1.0.0", "dest/%") |
| File download | fetch("https://url.com/file", "dest/%") |
| Path helpers | exists("x"), which("qemu-system-x86_64"), join(a, b) |
| Split configurations | include = ["tests/*.tomlx"] |
| Shell left untouched | command = 'echo "${HOME}" $?' (literal strings) |
See docs/tomlx-spec.md for the full grammar and
semantics, and examples/advanced.tomlx for
a complete example.
┌───────────────────┐ ┌──────────────────────────┐
│ *.toml / *.tomlx │──────▶│ neomake-tomlx │
└───────────────────┘ │ (lexer / parser / eval) │
└───────────┬──────────────┘
│ toml::Value
▼
┌──────────────────────────┐
│ neomake-core │
│ TaskSet → DAG │
│ │ │
│ ▼ │
│ Executor (tokio) │
│ │ │
│ ▼ │
│ Cache (.neomake/cache) │
└──────────────────────────┘
- DAG construction uses Kahn's algorithm for topological sorting; if
a cycle is detected, an iterative DFS recovers the concrete cycle
path (e.g.
a -> b -> c -> a) and reports it. - Cache keys are SHA-256 over a versioned canonical byte layout
(
neomake-cache-v2) built from the command, env, sorted input file hashes, and upstream dep keys. Seeneomake-core/src/cache.rsfor the exact format. - Parallel execution uses one tokio task per DAG node, a per-task
watchchannel to propagate outcomes to dependents, and aSemaphoreto cap concurrent shell commands. Cache hits bypass the semaphore entirely.
MIT