Skip to content
nodecodaPublic

About

gisel

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

gisel

设计论证:文章链接

One-page brief (Chinese): docs/BRIEF-zh.md

简体中文

ci

A free-form language for GlobalISel legalizer rules, and the toolchain that turns it into LLVM artifacts without llvm-tblgen.

gisel has two functions: turn a .gisel file into rule artifacts (gen/check/lint) and generate test cases (cases/domain). Importing LLVM's hand-written legalizer into .gisel and the two-implementation A/B live in the workbench (tools/) and are not part of gisel — see docs/SCOPE.md for the boundary and the rule that keeps it fixed. The loop those tools implement (pin → cases → A/B → attribution) is in docs/LOOP.md. Eight copy-paste tasks are in docs/COOKBOOK-zh.md and the vocabulary is in docs/GLOSSARY-zh.md (both Chinese).

# one command, no checkout: pipx -> uv tool -> venv, whichever exists
curl -fsSL https://raw.githubusercontent.com/nodecoda/gisel/main/scripts/install.sh | sh

gisel doctor                    # is this install able to work?
gisel demo                      # lint -> inc -> cases -> graph, ~25 s
gisel cases --corpus x86 --out x86.cases.jsonl    # 3014 answerless queries
gisel gen   --corpus x86 --emit=inc --out x86.inc   # the C++ chain
gisel check --corpus x86 --emit=inc --against x86.inc   # still current?
gisel lint  --corpus x86    # static checks, no LLVM

Or with no Python on the host at all:

docker run --rm ghcr.io/nodecoda/gisel:latest gisel demo

The nine corpora and their feature tables ship inside the wheel (gisel/data/corpus, gisel/data/cases); tests/ keeps the canonical copies because every frozen artifact records tests/corpus/... as the origin of its rulesets. --corpus NAME is sugar over "source + features" and works with or without a checkout.

No LLVM anywhere: docs/QUICKSTART-no-llvm.md walks install → doctor → questions → answers → graph in about five minutes.

Five minutes

One rule set, four rules, no LLVM build, no llvm-tblgen:

git clone https://github.com/nodecoda/gisel && cd gisel
mkdir -p /tmp/demo && cat > /tmp/demo/hello.gisel <<'EOF'
feature Is64Bit = Subtarget.is64Bit()
type I32 = LLT::scalar(32)
type I64 = LLT::scalar(64)

ruleset X_ADD: G_ADD
    legal in(slot(0), I32)
    legal (Is64Bit and in(slot(0), I64))
    widen(slot(0), pow2_min(32))
    scalarize(slot(0))
EOF

python3 -m gisel lint /tmp/demo/hello.gisel
# /tmp/demo/hello.gisel: 0 error(s), 0 warning(s), 4 rule(s)

python3 -m gisel gen   /tmp/demo/hello.gisel --emit=inc --out /tmp/demo/hello.inc
python3 -m gisel check /tmp/demo/hello.gisel --emit=inc --against /tmp/demo/hello.inc
# /tmp/demo/hello.inc: up to date

The generated .inc is the C++ chain, one call per rule:

  bool Is64Bit = Subtarget.is64Bit();

  const LLT I32 = LLT::scalar(32);
  const LLT I64 = LLT::scalar(64);

  getActionDefinitionsBuilder(G_ADD)
      .legalFor({I32})
      .legalFor(Is64Bit, {I64})
      .widenScalarToNextPow2(0, 32)
      .scalarize(0);

Every call is a public LegalizeRuleSet builder: the .inc compiles against an unmodified upstream LegalizerInfo.h, so replacing a target's hand-written rules needs no header patch. gisel-ab/scripts/inc/check-pin-header.sh compiles all three artifacts against a pin-only tree and self-tests that a core* name is rejected.

--emit=check and --emit=core give the other two shapes from the same source. The C++ .inc is the product; check and core are its inspection forms.

Commands

Nine commands, one product surface; spec/cli.md is the normative contract.

command what it does
gen emit one artifact: --emit=inc|check|core
check does an artifact still match the source?
lint static checks, no artifact
cases answerless test queries: equivalence, boundary, outside
domain the same space as a full Cartesian enumeration
eval answer queries against the schema-0.4 normal form
graph build the quotient graph and look for cycles
doctor is this install able to work? (no LLVM)
demo all of the above once, into one directory (~25 s)

Exit codes are shared by all nine: 0 success, 1 the answer is "no" (stale artifact, lint error, disagreeing answer, a cycle), 2 the run could not start (usage, IO, syntax). 2 is never a rule difference.

The graph, in one command

A legalizer that widens s24 to s32 and then narrows it back does not time out — it loops. graph builds the finite state space (opcode × the class of every type slot) from the normal form and runs a three-colour DFS on it:

gisel graph tests/corpus/X86LegalizerInfo.gisel --emit=core --config none
gisel graph tests/corpus/X86LegalizerInfo.gisel --emit=core --config none \
  --inject-edge                 # negative control: must report a 2-cycle
gisel graph tests/corpus/X86LegalizerInfo.gisel --emit=core --config none \
  --no-check-closure            # skip the audit (cycles=0 loses its premise)
gisel graph tests/corpus/AArch64LegalizerInfo.gisel --emit=core \
  --features tests/cases/features.json --config baseline --refine --jobs 8

spec/graph.md is the contract, docs/REPORT-graph.md the evidence. The closure audit is on by default: if a class member lands outside the enumerated domain or in a different successor class, the command exits 1 and cycles=0 is a statement about the partition that was checked, not a proof about the whole domain. --refine splits the offending classes; for X86 C1 that raises rc to 0 (docs/REPORT-graph.md §3). Every report also carries verdict: safe needs no cycles, a closed partition, and zero outside/opaque; a real target normally reads conditional with those counts on the same line, which is the honest boundary, not a failure.

Architecture

The front end (gisel/ast.py) parses .gisel into one model: a Program of declarations, rule sets and rules. Types are structured (Scalar(32), Pointer(0, None), Vector(...)), rule type positions hold those values, and a rule set is (name, opcodes, rules). Record names such as X86_s32, LegalizeFeature<...> and LegalizeRuleSet<...> are not part of the model: the C++ renderer computes declaration names at its exit through NameTable, and core/check never see them. ast.canonicalize_rulesets is the one AST transform before emission; it exists to keep the emitted C++ and the frozen core JSON at one table per opcode, not because the language forbids it.

Test cases are derived from that same model, not written by hand: gisel cases emits one representative per equivalence class (rule region), boundaries where the answer actually flips (N-1/N+1 on bits, lanes, memory size and alignment), and outside probes that fall through to Unsupported. --against compares only the overlap with an oracle dump; uncovered is reported separately and is never counted as a pass. spec/cases.md is the contract, docs/REPORT-cases.md the evidence.

Proving that two legalizer implementations answer the same way is done with gisel, not by gisel: gisel-ab/ freezes an answerless query set, runs the official build and the build carrying the generated .inc on that same file, and compares the two answer streams. Both directions are closed for X86, AArch64 and RISC-V: gisel-ab/docs/EVIDENCE-WP2.md asks gisel's own questions (no rule is missing upstream), gisel-ab/docs/EVIDENCE-U.md asks the oracle's questions (no upstream rule is missing in gisel). That workbench has its own report, gisel-ab/docs/REPORT-ab.md; docs/SCOPE.md is the boundary.

What it does not do: import-cxx refuses any statement it cannot translate rather than guessing, and none of the nine corpora is an independent hand-written second statement -- they were extracted from LLVM's hand-written C++ constructors with our own tooling as a preparation for self-proof. All of them go straight from the hand-written .cpp constructor to .gisel (gisel-anchor/scripts/import-cxx.py). docs/CORPORA.md says, per corpus, which upstream file it was extracted from, which reviewed deviations it carries, and what its frozen artifacts can and cannot prove (all three are self-baselines; rule fidelity comes from import-cxx --check and the end-to-end A/B).

Status

The current claims, their commands and their gates are docs/CLAIMS.md; the per-layer evidence is docs/CORPORA.md and gisel-ab/docs/EVIDENCE-*.md.

Migration and A/B tooling is workbench, not a gisel function (docs/SCOPE.md): gisel-anchor/scripts/import-inventory.py prices a target, gisel-anchor/scripts/import-cxx.py turns a constructor into .gisel (refusing, exit 1, any statement it cannot translate), and gisel-ab/ compares two implementations. import-cxx is a one-shot transcriber for the pinned constructors, not a general C++ parser: --all --check re-transcribes them and diffs the checked-in corpora, and a rule-count drift fails loudly. The evidence lives in docs/REPORT-riscv.md and gisel-ab/docs/REPORT-ab.md. The transcriber lives in the sibling gisel-anchor checkout; tools/import-cxx.py is a forwarding shim, not an implementation.

tests/artifacts/ holds the nine frozen C++/JSON artifacts and their SHA256: three each for X86, AArch64 and RISC-V. Upstream LLVM's legalizer rules live in hand-written *LegalizerInfo.cpp constructors, and the corpora are transcribed from those. The artifacts are therefore gisel's own frozen output — a drift baseline, not a byte oracle against a second implementation. Rule fidelity is checked on two other layers: gisel-anchor import --check re-transcribes the pinned C++ and must reproduce the checked-in .gisel byte for byte, and gisel-ab/ builds the official legalizer and the one carrying our .inc, feeds both the same answerless query file, and compares the answer streams. docs/CORPORA.md has the chain and the reviewed deviations.

python3 -m pytest -q                          # ok
python3 -m unittest discover -q -s tests      # ok
(cd tests/artifacts && sha256sum -c SHA256)   # 9/9
python3 tools/audit-docs.py                    # docs that can no longer be true

gisel lint runs the J1/J2b/J3a/J3b checks over the normal form and prints path:line: RULESET: message (exit 1 on errors, 2 on parse/IO). It needs no LLVM source tree: the opcode arity table ships in gisel/data/.

Errors point at the source, not at a synthetic string:

$ gisel gen bad.gisel --emit=inc --out x.inc
bad.gisel:6: expected ,, got ID ('pow2')      # exit 2

Evaluating without LLVM

--emit=core produces the schema-0.4 normal form, and gisel eval answers queries against it without any LLVM binary:

gisel gen tests/corpus/AArch64LegalizerInfo.gisel --emit=core --out a.core.json
gisel cases tests/corpus/AArch64LegalizerInfo.gisel \
  --features tests/cases/features.json --out a.cases.jsonl
gisel eval --doc a.core.json --features tests/cases/features.json \
  --cases a.cases.jsonl --answers a.answers.jsonl
# cases=... errors=0

gisel cases produces questions, not answers. --answers writes what the evaluator says in the same JSONL shape a compiled runner writes, so gisel-ab/diff.py can compare the two answer streams:

python3 ../gisel-ab/diff.py runner.answers.jsonl a.answers.jsonl --json

When a runner has already answered the same question file, gisel cases --against answers.jsonl compares only the overlap; uncovered is reported separately and is never counted as a pass.

Layout

gisel/ast.py            Program / declarations / rule sets / structured LLT
gisel/dsl.py            lexer, rule parser, core-IR normal form
gisel/backend/llvm_inc.py  AST -> .inc (M1)
gisel/backend/check_json.py / core_json.py              M2 / M4
gisel/eval.py           standalone evaluator (answers any question file)
gisel/cases.py          equivalence/boundary/outside case generation
gisel/answers.py        answer-file reader (the A/B comparison is gisel-ab/diff.py)
gisel/contract.py       shape contract: which type tuples are questions
gisel/domain.py         opcode-slot cartesian enumeration (the other question set)
gisel/graph.py          quotient graph over (opcode, type classes, mem, imm)
gisel/data/opcode-slots.json   283-opcode arity/slot snapshot
gisel/data/contract-shapes.jsonc / contract-aarch64.jsonc  shape tables
spec/cli.md             the command surface, exit codes, guarantees (normative)
spec/grammar.md         the .gisel grammar and AST, normative
spec/axes.md            the axis/profile format and its compatibility pin
spec/cases.md           what a generated case means, and the subset contract
spec/cases-format.md    the three JSONL projections and their shared key
spec/contract.md        shape contract: which tuples are questions at all
spec/escape-points.md   feature vs opaque: the two host escape points
spec/graph.md           the quotient-graph state, successor and outcomes
spec/semantics.md       what a rule means (and what the evaluator does)
docs/CORPORA.md         per-corpus source, deviations, and evidence layers
docs/LOOP.md            pin -> questions -> A/B -> attribution
docs/REPORT-cxx-corpus.md  C++ -> .gisel transcription (workbench tool)
docs/ALGORITHM-cases.md  case generation as a bounded quotient (article source)
docs/REPORT-cases.md    case generation: evidence, rejected options
docs/REPORT-coverage.md coverage evidence for the case set
docs/REPORT-graph.md    quotient-graph evidence and the cycle history
docs/REPORT-closure-why.md  why the unrefined partition is not closed
docs/DESIGN-graph.md    graph design record
docs/PLAN-internal-authority.md  fork 内部权威:双向题面、负控、oracle 保全
docs/PROPOSAL-host-escape.md  rejected host-escape proposal (AMDGPU)
docs/SCOPE.md           what gisel is, what only uses it

Ported emitter logic keeps the LLVM license header and names its source commit (d48f5512a162).

About

gisel

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages