Skip to content

[Ascend] feat: integrate Triton-distributed-ascend into FlagTree - #156

Open
TelGome wants to merge 5 commits into
KernelLLM:triton_v3.6_ascendfrom
TelGome:ascend_triton_distributed
Open

TelGome wants to merge 5 commits into
KernelLLM:triton_v3.6_ascendfrom
TelGome:ascend_triton_distributed

Conversation

@TelGome

@TelGome TelGome commented Nov 17, 2026

Copy link
Copy Markdown

Motivation:
The Ascend backend could not compile Triton-distributed kernels: the Distributed dialect, its TTIR->HIVM conversion and the frontend builder support were missing.

Design:

  • Add TRITON_BUILD_TD and build the TritonDistributed plugin from the vendored Triton-distributed-ascend C++ subset (Distributed dialect, distributed->hivm conversion, Python bindings) into libtriton as the distributed submodule.
  • Register "distributed" in TRITON_PLUGIN_NAMES and link the plugin against the AscendNPU-IR/BiShengIR tablegen targets, adding FlagPrism include paths when the FlagPrism profiler replaces Proton.
  • Build the frontend with distributed.ir.DistributedOpBuilder in both the core and ascend-spec code generators so triton_dist primitives can emit Distributed dialect ops.
  • Vendor the triton_dist Python package and the Ascend tutorials, and package triton_dist in the wheel when TRITON_BUILD_TD is enabled.
  • Add the NvidiaGPU tablegen targets to TritonGPUIR's dependencies so parallel builds do not race on the generated headers.

Test:
Two-rank Ascend 950DT sweeps of the triton_dist allgather and MoE dispatch examples pass when the build also enables TRITON_BUILD_NPUIR, which bundles the BiShengIR payload carrying the ACLSHMEM device templates.

Risks:
The new path is opt-in (TRITON_BUILD_TD defaults to OFF); existing Ascend and other backend builds are unchanged.

Motivation:
The Ascend backend could not compile Triton-distributed kernels: the
Distributed dialect, its TTIR->HIVM conversion and the frontend builder
support were missing.

Design:
- Add TRITON_BUILD_TD and build the TritonDistributed plugin from the
  vendored Triton-distributed-ascend C++ subset (Distributed dialect,
  distributed->hivm conversion, Python bindings) into libtriton as the
  `distributed` submodule.
- Register "distributed" in TRITON_PLUGIN_NAMES and link the plugin against
  the AscendNPU-IR/BiShengIR tablegen targets, adding FlagPrism include
  paths when the FlagPrism profiler replaces Proton.
- Build the frontend with distributed.ir.DistributedOpBuilder in both the
  core and ascend-spec code generators so triton_dist primitives can emit
  Distributed dialect ops.
- Vendor the triton_dist Python package and the Ascend tutorials, and
  package triton_dist in the wheel when TRITON_BUILD_TD is enabled.
- Add the NvidiaGPU tablegen targets to TritonGPUIR's dependencies so
  parallel builds do not race on the generated headers.

Test:
Two-rank Ascend 950DT sweeps of the triton_dist allgather and MoE dispatch
examples pass when the build also enables TRITON_BUILD_NPUIR, which bundles
the BiShengIR payload carrying the ACLSHMEM device templates.

Risks:
The new path is opt-in (TRITON_BUILD_TD defaults to OFF); existing Ascend
and other backend builds are unchanged.
Motivation:
The vendored Triton-distributed-ascend subtree and the triton_dist package
carried the NVIDIA/AMD/METAX backends even though the Ascend build never
compiles or imports them, which made the tree large and harder to maintain.

Design:
- Drop the unused LLVM/TritonGPU conversions, the SIMT dialect and the
  non-Ascend pybind entry from the vendored TDA subtree, and simplify the
  parent CMakeLists accordingly.
- Remove the NVIDIA/AMD kernels, layers, function, models, benchmark,
  mega-triton-kernel and their tests/utilities from triton_dist; keep the
  shared platform dispatch, tools, Ascend kernels and Ascend tests.
- Trim the triton_dist package list in setup.py to match.

Test:
- Full Ascend build succeeds and still produces the seven TDA objects; the
  `distributed` plugin exposes ascend_passes and DistributedOpBuilder.
- IR-level distributed tests: 11 passed.
- Two-rank Ascend 950DT sweeps with the pruned package: triton_dist
  allgather 10/10 and MoE dispatch 10/10 passed.

Risks:
The vendored TDA subtree now only supports the Ascend lowering and can no
longer build the NVIDIA/AMD paths. Existing Ascend and other backend builds
are otherwise unchanged.
Motivation:
The Ascend-only triton_dist still shipped files for backends it cannot run
on: the CUDA/HIP/MetaX language extras, MetaX and shared NV kernel helpers,
the NV-only profiler and its common_ops dependency, and a benchmark config
module that nothing imports. common_ops also raises at import time outside
CUDA/HIP, so it could never be imported in this tree.

Design:
- Remove language/extra/{cuda,hip,maca}, kernels/{metax,allreduce.py,
  common_ops.py}, tools/profiler/ and test/utils.py.
- Trim the setup.py package list to match (no .extra.cuda/hip,
  tools.profiler).
- run_dist_test() now hides the source-tree python/ directory while
  spawning workers. pytest's prepend import mode adds that directory at
  package-setup time, so spawned workers imported the unbuilt source triton
  and died with "cannot import name 'getenv' from triton._C.libtriton".

Test:
- Repo-source imports work: triton_dist, language.extra, kernels.ascend,
  tools.
- IR tests: 11 passed.
- Previously failing spawn tests now pass with the default import mode
  (test_wait_notify 4/4, test_symm_at, test_signal_op, ...).
- The remaining test_sync_ops/test_team_pe failures are unresolved ACLSHMEM
  device-template symbols in the bundled BiShengIR
  (_mlir_ciface_aclshmem_sync_all.{cube,vector},
  _mlir_ciface_aclshmemx_udma_put_nbi_int32,
  _mlir_ciface_aclshmem_team_pe_mapping), unrelated to this change.

Risks:
The tree is Ascend-only; the removed modules were unreachable or
unimportable here. Other backends must use upstream triton-distributed.
Motivation:
Review feedback: the shared python/triton/compiler/code_generator.py was
patched to build a distributed.ir.DistributedOpBuilder, but that hook is
Ascend-only and already lives in
third_party/ascend/spec/triton/compiler/code_generator.py, which shadows the
shared module through spec_path() on Ascend builds.

Design:
- Revert the shared file to self.builder = ir.builder(context); the Ascend
  spec overlay keeps the DistributedOpBuilder selection unchanged.
- The shared file is now identical to the base revision.

Test:
- python -m py_compile on both code_generator.py files.
- git diff against e8b1b75 for the shared file is empty.
@TelGome
TelGome requested a review from togepro November 19, 2026 08:13
Motivation:
The pinned AscendNPU-IR (efb714158, release/v2.0.x) predates the
ACLSHMEM device wrappers that Triton-distributed kernels need, so
kernels failed to link with undefined _mlir_ciface_aclshmem_* symbols
and the bundled BiShengIR templates could not resolve sync/team symbols.

Design:
- Bump AscendNPU-IR to 8bc48fb585 (on origin/stable), whose Shmem.cpp
  provides the aclshmem_sync_all / team_pe_mapping wrappers and whose
  --build-shmem-template output contains the .cube/.vector symbols.
- Link BiShengIRTensorViewDialect and MLIRAVEDialect into triton-opt and
  triton-mlir-opt, as required by the newer AscendNPU-IR.
- Hash libtriton_distributed in triton_dist_key() only when the
  standalone library exists; the Ascend build links the distributed
  bindings into libtriton.so.
- Allow the BiShengIR build tree to live outside the source tree and be
  reused via ASCENDNPUIR_BUILD_ROOT / ASCENDNPUIR_REUSE_BUILD.

Test:
- Built 0.7.0+ascend.git0543cf98 with TRITON_BUILD_TD=1 and
  TRITON_BUILD_NPUIR=ON; the bundled mix templates now contain
  _mlir_ciface_aclshmem_sync_all and _mlir_ciface_aclshmem_team_pe_mapping.
- Distributed api suite (per-file, 4x Ascend950DT): 3060 passed,
  14 skipped, 0 failed; test_sync_ops 19/19, test_team_pe 4/4,
  test_wait_notify 4/4.
- Remaining: 6 RoCE collective runtime cases fail because the installed
  shmem wheel is built with RDMA Backend NONE (environment only).

Risks:
- The pin is a commit on origin/stable; no release tag contains it yet.
- ASCENDNPUIR_REUSE_BUILD may keep stale objects when sources change;
  the default remains a from-scratch build.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants