Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 48 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,13 @@ permissions:

jobs:
dart:
name: Dart package
name: Dart package (code_assets ${{ matrix.code-assets }})
runs-on: ubuntu-24.04
timeout-minutes: 15
strategy:
fail-fast: false
matrix:
code-assets: [lowest, latest]
permissions:
contents: read
steps:
Expand All @@ -30,10 +34,53 @@ jobs:
with:
sdk: '3.13.0'
- run: dart pub get
- name: Resolve the oldest supported code_assets
if: matrix.code-assets == 'lowest'
run: dart pub downgrade code_assets
- run: dart format --output=none --set-exit-if-changed .
- run: dart analyze --fatal-infos
- run: dart test

windows-wsl:
name: Native Windows Dart to WSL2
runs-on: windows-2025
timeout-minutes: 30
permissions:
contents: read
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: dart-lang/setup-dart@6afc89df92d6eb3834022f73cd65adc8cdfcb92d # v1.8.1
with:
sdk: '3.13.0'
- uses: Vampire/setup-wsl@4c48163d9f981914dd8caa0fc335fe19cca5bff4 # v7.2.0
with:
distribution: Ubuntu-24.04
wsl-version: 2
set-as-default: 'true'
wsl-conf: |
[boot]
systemd=true
additional-packages: curl xz-utils git ca-certificates
- name: Install Nix inside WSL2
shell: wsl-bash {0}
run: |
# Keep this installer pin in sync with native.yml.
curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix/tag/v3.11.2 \
| sh -s -- install linux --no-confirm
- name: Resolve dependencies with native Windows Dart
shell: pwsh
run: dart pub get
# Run Dart in Windows, so the smoke tests must call wsl.exe themselves.
# The tiny store fixture needs no compiler or native-library rebuild.
- name: Test the native Windows to WSL process and filesystem boundary
shell: pwsh
env:
DNIX_TEST_NIX: '1'
DNIX_TEST_WSL_DISTRIBUTION: Ubuntu-24.04
run: dart test --reporter expanded

# The fixture crate goes through the same reusable workflow consumers use.
fixture:
name: Fixture
Expand Down
13 changes: 13 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,18 @@
## Unreleased

* Native Windows source and development builds invoke Nix through WSL2.
The hook translates paths, builds the requested non-Apple target, copies
the library back, and retains source mode's pinned hash verification.
The optional `wsl_distribution` user define selects a distribution;
otherwise the default is used. `verify_release`, and `update_lock` for
packages without Apple targets, also work from Windows through WSL2.
This changes only the Dart build runner; Nix recipes, native sources,
and expected library hashes are unchanged.

* Source builds resolve the flake from the package directory, supporting
parent paths containing `#`. Windows source dependencies with non-ASCII
filenames are tracked correctly.

* Three more Windows targets beside `x86_64-pc-windows-msvc`:
`x86_64-pc-windows-gnullvm` and `aarch64-pc-windows-gnullvm` build with zig's
MinGW-w64 toolchain, for C and C++ dependencies that need GCC-style flags,
Expand Down
8 changes: 7 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,13 @@ dart tool/verify_release.dart linux
| Linux aarch64 | yes | Linux and Windows targets |
| macOS on Apple Silicon | yes | every target; Apple ones need Xcode 27.0 |
| macOS on Intel | yes | no |
| Windows | yes | no; inside WSL2 `verify_release` covers Linux, Android, and Windows targets, and locks come from CI |
| Windows | yes | non-Apple targets through WSL2; the Linux host limits above apply |

On Windows, install Nix inside a WSL2 Linux distribution. Native Windows
Dart and Flutter invoke it through `wsl.exe`, then copy the built library
back and verify its hash. The default WSL distribution is used unless the
package's hook user defines specify `wsl_distribution`; see
[Windows source builds](doc/configuration.md#windows-source-builds).

## More

Expand Down
51 changes: 46 additions & 5 deletions doc/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,12 +190,11 @@ hooks:
| Mode | What happens | Needs |
|---|---|---|
| `prebuilt` | Downloads the library of the release `dnix.lock` names and verifies its hash. | Dart only. |
| `source` | Builds with Nix exactly as the release did and verifies the same hash. | Nix on Linux or an Apple Silicon Mac; the pinned Xcode for Apple targets. |
| `dev` | Builds with Nix, skipping the Xcode pin and the hash check. | Nix on Linux or an Apple Silicon Mac. |
| `source` | Builds with Nix exactly as the release did and verifies the same hash. | Nix on Linux, an Apple Silicon Mac, or WSL2 on Windows; the pinned Xcode for Apple targets. |
| `dev` | Builds with Nix, skipping the Xcode pin and the hash check. | Nix on Linux, an Apple Silicon Mac, or WSL2 on Windows. |

On Windows and on Intel Macs the hook can only use prebuilt libraries. Nix
runs on Windows inside WSL2, where the tools work but the hook of a Windows
Dart process does not.
Intel Macs can only use prebuilt libraries. On Windows the hook invokes
Nix inside WSL2 for non-Apple targets, as described below.

In `source` and `dev` mode the hook declares the flake files and the
workspace's files (the git-tracked ones in a checkout, which is what Nix
Expand All @@ -207,3 +206,45 @@ The hook refuses static linking and refuses builds whose deployment target
is below the floors, since those could not match the release. Flutter asks
for iOS 13 and macOS 13 whatever the application's own deployment target,
so floors above those make every Apple build fail.

### Windows source builds

Install WSL2 with a Linux distribution and install Nix inside it. Flutter
and Dart stay on Windows. In `source` and `dev` mode, dnix starts
`wsl.exe`, translates the package and output paths with `wslpath`, and
builds the requested target in Nix's Linux store. A Windows target still
builds a Windows DLL. The library is copied back to the Windows hook
output directory; `source` mode then checks its hash against `dnix.lock`
before Flutter bundles it. A build or hash failure stops the hook.

The build reads the checked-out native files, including local edits. Keep
native text in LF form through the package's `.gitattributes`, and preserve
any required symlinks and executable permissions. CI builds a fresh clone
on WSL's Linux filesystem, so check for checkout differences when diagnosing
a Windows-only hash mismatch. The expected hash remains the release hash.

By default, dnix uses the default WSL distribution. Inspect installed
distributions with `wsl.exe --list --verbose`, or select one per package:

```yaml
hooks:
user_defines:
frostdart:
build: source
wsl_distribution: Ubuntu-24.04
```

The selected distribution must be set up for WSL2, and must be able to
access the package and output directories. Nix is found on its `PATH` or
in `~/.nix-profile/bin`, `/nix/var/nix/profiles/default/bin`, or NixOS's
`/run/current-system/sw/bin`, without loading login-shell profiles. The
distribution's existing Nix configuration is retained, and flakes are
enabled for the build. Configure proxies and other Nix environment settings
inside the distribution; Windows environment variables are not forwarded
automatically. The first build downloads Nix's pinned inputs; later builds
reuse its store as usual.

No separate application `dnix.lock` is needed. The package supplies the
same flake, native sources, and expected hashes used on other hosts.
Apple targets still require a Mac; Android targets need an x86_64 Linux
distribution because the NDK is not available on Linux arm64.
41 changes: 39 additions & 2 deletions doc/tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,49 @@ void main(List<String> args) async => exit(await verifyRelease(args));
```

Rebuilds one platform and compares with the lock: `linux`, `android`,
`windows`, `macos`, `ios`, or a target triple. Inside WSL2 it covers the
non-Apple platforms.
`windows`, `macos`, `ios`, or a target triple. Native Windows Dart invokes
Nix in the default WSL2 distribution for non-Apple platforms. Set that
distribution with `wsl.exe --set-default <name>`; the `wsl_distribution`
user define applies to build hooks, not these standalone tools.

`update_lock` also uses WSL2 when called from Windows for a package with
only non-Apple targets. A package shipping Apple targets still needs a
Mac to write its full lock, or the lock printed by CI.

`writeLock`, also in `package:dnix/tools.dart`, writes a lock from
`SHA256SUMS` files. CI uses it; a package rarely needs a shim for it.

Both tools exit 1 with a one-line reason on any problem dnix can name:
a mistake in `dnix.json`, a missing or inconsistent lock, a platform the
package does not ship, or Nix missing from PATH.

## Testing the build runner

`dart test` checks WSL argument handling and failures, and exercises the
shell script with stand-in tools on POSIX hosts. A separate opt-in smoke
test exercises the real Nix process and file copying with a tiny store
fixture. It also runs the build hook to check Windows target selection,
hash verification, and rejection of mismatched bytes before an asset is
registered. It requires no compiler toolchains or native-library rebuilds:

```sh
DNIX_TEST_NIX=1 dart test test/nix_build_test.dart
```

CI runs the smoke tests from native Windows Dart in PowerShell, with Nix
installed in WSL2. They cover both the default and an explicitly selected
distribution, paths containing spaces, Unicode and literal percent escapes,
and replacing an existing output. The Dart analysis and unit tests also run
with the oldest and latest supported `code_assets` versions.

Run it from native Windows Dart to exercise the WSL handoff. In PowerShell,
with Nix installed in the default WSL2 distribution:

```powershell
$env:DNIX_TEST_NIX = '1'
# Optional: also exercise explicit distribution selection.
$env:DNIX_TEST_WSL_DISTRIBUTION = 'Ubuntu-24.04'
dart test test/nix_build_test.dart
Remove-Item Env:DNIX_TEST_NIX
Remove-Item Env:DNIX_TEST_WSL_DISTRIBUTION
```
9 changes: 5 additions & 4 deletions doc/workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,8 +102,9 @@ else
fi
```

Windows runners can only use `prebuilt`, so skip the tests there until the
release exists.
Windows runners without WSL2 and Nix can only use `prebuilt`, so skip the
tests there until the release exists. Configured runners can use `source`
through the native Windows hook's WSL handoff.

## Before publishing to pub.dev

Expand All @@ -124,8 +125,8 @@ dart tool/verify_release.dart linux # or android, windows, macos, ios, or a t

Rebuilds that platform with Nix and compares each library with the lock.
The Apple targets use the Xcode bundle under `/Applications` that reports
the pinned version, whatever `xcode-select` points at. Inside WSL2 this
covers the Linux, Android, and Windows targets.
the pinned version, whatever `xcode-select` points at. On Windows the tool
invokes Nix in the default WSL2 distribution for non-Apple targets.

## Bumping dnix, Rust, or nixpkgs

Expand Down
41 changes: 26 additions & 15 deletions lib/src/hook.dart
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import 'dart:convert';
import 'dart:io';

import 'package:code_assets/code_assets.dart';
Expand Down Expand Up @@ -110,6 +111,7 @@ Future<void> _build(BuildInput input, BuildOutputBuilder output) async {
fileName: fileName,
outputDirectory: input.outputDirectory,
anyXcode: false,
wslDistribution: input.userDefines['wsl_distribution'],
);
verifyHash(
bytes: file.readAsBytesSync(),
Expand All @@ -124,6 +126,7 @@ Future<void> _build(BuildInput input, BuildOutputBuilder output) async {
fileName: fileName,
outputDirectory: input.outputDirectory,
anyXcode: true,
wslDistribution: input.userDefines['wsl_distribution'],
);
output.dependencies.addAll(sourceFiles(root, config).where(declared.add));
}
Expand All @@ -144,9 +147,8 @@ String pinnedHash(DnixLock lock, String target, String fileName) {
throw DnixException(
'dnix.lock has no $fileName for $target: the package was locked '
'without that target, and its maintainer adds it with update_lock. '
'Build mode "dev" builds it locally without a pinned hash, on Linux '
'or on an Apple Silicon Mac; a Windows process can only use a '
'prebuilt library.',
'Build mode "dev" builds it locally without a pinned hash, using '
'Nix on Linux, an Apple Silicon Mac, or WSL2 on Windows.',
);
}
return expected;
Expand All @@ -155,20 +157,26 @@ String pinnedHash(DnixLock lock, String target, String fileName) {
/// Fails when the consumer's deployment target is below the floor the
/// libraries are built for, since such a build could not match the release.
void checkFloors(CodeConfig code, DnixConfig config) {
// code_assets 1.2.1 has no deploymentVersion API. Keep its integer API
// while supporting both 1.x and 2.x; dnix's floors are integer majors.
final (String, num, int, String)? check = switch (code.targetOS) {
OS.macOS => (
'macOS',
// ignore: deprecated_member_use
code.macOS.targetVersion,
config.macOSFloor,
'This build asks every native asset for macOS '
// ignore: deprecated_member_use
'${code.macOS.targetVersion}, whatever the application targets, so '
'${config.name} must lower "floors.macos" in ${DnixConfig.fileName}.',
),
OS.iOS => (
'iOS',
// ignore: deprecated_member_use
code.iOS.targetVersion,
config.iOSFloor,
'This build asks every native asset for iOS '
// ignore: deprecated_member_use
'${code.iOS.targetVersion}, whatever the application targets, so '
'${config.name} must lower "floors.ios" in ${DnixConfig.fileName}.',
),
Expand Down Expand Up @@ -202,11 +210,10 @@ void checkFloors(CodeConfig code, DnixConfig config) {
/// as modified.
Iterable<Uri> sourceFiles(Uri packageRoot, DnixConfig config) sync* {
yield* existingFiles(packageRoot, ['flake.nix', 'flake.lock']);
final rootPath = Directory.fromUri(packageRoot).path
.replaceAll(RegExp(r'/+$'), '');
for (final directory in config.sourceDirectories) {
final base = Directory(
directory == '.' ? rootPath : '$rootPath/$directory',
final base = Directory.fromUri(
// The config holds a filesystem path, not an already-encoded URI.
packageRoot.resolveUri(Uri(pathSegments: [...directory.split('/'), ''])),
);
if (!base.existsSync()) continue;
yield* gitTrackedFiles(base) ?? _files(base, root: base);
Expand All @@ -221,7 +228,9 @@ List<Uri>? gitTrackedFiles(Directory directory) {
result = Process.runSync(
'git',
['-C', directory.path, 'ls-files', '-z', '--cached', '--', '.'],
stdoutEncoding: systemEncoding,
// Git emits UTF-8 paths with -z, including on Windows where the
// system encoding may otherwise turn them into nonexistent names.
stdoutEncoding: utf8,
stderrEncoding: systemEncoding,
);
} on ProcessException {
Expand All @@ -230,13 +239,15 @@ List<Uri>? gitTrackedFiles(Directory directory) {
if (result.exitCode != 0) return null;
// Dart's own tool directory is written during a build, so even if it is
// tracked it must not be an input.
return [
for (final name in (result.stdout as String).split('\u0000'))
if (name.isNotEmpty &&
!name.startsWith('.dart_tool/') &&
File('${directory.path}/$name').existsSync())
File('${directory.path}/$name').uri,
];
final files = <Uri>[];
for (final name in (result.stdout as String).split('\u0000')) {
if (name.isEmpty || name.startsWith('.dart_tool/')) continue;
final file = File.fromUri(
directory.uri.resolveUri(Uri(pathSegments: name.split('/'))),
);
if (file.existsSync()) files.add(file.uri);
}
return files;
}

Iterable<Uri> _files(Directory directory, {required Directory root}) sync* {
Expand Down
Loading
Loading