Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .template_sync/manifest.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
template_version: 0.1.31
template_version: 0.1.32
migrations:
- from: 0.0.0
to: 0.0.1
Expand Down Expand Up @@ -99,3 +99,6 @@ migrations:
- from: 0.1.30
to: 0.1.31
script: migrations/v0_1_30_to_v0_1_31.py
- from: 0.1.31
to: 0.1.32
script: migrations/v0_1_31_to_v0_1_32.py
132 changes: 132 additions & 0 deletions .template_sync/migrations/v0_1_31_to_v0_1_32.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
"""Template 0.1.31 -> 0.1.32: the user docs link to the guide that owns a topic instead of repeating it.

Design note: https://github.com/CellMigrationLab/LabConstrictor/blob/main/.tools/docs/migration_0_1_32.md

Each topic of the LabConstrictor documentation is written on one page, and every other page links to it. The plugins' install steps belong
to the plugin guides on the website, and how to write tools belongs to LabConstrictor-Tools. The template keeps what is its own: where an
app declares its tools and what the installer does with them.

What this changes in an app repository (every step is independent, idempotent and never deletes anything):

1. .tools/docs/tools_bridge.md is replaced by the 0.1.32 guide when it is still the 0.1.31 template file. The guide no longer repeats the
plugins' install steps, the list of form markers and the test commands of LabConstrictor-Tools, or the version of the Tools pin; it links
to the owner guides instead. A file the app changed itself stays as it is (SKIPPED).
2. .tools/docs/install.md: "Where the app is listed" writes napari in lower case and links to the guide that says how to run the tools.
3. .tools/docs/README.md: the entry of the tools guide writes napari in lower case.

Steps 2 and 3 replace one exact piece of template text each and leave a file alone when that text is not there (SKIPPED). The line endings
of each file are kept. The changes reach users with the app's next release.
"""
from __future__ import annotations

from pathlib import Path
from typing import Any


DOCS_URL = "https://github.com/CellMigrationLab/LabConstrictor/blob/main/.tools/docs/migration_0_1_32.md"
TAG = "[migration 0.1.32]"
RESOURCE_ROOT = Path(__file__).resolve().parents[1] / "resources" / "v0_1_31_to_v0_1_32"
PREVIOUS_ROOT = RESOURCE_ROOT / "previous"

TOOLS_DOC = Path(".tools/docs/tools_bridge.md")
INSTALL_DOC = Path(".tools/docs/install.md")
DOCS_INDEX = Path(".tools/docs/README.md")

SITE = "https://cellmigrationlab.github.io/LabConstrictor-Site/"

# Files replaced whole, only when they are the 0.1.31 template file (compared with LF line endings).
WHOLE_FILES = (TOOLS_DOC,)

# (old, new) pairs, written with LF line endings; a file with CRLF line endings gets them with CRLF.
EDITS: dict[Path, tuple[tuple[str, str], ...]] = {
INSTALL_DOC: (
("If the app ships tools for Napari, Fiji or QuPath, the installer registers them with LabConstrictor Tools",
"If the app ships tools for napari, Fiji or QuPath, the installer registers them with LabConstrictor Tools"),
("with `hosts: [notebook, napari]` the app is listed in Napari, and not in Fiji or QuPath.",
"with `hosts: [notebook, napari]` the app is listed in napari, and not in Fiji or QuPath."),
("so it is listed in every host; ...`).\n",
"so it is listed in every host; ...`).\n"
"\n"
"To run the tools, install the plugin for your image software once: see"
f" [Use apps inside napari, Fiji and QuPath]({SITE}guides/image-software/).\n"),
),
DOCS_INDEX: (
("- [Expose tools to Napari, Fiji and QuPath (experimental)](tools_bridge.md)\n",
"- [Expose tools to napari, Fiji and QuPath (experimental)](tools_bridge.md)\n"),
),
}
FILES = (*WHOLE_FILES, *EDITS)


def log(message: str) -> None:
print(f"{TAG} {message}")


def apply_edits(text_lf: str, edits: tuple[tuple[str, str], ...]) -> tuple[str, int, int]:
"""`text_lf` with each pair applied whose old text occurs exactly once and whose new text is not there yet. Returns the text, the
pairs applied and the pairs not found (neither the old text once nor the new text present)."""
applied = missing = 0
for old, new in edits:
if new in text_lf:
continue
if text_lf.count(old) == 1:
text_lf = text_lf.replace(old, new)
applied += 1
else:
missing += 1
return text_lf, applied, missing


def replaced_file(relative: Path, text_lf: str) -> tuple[str, str]:
"""The 0.1.32 text of a whole-file step, and what was done. The text is unchanged when the file is not the 0.1.31 template file."""
new = (RESOURCE_ROOT / relative).read_bytes().decode("utf-8").replace("\r\n", "\n")
if text_lf == new:
return text_lf, "already up to date"
previous = (PREVIOUS_ROOT / relative).read_bytes().decode("utf-8").replace("\r\n", "\n")
if text_lf != previous:
return text_lf, f"SKIPPED: customised (not the 0.1.31 template file), so it was left unchanged; compare it with the template's file. See {DOCS_URL}"
return new, "replaced by the 0.1.32 guide"


def edited_file(relative: Path, text_lf: str) -> tuple[str, str]:
"""The text after the exact-text edits of `relative`, and what was done."""
edits = EDITS[relative]
text, applied, missing = apply_edits(text_lf, edits)
if not missing:
return text, f"{applied} text change(s)" if applied else "already up to date"
if missing == len(edits):
return text, f"SKIPPED: customised (the 0.1.31 text was not found), so it was left unchanged. See {DOCS_URL}"
skipped = f"SKIPPED {missing} passage(s) that differ from the template text (left as they are). See {DOCS_URL}"
return text, f"{applied} text change(s); {skipped}" if applied else f"the other changes are there; {skipped}"


def update(path: Path, relative: Path) -> str:
data = path.read_bytes().decode("utf-8")
newline = "\r\n" if "\r\n" in data else "\n"
original = data.replace("\r\n", "\n")
text, note = replaced_file(relative, original) if relative in WHOLE_FILES else edited_file(relative, original)
if text == original:
return note if note.startswith("SKIPPED") else f"unchanged: {note}"
path.write_bytes(text.replace("\n", newline).encode("utf-8"))
return f"updated: {note}"


def migrate(repo_root: Path, context: dict[str, Any]) -> None:
"""Apply the 0.1.32 documentation changes (idempotent, surgical, never raises for a customised file)."""
_ = context
changed = 0
for relative in FILES:
path = repo_root / relative
if not path.is_file():
log(f"{relative}: not present in this repository (nothing was created)")
continue
try:
status = update(path, relative)
except (OSError, UnicodeDecodeError) as exc:
status = f"SKIPPED: {exc}; left unchanged. See {DOCS_URL}"
changed += status.startswith("updated")
log(f"{relative}: {status}")
if changed:
log(f"done: {changed} file(s) changed. The changes reach users with the next release of the app.")
else:
log("done: no repository changes were required.")
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# Expose your app's tools to napari, Fiji and QuPath (optional, experimental)

If your repository ships Python code (see [Upload external code](external_code_upload.md)), you can declare some of its functions
as **tools**. After installation, people can run them in napari, Fiji and QuPath and from the command line, with forms made from the
function signatures. Nothing changes for apps that do not declare tools.

This page covers what the template does: where you declare the tools and what the installer does with them. Two other guides cover the rest:

- [Writing tools](https://github.com/CellMigrationLab/LabConstrictor-Tools/blob/main/docs/AUTHORING.md), in LabConstrictor-Tools: the inputs
and outputs a tool can have, how to lay out its form, what each program supports, and how to test a tool.
- [Use apps inside napari, Fiji and QuPath](https://cellmigrationlab.github.io/LabConstrictor-Site/guides/image-software/): what the people
who install your app do to run its tools, with the install steps of the
[napari plugin](https://cellmigrationlab.github.io/LabConstrictor-Site/guides/napari/), the
[Fiji plugin](https://cellmigrationlab.github.io/LabConstrictor-Site/guides/fiji/) and the
[QuPath extension](https://cellmigrationlab.github.io/LabConstrictor-Site/guides/qupath/).

## Declare your tools

1. Add a module named `<your_package>_lc_tools` next to your package under `src/` (so `src/mytool_lc_tools/__init__.py` for
the package `mytool`). The installer registers tools only from a module with exactly this name. Declare tools in it with typed
functions, and import your science code **inside** the functions:

```python
from typing import Annotated
from labconstrictor_tools import Image, ImageOut, Min, tool

@tool("Gaussian blur")
def blur(image: Image, sigma: Annotated[float, Min(0)] = 2.0) -> ImageOut:
from mytool.filters import gaussian
return gaussian(image, sigma)
```

Keep the module light: it is imported to read the tool list, so heavy imports at the top slow every start.
2. Import `labconstrictor_tools` only in `<your_package>_lc_tools`, or inside functions, if you can. The rest of your package then also works
where LabConstrictor Tools is not installed: a notebook run on Google Colab, a copy of the package used elsewhere, or an installation where
Tools failed to install. An import at the top of your package does not stop the installation, because the installer installs Tools before
it checks that your package can be imported.
3. Name the programs your tools work in under `hosts:` in `labconstrictor-app.yaml`: see [Describe your app for the App Centre](app_manifest.md).
4. Test your tools before you publish a release, as [Writing tools](https://github.com/CellMigrationLab/LabConstrictor-Tools/blob/main/docs/AUTHORING.md)
shows (`labconstrictor-tools check` and `labconstrictor-tools test`).

## What the installer does

The installer's `post_install` step:

1. installs your package;
2. if your package has a `<your_package>_lc_tools` module, installs LabConstrictor Tools;
3. checks that your package can be imported;
4. registers the app (`labconstrictor-tools register ...`) with the version from the `version:` line of `construct.yaml` and the hosts of
`labconstrictor-app.yaml`.

When the app is uninstalled, `pre_uninstall` unregisters it.

A failure in step 2 or 4 is written as a `WARNING` to `menuinst_debug.log` in the install folder, the registration is skipped, and the
installation goes on. The one exception is a package that needs Tools to be imported: then the import check of step 3 fails, as it should.

The Windows step (`post_install.bat`) runs in the template's automated tests on a Windows machine, and people have installed apps on Windows
and macOS from a clean system.

### Which LabConstrictor Tools the installer installs

LabConstrictor Tools is not on PyPI yet. The installer downloads the GitHub archive of one release tag of LabConstrictor-Tools, a plain zip
file, so the user's computer does not need git. The tag is pinned: two installers built from the same app release install the same Tools,
and a change to Tools does not reach new installations until the pin is moved on purpose. Template updates move the pin in your app.

The tag is the value of `LC_TOOLS_PIN` in both scripts:
[app/bash_bat_scripts/post_install.sh](../../app/bash_bat_scripts/post_install.sh) (`LC_TOOLS_PIN="..."`) and
[app/bash_bat_scripts/post_install.bat](../../app/bash_bat_scripts/post_install.bat) (`SET "LC_TOOLS_PIN=..."`).

To install Tools from somewhere else, set the environment variable `LC_TOOLS_SPEC` when the installer runs: a wheel, a git URL, a mirror, or
the name `labconstrictor-tools` once it is on PyPI. It replaces the pin completely.

**Move the pin (template maintainers).** Create and test the tag in LabConstrictor-Tools (the procedure is in its `docs/OPERATIONS.md`).
Change `LC_TOOLS_PIN` in both scripts of the template, and in the copies under `.template_sync/resources/` of the latest migration if it has
them (the test `test_resources_are_identical_to_the_template_files` fails when they differ). Then write a migration that changes the string in
the apps' scripts, and bump `template_version`.

## If something does not work

- `menuinst_debug.log` in the app's install folder says what the installer did with your tools.
- `labconstrictor-tools doctor` checks the Tools installation, and `labconstrictor-tools logs` shows its log.
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Expose your app's tools to Napari, Fiji and QuPath (optional, experimental)

If your repository ships Python code (see [Upload external code](external_code_upload.md)), you can declare some of its functions
as **tools**. After installation they appear in a generic **Napari** widget, a generic **Fiji** command and a generic **QuPath** command, and work from the
command line, with forms generated automatically from the function signatures. Nothing changes for apps that do not declare tools.

1. Add a module named `<your_package>_lc_tools` next to your package under `src/` (so `src/mytool_lc_tools/__init__.py` for
the package `mytool`). Declare tools in it with typed functions, importing your science code **inside** the functions:

```python
from typing import Annotated
from labconstrictor_tools import Image, ImageOut, Min, tool

@tool("Gaussian blur")
def blur(image: Image, sigma: Annotated[float, Min(0)] = 2.0) -> ImageOut:
from mytool.filters import gaussian
return gaussian(image, sigma)
```

Keep the module light: it is imported to read the tool list, so heavy imports at the top slow every start.
**Import `labconstrictor_tools` only in the tools module if you can.** Since template 0.1.17 the installer installs `labconstrictor-tools` *before* it checks that
your package can be imported, so a package that imports `labconstrictor_tools` at module level no longer stops the installation. It is still better practice to import it lazily (inside functions)
or only in `<your_package>_lc_tools`: the rest of your package then also works where Tools is not installed (a notebook run on Colab, a copy of the package used elsewhere, a failed Tools install).
2. The installer (`post_install`) detects the module, installs `labconstrictor-tools` and registers the app
(`labconstrictor-tools register ...`). `pre_uninstall` unregisters it. A failure in this step is logged to
`menuinst_debug.log` and never fails the installation.
**Order in `post_install`.** (1) your package is installed, (2) if it ships `<your_package>_lc_tools`, `labconstrictor-tools` is installed, (3) the package import check runs, (4) the app is registered.
Installing Tools before the import check is what lets your package import `labconstrictor_tools` at module level. A failed Tools install is a `WARNING` in `menuinst_debug.log`
and the registration is skipped; the installation continues, unless your package needs Tools to be imported (then the import check fails, as it should).
**Where `labconstrictor-tools` comes from.** It is not on PyPI yet, so by default the installer downloads the GitHub archive of a **pinned release tag**,
`https://github.com/CellMigrationLab/LabConstrictor-Tools/archive/refs/tags/v0.3.0-alpha.2.zip` (a plain zip: git is not needed on the user's computer).
Set the environment variable `LC_TOOLS_SPEC` when the installer runs to install it from somewhere else (a wheel, a git URL, a mirror, or the
name `labconstrictor-tools` once it is published on PyPI); it overrides the pin completely.
**Pinned (important).** Two installers built from the same app release get the same version of the tools package, and a change to Tools `main`
does not reach new installations until the pin is moved on purpose.
**How to move the pin.** The tag name is written in one place per script: the variable `LC_TOOLS_PIN` (`LC_TOOLS_PIN="v0.3.0-alpha.2"` in
`app/bash_bat_scripts/post_install.sh`, `SET "LC_TOOLS_PIN=v0.3.0-alpha.2"` in `post_install.bat`). To use a new release: create and test the tag in
LabConstrictor-Tools (procedure in its `docs/OPERATIONS.md`), change that string in both scripts and in the copies under
`.template_sync/resources/` of the latest migration (the test `test_resources_are_identical_to_the_template_files` fails when they differ),
update the tag in this paragraph, and bump `template_version` so apps receive the change. Apps already migrated need a new migration to change the
string in their own scripts.
3. Test your tools before releasing: `labconstrictor-tools check --module mytool_lc_tools` and
`labconstrictor-tools test --module mytool_lc_tools --cases lc_tests/cases.json`.
4. Install the Napari plugin (`napari-labconstrictor`), the Fiji plugin (`labconstrictor-fiji`) or the QuPath extension once per machine. None of them is on PyPI or an update site yet:
see the READMEs of [LabConstrictor-napari](https://github.com/CellMigrationLab/LabConstrictor-napari),
[LabConstrictor-Fiji](https://github.com/CellMigrationLab/LabConstrictor-Fiji) and [LabConstrictor-QuPath](https://github.com/CellMigrationLab/LabConstrictor-QuPath) for the install steps.
5. Optional: make a long form readable and rich with markers (the authoring guide in the Tools repository lists them all):
- layout: `Group("...")`, `Advanced()`, `Collapsed()`, `EnabledWhen(...)`;
- input widgets: `Widget("slider")` for a bounded number and `Widget("radio")` for a short list of choices;
- `PickChannel()` to let the person choose one channel of a multi-channel image (your function receives that channel as a 2D array);
- `ChoicesFrom("other_tool")` to fill a dropdown from another tool of yours (for example the available devices);
- `RegionOf("image")` (optional labels input) to run on the selection in the host (Napari Shapes layer, Fiji ROI, QuPath annotations);
- outputs: `ImageOut`, `LabelsOut`, `TableOut`, `Scalars`, `MessageOut`, `PointsOut` and `ShapesOut` (polygons: Napari shapes layer, Fiji ROIs, QuPath annotations);
- say "nothing found" with `ToolError("no_match", ...)`.
Napari, Fiji and QuPath do not support every feature in the same way: the table "What each host does with each feature" in `docs/HOST_FEATURES.md` of the Tools repository is the reference.
Documentation and source: https://github.com/CellMigrationLab/LabConstrictor-Tools.
The registered version is read from the `version:` line of `construct.yaml`. The Windows hook (`post_install.bat`) runs in the template's automated tests on a Windows
machine, and people have installed apps on Windows and macOS from a clean system.
Troubleshooting: `labconstrictor-tools doctor`, `labconstrictor-tools logs`, and `~/.labconstrictor/logs/labconstrictor.log`.
2 changes: 1 addition & 1 deletion .tools/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ This file temporarily lists all internal LabConstrictor documentation. After you
- [Customise the installer images](installer_images.md)
- [Describe your app for the App Centre](app_manifest.md)
- [Upload external code, and publish it on PyPI (optional)](external_code_upload.md)
- [Expose tools to Napari, Fiji and QuPath (experimental)](tools_bridge.md)
- [Expose tools to napari, Fiji and QuPath (experimental)](tools_bridge.md)

## Keep your app up to date (optional)

Expand Down
Loading
Loading