Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .template_sync/manifest.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
template_version: 0.1.37
template_version: 0.1.38
migrations:
- from: 0.0.0
to: 0.0.1
Expand Down Expand Up @@ -117,3 +117,6 @@ migrations:
- from: 0.1.36
to: 0.1.37
script: migrations/v0_1_36_to_v0_1_37.py
- from: 0.1.37
to: 0.1.38
script: migrations/v0_1_37_to_v0_1_38.py
141 changes: 141 additions & 0 deletions .template_sync/migrations/v0_1_37_to_v0_1_38.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
"""Template 0.1.37 -> 0.1.38: the guides for app users and the update guide are merged, and several guides get clearer.

Design note: https://github.com/CellMigrationLab/LabConstrictor/blob/main/.tools/docs/migration_0_1_38.md

notebook_usage.md becomes "Open and use an app", the one first-run page for app users (it takes in the website's page of that name).
keep_app_up_to_date.md is merged into template_synchronization.md, now titled "Keep your app up to date", and becomes a short pointer
page. The token, requirements, installed-app troubleshooting, installer and manifest guides are made clearer, and the docs README drops
the merged page.

What this changes in an app repository: each file in FILES is replaced by the 0.1.38 page when it is still the 0.1.37 template file
(compared with LF line endings; the file keeps its own line endings). A file the app changed itself stays as it is (SKIPPED, with a note).
That includes keep_app_up_to_date.md, which becomes the pointer page only when it is the 0.1.37 template copy, and .tools/docs/README.md
(in most apps the repository form replaced it with the template's README, so it is skipped with a note of what to change by hand). A
missing file is not created, and nothing is deleted. external_code_upload.md is also recognised with the app's Python package name in place
of PYTHON_PROJ_NAME. The changes reach users with the app's next release.
"""
from __future__ import annotations

import re
from pathlib import Path
from typing import Any


DOCS_URL = "https://github.com/CellMigrationLab/LabConstrictor/blob/main/.tools/docs/migration_0_1_38.md"
TAG = "[migration 0.1.38]"
RESOURCE_ROOT = Path(__file__).resolve().parents[1] / "resources" / "v0_1_37_to_v0_1_38"
PREVIOUS_ROOT = RESOURCE_ROOT / "previous"

DOCS = Path(".tools/docs")
EXTERNAL_CODE_DOC = DOCS / "external_code_upload.md"

DOCS_INDEX = DOCS / "README.md"
POINTER_PAGE = DOCS / "keep_app_up_to_date.md"

# Files replaced whole, only when they are the 0.1.37 template file (compared with LF line endings).
FILES = tuple(
DOCS / name
for name in (
"README.md",
"app_manifest.md",
"before_getting_started.md",
"create_repository.md",
"executable_creation.md",
"external_code_upload.md",
"install.md",
"keep_app_up_to_date.md",
"notebook_requirements.md",
"notebook_upload.md",
"notebook_usage.md",
"personal_access_token.md",
"template_synchronization.md",
"tools_bridge.md",
"troubleshooting.md",
"troubleshooting_installed_app.md",
)
)

# What to change by hand in a customised file, by file (added to the SKIPPED note).
BY_HAND = {
DOCS_INDEX: (
"if it lists keep_app_up_to_date.md, remove that entry (template_synchronization.md is now the update guide, "
'titled "Keep your app up to date"), and notebook_usage.md is now titled "Open and use an app"'
),
POINTER_PAGE: "its content is now in template_synchronization.md; you can replace it with a link to that page",
}

# The placeholder the repository form replaces in a guide, by guide: the form writes the app's Python package name in place of it.
RENDERED = {EXTERNAL_CODE_DOC: "PYTHON_PROJ_NAME"}
IDENTIFIER = r"[A-Za-z_][A-Za-z0-9_]*"


def log(message: str) -> None:
print(f"{TAG} {message}")


def lf_text(path: Path) -> str:
return path.read_bytes().decode("utf-8").replace("\r\n", "\n")


def filled_in_name(template_lf: str, text_lf: str, placeholder: str) -> str | None:
"""The name written in `text_lf` everywhere `template_lf` has `placeholder` (the same name each time), or None when the texts differ
elsewhere."""
if placeholder not in template_lf:
return None
parts = template_lf.split(placeholder)
pattern = re.escape(parts[0]) + f"(?P<name>{IDENTIFIER})" + "".join(re.escape(part) + "(?P=name)" for part in parts[1:-1])
pattern += re.escape(parts[-1])
match = re.fullmatch(pattern, text_lf, flags=re.DOTALL)
return match.group("name") if match else None


def replaced_file(relative: Path, text_lf: str) -> tuple[str, str]:
"""The 0.1.38 text of `relative` for the app's `text_lf`, and what was done. The text is unchanged when it is not the 0.1.37 template file."""
new = lf_text(RESOURCE_ROOT / relative)
previous = lf_text(PREVIOUS_ROOT / relative)
if text_lf == new:
return text_lf, "already up to date"
if text_lf == previous:
return new, "replaced by the 0.1.38 page"
placeholder = RENDERED.get(relative)
if placeholder:
name = filled_in_name(new, text_lf, placeholder)
if name:
return text_lf, "already up to date"
name = filled_in_name(previous, text_lf, placeholder)
if name:
return new.replace(placeholder, name), f"replaced by the 0.1.38 page (with the package name {name})"
by_hand = f" By hand: {BY_HAND[relative]}." if relative in BY_HAND else ""
return text_lf, f"SKIPPED: customised (not the 0.1.37 template file), so it was left unchanged; compare it with the template's page.{by_hand} See {DOCS_URL}"


def update(path: Path, relative: Path) -> str:
data = path.read_bytes().decode("utf-8")
newline = "\r\n" if "\r\n" in data else "\n"
original = data.replace("\r\n", "\n")
text, note = replaced_file(relative, original)
if text == original:
return note if note.startswith("SKIPPED") else f"unchanged: {note}"
path.write_bytes(text.replace("\n", newline).encode("utf-8"))
return f"updated: {note}"


def migrate(repo_root: Path, context: dict[str, Any]) -> None:
"""Apply the 0.1.38 documentation changes (idempotent, never raises for a customised file, creates and deletes nothing)."""
_ = context
changed = 0
for relative in FILES:
path = repo_root / relative
if not path.is_file():
log(f"{relative}: not present in this repository (nothing was created)")
continue
try:
status = update(path, relative)
except (OSError, UnicodeDecodeError) as exc:
status = f"SKIPPED: {exc}; left unchanged. See {DOCS_URL}"
changed += status.startswith("updated")
log(f"{relative}: {status}")
if changed:
log(f"done: {changed} file(s) changed. The changes reach users with the next release of the app.")
else:
log("done: no repository changes were required.")
39 changes: 39 additions & 0 deletions .template_sync/resources/v0_1_37_to_v0_1_38/.tools/docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Internal LabConstrictor documentation

This file temporarily lists all internal LabConstrictor documentation. After you set up and customise the repository, it will be replaced by the [main README.md](../../README.md) file.

## From zero to installed notebooks

- [Before getting started](before_getting_started.md)
- [Create a new repository from this template](create_repository.md)
- [Initialise your repository](initialise_repository.md)
- [Accept a Pull Request](accept_pull_request.md)
- [Generate notebook requirements](notebook_requirements.md)
- [Upload your notebooks](notebook_upload.md)
- [Create executable installers](executable_creation.md)
- [Install an app](install.md)
- [Download and install the application](download_executable.md) (this file will be available after creating executables)
- [Open and use an app](notebook_usage.md)
- [Troubleshooting an installed app](troubleshooting_installed_app.md)

## Customise and share (optional)

- [Customise the installer images](installer_images.md)
- [Describe your app for the App Centre](app_manifest.md)
- [Upload external code, and publish it on PyPI (optional)](external_code_upload.md)
- [Expose tools to napari, Fiji and QuPath](tools_bridge.md)

## Keep your app up to date (optional)

- [Keep your app up to date](template_synchronization.md)

## Notebook authoring tips

- [Write notebooks that run in both Google Colab and JupyterLab](notebook_portability.md)
- [Hide code cells in notebooks](code_hiding.md)

## GitHub management

- [GitHub actions/workflows overview](workflow_status.md)
- [Troubleshooting repository setup](troubleshooting.md)
- [Create a personal access token for notebook access](personal_access_token.md)
Original file line number Diff line number Diff line change
@@ -0,0 +1,200 @@
# Describe your app: the `labconstrictor-app.yaml` file

`labconstrictor-app.yaml` is a small text file in the root of your repository. It says what your app is, in your own words. You write it once and keep it up to date yourself.

Three things read it:

- **The [App Centre](https://cellmigrationlab.github.io/LabConstrictor-Site/apps/)** builds your app's card and page from it.
- **Your release workflow** reads `platforms` to decide which installers to build.
- **Your installer** passes `hosts` and `host_requirements` to napari, Fiji and QuPath, so they know whether to list your app.

Version numbers and download links are not in the file. They always come from your latest GitHub release.

## Step 1: get the file

The [repository initialiser](initialise_repository.md) asks for a one-sentence summary, the kind of app and the software it also works in. It adds the file to the initialisation pull request. Read it there and edit it if you like.

If you skipped that section, or your repository is older than the file, create `labconstrictor-app.yaml` in the root of your repository and paste the minimal example below. The [Submit page](https://cellmigrationlab.github.io/LabConstrictor-Site/submit/) also has a form that writes the file for you and checks it as you type.

## Step 2: start with the minimal file

These five fields are required. This file is complete and valid:

```yaml
schema: 1
name: My App
summary: Counts nuclei in 2D fluorescence images and saves a table.
kind: analysis
distribution:
installer: true
```

## Step 3: add what helps people choose your app

A fuller example. Everything after `distribution` is optional.

```yaml
# yaml-language-server: $schema=https://cellmigrationlab.github.io/LabConstrictor-Site/schema/labconstrictor-app.schema.json
schema: 1
name: My App
summary: Counts nuclei in 2D fluorescence images and saves a table.
description: |-
A few paragraphs of plain text.

Separate paragraphs with a blank line.
kind: analysis
hosts: [notebook, napari, qupath]
host_requirements:
napari:
min_version: 0.1.0-alpha.5
why: Older plugins do not show the outlines
required: false
distribution:
installer: true
platforms: [windows-x86_64, macos-arm64, linux-x86_64]
colab:
- label: Notebook
url: https://colab.research.google.com/github/you/my-app/blob/main/notebooks/App/App.ipynb
pip: my-app
status: Available
license: MIT
authors:
- name: Ada Lovelace
orcid: 0000-0002-1825-0097
doi: 10.1371/journal.pbio.3002740
inputs:
- 2D TIFF images, one channel per file
outputs:
- Label masks and a CSV table
limitations:
- Not tested on 3D data
quickstart:
- Open the app and run the Welcome notebook.
categories: [nuclei, counting]
homepage: https://github.com/you/my-app
docs_url: https://github.com/you/my-app#readme
icon: app/logo/logo.png
screenshots:
- path: docs/screenshot.png
caption: The result table next to the segmented image
```

The first line lets editors with YAML support (for example VS Code) check the file as you type. Everything is plain text: no HTML and no Markdown.

## The fields

| Field | Required | What it does | Example |
|---|---|---|---|
| `schema` | Yes | The format version. Always `1`. | `1` |
| `name` | Yes | The name people know the app by (2 to 60 characters). | `My App` |
| `summary` | Yes | One sentence for the app card (20 to 200 characters). | `Counts nuclei in 2D images.` |
| `kind` | Yes | What the app is for. One of `analysis`, `segmentation`, `simulation`, `game`, `training`, `utility`. It drives the filter in the App Centre. | `analysis` |
| `distribution` | Yes | How people get the app. At least one of `installer`, `colab`, `pip`. See below. | `installer: true` |
| `hosts` | No | Where the app's tools run: `notebook`, `napari`, `fiji`, `qupath`. See below. | `[notebook, napari]` |
| `host_requirements` | No | The oldest napari, Fiji or QuPath plugin your tools need. See below. | |
| `description` | No | A few paragraphs for the app page (up to 2000 characters). | |
| `status` | No | `Experimental`, `Available` or `Deprecated`. | `Available` |
| `license` | No | The licence, as an SPDX name. | `MIT` |
| `authors` | No | A list of people: `name`, and optionally `orcid`. | `- name: Ada Lovelace` |
| `doi` | No | The DOI of the paper to cite, not of a software archive. The app page shows it as the citation. | `10.1371/journal.pbio.3002740` |
| `inputs`, `outputs`, `limitations` | No | Short lists for the app page: one to six lines of 5 to 200 characters each. | `- 2D TIFF images` |
| `quickstart` | No | Up to six short steps for a first run. | `- Open the app.` |
| `categories` | No | Up to six words a scientist would search for. | `[nuclei, counting]` |
| `homepage`, `docs_url` | No | Web addresses. They must start with `https://`. | |
| `icon` | No | A square PNG in your repository. SVG is not accepted. | `app/logo/logo.png` |
| `screenshots` | No | One to six pictures in your repository (PNG, JPG or GIF), each with a `path` and a `caption`. The caption is also the text for screen readers. | |

The App Centre asks for a few optional fields: `status`, `quickstart`, `screenshots`, a licence, and `inputs` and `outputs` for analysis, segmentation and simulation apps. Without them your app is still listed; its page says "Not stated" or leaves that part out.

### `distribution`: how people get the app

| Key | What it does |
|---|---|
| `installer: true` | Your latest GitHub release carries the installers LabConstrictor builds. |
| `platforms` | The systems you support. Needs `installer: true`. Only these installers are built and offered. |
| `colab` | A list of Google Colab notebooks, each with a `label` and a `url` on `colab.research.google.com`. Only the notebooks you list are shown. |
| `pip` | The name of your package on PyPI. |

`platforms` also decides which installers your releases build. Leave a system out when your app cannot run there. Without `platforms`, every installer is built: `windows-x86_64`, `linux-x86_64`, `macos-arm64` (Apple silicon) and `macos-x86_64` (Intel Macs). `windows-arm64` and `linux-arm64` are accepted, but they build nothing yet: no installer is made for them, and a list with only these two stops the release.

### `hosts`: where your tools are offered

This field matters only if your app [exposes tools](tools_bridge.md). napari, Fiji and QuPath list your app only if you name them here. Write the hosts you have tested the app in.

- `hosts: [notebook]` means no plugin lists the app. Its tools still work in the notebook and on the command line.
- Without a `hosts` line, every plugin lists the app. Say what you mean instead.
- Names are lower case. The check refuses a misspelt, repeated or capitalised name.

## Plugin versions your tools need (optional)

If your tools rely on a fix or a feature of the napari, Fiji or QuPath plugin that older plugins lack, say so with `host_requirements`. People with an older plugin keep seeing your app, with one sentence saying which plugin to update and where; your app page on the App Centre lists the versions under "Plugin versions".

```yaml
host_requirements:
qupath:
min_version: 0.1.0-alpha.7 # a release tag of the plugin, with or without v
features: [shapes_owner_v1] # optional: a host feature id
why: Replace keeps your own annotations # optional: one line shown to the person
fiji:
min_version: 0.1.0-alpha.7
```

- Name only `napari`, `fiji` or `qupath`: the notebook ships with your app and needs no plugin.
- Each needs `min_version`, `features`, or both. `required: true` makes an older plugin refuse your tools instead of warning; use it only when they really fail there.
- Write it as a block, as above. The installer passes this file to `labconstrictor-tools register`, which refuses a malformed block (a misspelt host or key, a version like `latest`); `labconstrictor-tools check --module <your>_lc_tools --app-manifest labconstrictor-app.yaml` shows what it read.
- The feature ids and the rule are in [the protocol reference](https://cellmigrationlab.github.io/LabConstrictor-Site/guides/protocol/#host-requirements-which-plugin-an-app-needs).

## The automatic check

The **Check App Centre manifest** workflow runs when you change the file and when you publish a release:

- If the file is missing, it shows a warning and passes.
- If the file is invalid, it fails and says what to fix. This never stops your installers from being built.
- It rejects fields it does not know (this catches typos, so the schema is not loosened); the template keeps its copy of the schema in step with the App Centre, see [Keep your app up to date](template_synchronization.md).
- It also checks that the icon and screenshots you name exist in the repository.

## Check it yourself

To check the file before you push:

1. **On your computer.** From the root of your repository:

```bash
pip install pyyaml jsonschema
python .tools/python/check_app_manifest.py
```

2. **In your browser.** Paste your values into the form on the [Submit page](https://cellmigrationlab.github.io/LabConstrictor-Site/submit/). It checks them against the same schema.

If your app exposes tools, also run `labconstrictor-tools check --module <your>_lc_tools --app-manifest labconstrictor-app.yaml`. It prints the `hosts` and `host_requirements` it read, which is what the installer will register.

The rules themselves are in the [schema](https://cellmigrationlab.github.io/LabConstrictor-Site/schema/labconstrictor-app.schema.json). Your repository has a copy in `.tools/schema/`, and the check uses that copy.

## Common mistakes

| What the check says, or what you see | Fix |
|---|---|
| A field "is not allowed" or is "unexpected" | The name is misspelt, or it is at the wrong level. For example `platforms` belongs under `distribution`. |
| `summary` is too short or too long | Write one sentence of 20 to 200 characters. |
| `platforms` without `installer` | Add `installer: true`. `platforms` only applies to installers. |
| The icon or a screenshot is "not found" | The path starts at the root of your repository, with no leading `/`. The file must be committed. |
| A Colab link is refused | The address must start with `https://colab.research.google.com/`. |
| `min_version: 0.2` is refused | YAML reads `0.2` as a number. Put it in quotes: `"0.2"`. |
| Your app is missing from napari, Fiji or QuPath | The host is not in `hosts`, or the app was installed before you added it. Publish a release and install it again. |
| The check passes but nothing changes in the App Centre | The App Centre reads the file from your default branch and updates once a day. |
| Text shows `**stars**` or tags | The fields are plain text. Remove Markdown and HTML. |

## Get listed

Having the file does not list your app by itself. To list it, [submit your repository](https://cellmigrationlab.github.io/LabConstrictor-Site/submit/) to the App Centre. You do this once: later releases appear by themselves.

---

<div align="center">

[← Previous](installer_images.md) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
[🏠 Home](README.md) &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
[Next →](external_code_upload.md)


</div>
Loading
Loading