Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -455,6 +455,17 @@ src/morphobricks_preprocessing/
tuned per unit (edited), contradictions refused and tagged on their
own; channel_matrix.py (pure) its rules (trusted, layout_groups,
matrix_rows, set_template, fill)
notebook_channels_override.py the per-unit override of a unit the repeat rows list: an editor
that starts from the row's tags, shows each header name and
highlights the C indices where it disagrees; nothing is saved or
counted until Confirm (repeat_tags.Override, override_holds: the
confirmation holds while the unit's tags and own header names are
unchanged; one-click Confirm the same tags / Confirm all N like it for
listed units with a confirmed unit's headers; a refusal keeps the
aliases of its time)
repeat_review.py the refusals and confirmations kept in the draft file (pure: Override,
RefusedUnit, TagReview; draft_format's optional tag_review, restored
on opening by RepeatRows.restore before headers are trusted)
strict_draft.py fast mode unticked (pure): StrictShadow, the strict page's draft kept
beside the draft; every user save applies the same change to both
(channel_answers.tagged, patched), no-click tags the draft only;
Expand Down
35 changes: 35 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,41 @@ old pin are refused with `spec_commit_mismatch`, as at every re-pin.

### Added

- **Tag by repeat: another output folder is another draft** (owner requirement): pointing only the
output folder at another draft of the same experiment reopens the channel grid on that draft
(the draft is part of its reopening key), so its refused units stay listed and untagged; the
header trust restores the draft's own record first when the memory is another draft's
(`MatrixView.trust_headers`), and the save stays refused until it is.
- **Tag by repeat: override a listed unit** (owner requirement): from the list of units whose
headers disagree with their repeat's row, **Override this unit (start from the row's tags)**
opens the unit's tags prefilled with the row's, header names shown and disagreements
highlighted. The unit counts as tagged only after **Confirm these tags**; the confirmation
holds while the unit's own tags and header names are unchanged (`repeat_tags.Override`,
`override_holds`; `notebook_channels_override`). A refused unit is counted only when the user
confirms that unit: aliases saved by confirming or tagging another unit never clear its
refusal, never tag it in fast mode (`RepeatRows.standing`) and never change its highlights
(the refusal keeps the aliases of its time, `Refusal.channels`; `repeat_fill(frozen=...)`). A
listed unit with the same header names, at every C index, as a confirmed one shows "same
headers as <unit>, which you confirmed as <tags>" with **Confirm the same tags**, and
**Confirm all N like it** when several are listed: one click each through the same
validation and save as the editor (all or none), never automatic.
A unit tagged by hand on the grid is counted too, but is not a source of **Confirm the same
tags** (only a confirmation from the editor or a one-click is).
- A label renamed or removed after a refusal no longer breaks the editor: that cell opens blank
(the unit stays unconfirmed until it is tagged), and a label gone is never offered by a
one-click button. Before, the dropdown raised and every later redraw raised too.
- Refusals and confirmations are kept in the draft file (optional `tag_review`, draft version
still 4, `repeat_review.py`; written only while not empty; read strictly) and restored on
opening, before headers are trusted: after a kernel restart a refused unit is still listed,
untagged and unsaved until that unit is confirmed, and the one-click offer is still made.
- Fast mode decides which refusals stand against the row's current reference unit, as the
list does, and **Confirm all N like it** buttons keep a stable order.
- The review belongs to the draft it was made in (experiment folder and draft file): a notebook
pointed at another folder or output forgets the refusals, confirmations and editor state of
the first and restores the new draft's own (empty without one), before headers are trusted;
one draft's record is never written into another, and switching back restores the first
from its file. A failed save of the review shows a line on the tagging grid and is retried
at the next change.
- **Channel-identity check after the run** (contract §14.1, design
`channel-identity-check-2026-11-21` in MorphoFeature's `claude/restart-doc`): from the image
summaries the intensity checks already make (no pixel read), every imaging unit's channel is
Expand Down
3 changes: 2 additions & 1 deletion docs/contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -1356,7 +1356,7 @@ exclude=, missing_as_singleton=, memory_limit=, progress=)` (`conversion`), the

The answers a user has given while describing an experiment, before any `morphobricks.toml` is published (experiment-v1 §10, §12). It lives in the output folder, is never named `morphobricks.toml`, has no `schema` key, and no consumer reads it as a description (`RunInputs` refuses a draft: `A_DRAFT_REFUSED`). It holds **answers only**: the folders, counts and proposals are derived again from the folder (`draft_from_discovery`) on every resume, so a changed folder is never hidden by an old file. Code: `draft_answers.py` (the answers, `apply_answers`, `description_from_draft`), `draft_format.py` (strict bytes), `run/draft_file.py` (`new_answers`, `save_draft`, `load_draft`, `publish_draft`).

One JSON object, UTF-8, two-space indent, sorted keys, LF, a final newline, no duplicate key and no `NaN`. Every key below is required; a key of this list is the only key allowed at its level.
One JSON object, UTF-8, two-space indent, sorted keys, LF, a final newline, no duplicate key and no `NaN`. Every key below is required except `tag_review`; a key of this list is the only key allowed at its level.

| Key | Value |
|---|---|
Expand All @@ -1373,6 +1373,7 @@ One JSON object, UTF-8, two-space indent, sorted keys, LF, a final newline, no d
| `masks` | Mask ID → `{source_type, representation, source_channels?, display_name?}` (the notebook's *Your masks* screen: one answer per mask type, `representation` `instances`, `binary` or `parent_labels` (*grouped labels*, Spec experiment-v1 §4.5b) with no default and never the mask step's `grouped` kind; the ID is the mask type's folder name when it is a valid mask ID, else typed by the user; `source_channels` are declared channel IDs) |
| `channel_layout` | `null` for a strict per-unit draft, else the user's fast-mode statement (Spec ADR 0006, experiment-v1 §4.3a): `{channels, image_type, confirmed_on}`, the ordered channel IDs of the tagged unit, `channels` or `rgb`, and the day of the statement. A unit's object then also takes `channels_from` (`experiment`: published with this list materialised, or `unit`: tagged on its own); a unit with no answer in a fast-mode draft is drafted `experiment` (a new unit, P15), never `unit`. A marker in a strict draft is refused (`channels_from_without_layout`) |
| `switch_mode` | `true` only when the user switched a published file between fast and strict mode (the notebook's tick); publication refuses a candidate whose mode differs from the published one without it, and a recorded switch that changes nothing (`mode_switch_unconfirmed`, experiment-v1 §10: no reader or reconciliation switches the mode). A published file reopens in its own mode: its `channel_layout` and markers are the draft's answers |
| `tag_review` | Optional, same version 4: written only while it is not empty, so a draft without it is read as before. What *Tag by repeat* refused and the user confirmed (`repeat_review.py`), so a new notebook on the draft keeps the owner rule that a refused unit counts only when that unit is confirmed: `{refused, confirmed}`, two lists. `refused`: `{unit, tags, reference, aliases}`, the unit as `[condition, repeat, technical repeat]`, the row's tags at the click, the row's reference unit (the same three folders, or `null` for a unit whose images disagree) and the aliases (label → header names) as they were when the unit was first refused, which its header is read against. `confirmed`: `{unit, tags, names}`, the tags the user confirmed and the unit's own header names (text or `null`) then. A unit is listed once in each list. Read strictly: `null`, an unknown or missing key, a wrong type or a unit twice is refused, never dropped |

Rules.

Expand Down
21 changes: 21 additions & 0 deletions docs/user_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -365,6 +365,27 @@ editing, or the confirmed ones. An image that cannot be drawn gets a sentence sa
before the pairing: if you pair the `rep2` folders of different conditions with different
biological repeats, every row of that folder (whatever its channel count) says so and
fills nothing; tag those conditions on their own.
From a listed acquisition, **Override this unit (start from the row's tags)** opens that
acquisition's tags with the row's tags filled in. Under each channel the header name is
shown, and an orange box marks where it disagrees, so you change only what is different.
Nothing is saved while you edit; the acquisition counts as tagged only after **Confirm these
tags** (every channel tagged, each label once). If the acquisition's own header or its
tags change later, it is listed again.
A listed acquisition is counted only when you confirm *that* acquisition (**Confirm these
tags**, **Confirm the same tags**, or its own folder tagged on the grid). Confirming one
acquisition never counts another, and fast mode does not tag a listed acquisition from the
header names saved by confirming another: it stays listed with the same contradiction and
the same orange boxes. When a listed acquisition has the same header names, at every
channel, as one you confirmed, it says "same headers as *that one*, which you confirmed as
*tags*" and offers **Confirm the same tags**; when several listed acquisitions have them it
also offers **Confirm all N like it**. Nothing is counted until you click; the buttons save
exactly as **Confirm these tags** does (all N or none).
An acquisition you tagged by hand on the grid counts, but it is not offered as the source of
**Confirm the same tags** for others; only one confirmed with **Confirm these tags** (or by
a one-click button) is. If you rename or remove a label after an acquisition was listed, its
override opens with that channel left blank: choose a label that still exists. What is
listed and what you confirmed is saved with your answers: after you close and reopen the
notebook, a listed acquisition is still listed, and still not counted, until you confirm it.
In fast mode a row offers only the acquisitions that do not fit the layout, and those it
fills are tagged on their own (checked as in the strict mode); unticked, a row is offered
for every repeat folder. On a reopen, a row shows the tags every acquisition of its folder
Expand Down
2 changes: 2 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -188,7 +188,9 @@ baseline = "tools/code_rules_baseline.json"
"morphobricks_preprocessing.notebook_channels_grid" = "ui"
"morphobricks_preprocessing.notebook_channels_matrix" = "ui"
"morphobricks_preprocessing.notebook_channels_repeat" = "ui"
"morphobricks_preprocessing.notebook_channels_override" = "ui"
"morphobricks_preprocessing.repeat_tags" = "pure"
"morphobricks_preprocessing.repeat_review" = "pure"
"morphobricks_preprocessing.notebook_channels_rows" = "ui"
"morphobricks_preprocessing.notebook_channels_labels" = "ui"
"morphobricks_preprocessing.notebook_channels_type" = "ui"
Expand Down
2 changes: 2 additions & 0 deletions src/morphobricks_preprocessing/draft_answers.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
)
from .description_transition import DATE
from .draft import Draft, DraftError, OpenQuestion
from .repeat_review import TagReview

CHANNELS = "channels"
"""The answer "an ordinary channel image": published as an absent ``image_type`` (§4.4b)."""
Expand Down Expand Up @@ -146,6 +147,7 @@ class DraftAnswers:
pairing: Mapping[tuple[str, str], str] = field(default_factory=dict)
channel_layout: LayoutAnswer | None = None # the fast-mode statement; None: strict
switch_mode: bool = False # only the user's switch of a published file's mode (section 10)
tag_review: TagReview | None = None # refused/confirmed units of Tag by repeat (draft only)


class DraftIncomplete(DraftError):
Expand Down
65 changes: 62 additions & 3 deletions src/morphobricks_preprocessing/draft_format.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,10 @@
seven tables of the user's answers (``conditions``, ``technical_repeats``, ``repeats``,
``pairing``, ``units``, ``channels``, ``masks``), the fast-mode statement ``channel_layout``
(``null`` for a strict draft; version 4, Spec ADR 0006) and ``switch_mode`` (true only when the
user switched a published file between fast and strict mode), every key required. Reading never
user switched a published file between fast and strict mode), every key required, and one
optional key, ``tag_review`` (the units *Tag by repeat* refused and the user confirmed,
:mod:`.repeat_review`; written only while not empty, so a draft without it is read as before,
and a ``tag_review`` that is malformed is refused like any other key). Reading never
repairs: bytes that are not UTF-8 JSON, a duplicate key, ``NaN``, an unknown ``version``, a key the
format does not have, a missing key, a value of the wrong type or a date that is not a real
``YYYY-MM-DD`` day are refused with :class:`DraftFileError`, whose ``code`` says which.
Expand Down Expand Up @@ -32,6 +35,7 @@
UnitAnswer,
valid_day,
)
from .repeat_review import Override, RefusedUnit, TagReview, review_to_json
from .run.description_file import ABSENT

DRAFT_FORMAT = "morphobricks-description-draft"
Expand Down Expand Up @@ -59,6 +63,11 @@
"channel_layout",
"switch_mode",
)
OPTIONAL_TOP = ("tag_review",)
"""Top-level keys a file may leave out (a draft written before they existed has none)."""
REVIEW_KEYS = ("refused", "confirmed")
REFUSED_KEYS = ("unit", "tags", "reference", "aliases")
CONFIRMED_KEYS = ("unit", "tags", "names")
CONDITION_KEYS = ("id", "display_name", "reference", "target", "treatment_key")
REPEAT_KEYS = ("id", "confirmed_on", "display_name")
UNIT_KEYS = ("channels", "image_type", "display_name", "rgb_header_override", "channels_from")
Expand Down Expand Up @@ -235,6 +244,53 @@ def _layout(value: object) -> LayoutAnswer | None:
return LayoutAnswer(channels, kind, day)


def _unit_key(value: object, where: str) -> tuple[str, str, str]:
"""Read a unit's folders: condition, source repeat and technical repeat, three texts."""
keys = _texts(value, where)
if len(keys) != 3:
raise _bad(where, "three folder names are expected")
return keys[0], keys[1], keys[2]


def _review(value: object) -> TagReview:
"""Read ``tag_review`` strictly: null, a unit twice or a wrong value is refused."""
top = _table(value, "tag_review", REVIEW_KEYS, REVIEW_KEYS)
refused, confirmed = {}, {}
for number, raw in enumerate(_list(top["refused"], "tag_review.refused")):
where = f"tag_review.refused[{number}]"
item = _table(raw, where, REFUSED_KEYS, REFUSED_KEYS)
key = _unit_key(item["unit"], f"{where}.unit")
if key in refused:
raise _bad(f"{where}.unit", f"unit {'/'.join(key)} is listed twice")
reference = item["reference"]
aliases = {
k: _texts(v, f"{where}.aliases.{k}")
for k, v in _entries(item["aliases"], f"{where}.aliases").items()
}
refused[key] = RefusedUnit(
_texts(item["tags"], f"{where}.tags"),
None if reference is None else _unit_key(reference, f"{where}.reference"),
aliases,
)
for number, raw in enumerate(_list(top["confirmed"], "tag_review.confirmed")):
where = f"tag_review.confirmed[{number}]"
item = _table(raw, where, CONFIRMED_KEYS, CONFIRMED_KEYS)
key = _unit_key(item["unit"], f"{where}.unit")
names = _list(item["names"], f"{where}.names")
if key in confirmed:
raise _bad(f"{where}.unit", f"unit {'/'.join(key)} is listed twice")
if not all(n is None or isinstance(n, str) for n in names):
raise _bad(f"{where}.names", "each entry is a header name (text) or null")
confirmed[key] = Override(_texts(item["tags"], f"{where}.tags"), tuple(names))
return TagReview(refused, confirmed)


def _list(value: object, where: str) -> list:
if not isinstance(value, list):
raise _bad(where, "a list is expected")
return value


def _base(doc: dict) -> str:
base = doc.get("base_sha256")
if not isinstance(base, str) or not (base == ABSENT or SHA256.fullmatch(base)):
Expand All @@ -257,7 +313,7 @@ def _header(doc: object) -> dict:
f"version {version!r} is not supported: this brick reads version {DRAFT_VERSION}",
"unknown_version",
)
_table(doc, "the file", TOP)
_table(doc, "the file", (*TOP, *OPTIONAL_TOP))
if "base_sha256" not in doc:
raise DraftFileError("base_sha256 is missing", BASE_MISSING)
return doc
Expand All @@ -281,7 +337,7 @@ def answers_from_bytes(data: bytes) -> DraftAnswers:
except (UnicodeDecodeError, ValueError, RecursionError) as error:
raise DraftFileError(f"the file is not UTF-8 JSON ({error})") from error
doc = _header(raw)
_table(doc, "the file", TOP, TOP)
_table(doc, "the file", (*TOP, *OPTIONAL_TOP), TOP)
base = _base(doc)
commit = _text(doc["spec_commit"], "spec_commit")
if not COMMIT.fullmatch(commit):
Expand All @@ -304,6 +360,7 @@ def answers_from_bytes(data: bytes) -> DraftAnswers:
_pairing(doc["pairing"]),
_layout(doc["channel_layout"]),
_flag(doc["switch_mode"]),
_review(doc["tag_review"]) if "tag_review" in doc else None,
)


Expand Down Expand Up @@ -377,6 +434,8 @@ def answers_to_bytes(answers: DraftAnswers) -> bytes:
},
"switch_mode": answers.switch_mode,
}
if answers.tag_review is not None and not answers.tag_review.empty:
doc["tag_review"] = review_to_json(answers.tag_review)
try:
return (json.dumps(doc, ensure_ascii=False, indent=2) + "\n").encode("utf-8")
except UnicodeEncodeError as error:
Expand Down
Loading
Loading