Skip to content

DAQ light safety fixes (main stays 0.2.7-DEV) - #94

Merged
kalidke merged 14 commits into
mainfrom
fixes-0.2.7
Nov 4, 2026
Merged

kalidke merged 14 commits into
mainfrom
fixes-0.2.7

Conversation

@kalidke

@kalidke kalidke commented Nov 2, 2026 •

Copy link
Copy Markdown
Member

DAQ light safety fixes into main. main stays 0.2.7-DEV: this is not a release, and TagOnMerge skips a -DEV merge. The entries sit under [Unreleased] in the CHANGELOG. When to tag 0.2.7 is a separate decision (Keith, 2026-11-03: "we accumulate in the dev version, then decide when to make a tag").

Why

A review on 2026-11-02 found that the DAQ voltage lights on the adaptive microscope can report a laser off while it emits, and light the 561 nm laser through setpower while the driver says off. This PR fixes both, and every other confirmed item from that review, plus a stale stage position found in real files (Keith's rule, relayed 2026-11-03: a bug confirmed with a valid input is fixed whether or not lab code calls it).

No instrument loads this code. The adaptive microscope pins v0.3.0-rc4, and no release candidate follows it (Keith, 2026-11-02: release candidates are for Ship of Tools only). It gets these fixes by moving to a released main version. That move needs MicroscopeAdapt to drive its 642 nm laser through the modulator field and to handle the new errors. Its 642 MOD IN is on a Triggerscope DAC, so the move also waits for a Triggerscope modulator (branch triggerscope-modulator) and the Triggerscope fixes (#92), unless the cable goes back to an NI output.

What changes

CrystaLaser, VortranLaser, DaqTrLight

  • Write first, record after. A failed write or a missing channel throws.
  • Any failed off (initialize, light_off, shutdown) records is_on = true: off cannot be claimed.
  • On the Crysta and the DaqTrLight, level and on/off share one line. is_on reports, and a new line_enabled decides whether a level goes to the line: it is set only by a successful light_on. setpower otherwise stages the level for the next light_on. Without a staged level, light_on writes 1.0 V as it always did. shutdown and initialize clear a staged level.
  • The requested level lives in level_requested, never in properties.power, because MicroscopeAdapt writes a percent there. It reads NaN until something is requested. export_state reports it in volts, with power_requested added.
  • setpower with NaN or an out-of-range value throws ArgumentError. One lock per light serializes each write and record.
  • initialize writes 0 V on the Crysta and the DaqTrLight. New keywords daq, channel, ao_channel, do_channel.

lights_off(lights) is new and exported: it attempts every light, then throws one LightsOffError naming each failure.

Light panels work in the light's own unit and range, show a refused command, and poll the driver every 0.5 s, so a panel follows a light changed from another window. Opening a panel issues no command, and a poll never commands.

Also: PIStage moves throw when PI_MOV refuses and do not record the target, with the stage panel catching and restoring; stage step boxes refuse a non-positive or non-finite step; the camera panel menus work; ThorCamCSCCamera and MLSLM gain export_state (attributes only); roi_origin is recorded.

Drivers and saving:

  • DCAM4Camera: a failed exposure, width or height read-back throws, in the normal constructor too, instead of taking 0.
  • ThorCamCSCCamera: a poll timeout returns nothing from getlastframe and throws in capture and sequences, never an all-zero frame. The timeout follows the exposure unless set explicitly (constructor keyword, setter or field); a refused timeout write keeps the accepted value.
  • save_h5: copies what it writes before returning; writes in place as before (overwrite unchanged); every failure in the task logs @error once and rethrows, so wait throws; no partial file after a failure (a file the save created is removed, an existing one is truncated to zero bytes in place); the path is anchored once at call time.
  • MCLStage: negative Madlib returns throw and keep the cached positions and ranges.
  • Stage export_state (PIStage, MCLStage, N472, MCS2Stage, simulated stages) reads the position at save time when connected and adds position_fresh; a failed read warns and exports the last-read values. A read never moves a stage.
  • Camera panel buttons log a command that throws (a CSC capture or sequence that timed out) instead of passing it into the window's event loop.

Callers of the CSC change (each program's session, from its code): MicroscopeControl's live view already skips nothing; MicroscopeAdapt reaches a CSC only from four legacy calibration tools on a side branch (not run on quickbeam); MicroscopeSeqSR has no CSC; MicroscopeMinFlux's main has no MicroscopeControl use, and student scripts on a side branch would now raise instead of using a zero frame. Not breaking: a zero frame on a timeout was fabricated data.

Memory during a save (the snapshot doubles it while writing): quickbeam 127.1 GB; the largest save_h5 file on record is one 43.0 GB stack (100,000 frames of 840 x 256, 2025-07), 86.0 GB at the peak plus 12.3 GB for the running program, which fits; today's sequences are 5 to 400 frames. The longest full-sensor stack that fits falls from about 10,800 to about 5,400 frames. The SeqSR PC (31.9 GB) and MINFLUX save no images through save_h5.

Windows: h5_saving, csc_failures, stage_exports and driver_readback on hosted windows-latest at this head, Julia 1.13.3: 768 pass, 0 fail, 3 skipped (Linux-only failure injection; a cross-platform failed-save test passed). Branch wincheck (never merged) holds the workflow. The Windows runs found a CSC test reading past its fake frame, a POSIX-only path expectation, and one real fault: on Windows ispath is true for a dangling symlink, so a failed save through it emptied the new target; the save now follows link text to find the file it lands on.

Callers

  • light_off, setpower and initialize now throw. Teardown moves to lights_off.
  • setpower(2.5); light_on ended at 1.0 V and now ends at 2.5 V. A setpower with no light_on moved the line and now moves nothing.
  • Default properties are V and the voltage range, not mW and 0-100. Saved power, min_power and max_power are volts.

Evidence

A metadata harvest of 2,350 real HDF5 files (1,041 with driver state on two instruments), the NAS logs, a survey of MicroscopeAdapt's call sites, and DAQmx.jl's source. No hardware was touched.

Item Checked against Result Decision
M1 failed switch-off recorded as off DAQmx.jl source, logs A real failed write throws into the catch that swallowed it. No real failed write in the logs. Fixed (safety)
M2 setpower lights the 561 while off caller survey Reachable through adaptive overrides on an off 561 and the panel slider; not the default path. Fixed (safety)
M3 saved power wrong real files In all 6 files saved with the lamp on, its power reads 0.0 mW while MicroscopeAdapt recorded 15. Fixed
S1 refused PI move 244 recent files, caller survey Not seen, but weak evidence: the servo is off during sequences, so refusal is a normal state. Fixed
S3 camera menus throw real code in a real window Always. Fixed
S4 stage step boxes reading only NaN reaching the Z piezo was never tested; the fix is six lines. Fixed
S6 no CSC or SLM state real files Absent from all 562 adaptive microscope files. The package can now export them; those files change only when MicroscopeAdapt adds them. Fixed in the package
S7 ROI origin real files DCAM4 confirmed by 21 full-frame files; CSC is code only. Documented
S8 channel by index real files No wrong-line event. Additive. Added
M4 failed exposure read-back 1,041 files, logs; fake SDK returning DCAMERR_TIMEOUT Not in saved files; reproduced with a documented SDK return. Fixed
M5 CSC zero frames in a sequence real files, caller survey; SDK success with no frame after the wait No CSC data in any file; reproduced with a real timeout. Fixed
S2 save_h5 562 files; caller mutation, a real write failure (file-size limit) Not in saved files; reproduced. Fixed
S5 MCL error codes as positions 3,372 saved values; documented Madlib error returns Not in saved files; reproduced. Fixed
Stale saved stage position 318 real files Seen: PI stage files from 2025-05 to 2026-05 record 12.5, 12.5 throughout; MCL z gaps in 13 of 244 recent files. Fixed

The first four are real code defects that have not appeared in saved files, which is a different finding from not being real. Each is reproduced by a test on a valid input that fails at c9d66a1 and passes here.

Tests: every three-step failure walk of the commands, on all three lights, from a dark line and a line left high: 1,500,282 walks show no light emitting while the driver says off, and no emission rising except on light_on. The suite keeps every two-step walk and a fixed three-step sample. The fake DAQ never reaches DAQmx. Reviews: Codex over the diff, then main's review of record (2 must fix, 7 should fix, all taken), then Codex over the fixes: merge as is. The five driver and saving fixes went through seven Codex review rounds; save_h5 first used a temporary file and a rename, and each round found another way a replacement differs from an in-place write (file modes, symlinks, sticky directories, owners and ACLs, hard links), so it writes in place as before.

Tradeoffs

  • Nothing here ran on hardware. The check owed at the repin, on quickbeam with a power meter: setpower while off gives no light on the 561 and the lamp; setpower(2.5); light_on reads the 2.5 V level; light_off with the DAQ unplugged throws and the panel says on; restarting with the 561 left lit goes dark at initialize; lights_off with one line unplugged darkens the other lights.
  • A failed deletetask is not retained. It is loud and is_on stays true. DAQmx.jl's finalizer retries the clear at garbage collection, so the operator's steps are GC.gc(), light_off, then restart Julia, where initialize writes 0. Whether the channel stays reserved at all is untested on hardware.
  • initialize on a light with no DAQ channel now throws, so a system's startup stops on a host with no DAQ.
  • A failed live setpower darkens the light, so a transient error during a run ends that light's emission on purpose.
  • The panel slider can show a higher level than the line after a toggle, as it does today, because light_on after light_off writes 1.0 V.

🤖 Generated with Claude Code

kalidke and others added 9 commits November 2, 2026 12:48
… add lights_off

CrystaLaser, VortranLaser and DaqTrLight no longer record a light off when the switch-off failed, no longer
light through setpower while recorded off, and save the requested level in volts. Failed writes throw; lights_off
switches every light off and then raises every failure. Named-channel and daq keywords are additive. No hardware run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n stated

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s keep 1.0 V; panel and export fixes

Review round. light_on after light_off stays at 1.0 V unless setpower ran while off, so no working sequence
changes. A textbox entry commands once even when the slider does not move. export_state reads the DAQ outside the
lock. gui(::LightSource) returns its Screen again. Caller-supplied properties are left alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… section, known issues

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d enable; panels poll the driver

Main review of PR 94. is_on reports; line_enabled gates the line, so neither fault shows on any failure walk. Panels
follow a light changed from another window. PIStage moves throw when refused, stage step boxes refuse a non-positive
or non-finite step, and a fresh light exports NaN. CHANGELOG gains a Callers list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…I Home is guarded

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ailed second light_on

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… in this PR

Keith: we accumulate in the dev version, then decide when to make a tag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kalidke kalidke changed the title MicroscopeControl 0.2.7: DAQ light safety fixes DAQ light safety fixes (main stays 0.2.7-DEV) Nov 3, 2026
kalidke and others added 5 commits November 3, 2026 14:30
…uts, save_h5, MCL returns, stale stage position

Every confirmed bug gets fixed (Keith, relayed 2026-11-03), so the known issues
PR 94 carried go into it:
- DCAM4Camera: failed exposure, width and height read-backs throw, in the normal
  constructor too, instead of taking 0.
- ThorCamCSCCamera: a poll timeout returns nothing from getlastframe and throws in
  capture and sequence, never an all-zero frame; the timeout follows the exposure
  unless set explicitly (keyword, setter or field); a refused timeout write keeps
  the accepted value.
- save_h5: snapshots what it writes before returning; writes in place as before;
  every task failure logs once and rethrows; no partial file after a failure (a
  file it created is removed, an existing one truncated to zero bytes); the path
  is anchored once at call time.
- MCLStage: negative Madlib returns throw and keep cached positions and ranges.
- Stage export_state reads the position at save time when connected and adds
  position_fresh; a failed read warns and exports the last-read values.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A ThorCamCSCCamera capture or sequence whose frame wait times out now throws
(PR 94); the panel's Capture and Sequence buttons log it and the window keeps
working. Admiral's caller check, 2026-11-03.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…est assumed POSIX ..

- csc_failures.jl: the refused-timeout testset captured from a default
  1440 x 1080 camera while the fake SDK holds an 8 x 6 frame, so the copy read
  past the buffer (Windows: access violation; Linux read it silently). The
  camera now states the fake's sensor.
- h5_saving.jl: Windows resolves .. in the path text, so missing\..\frames.h5
  anchors to the directory; the expectation follows the platform.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An attribute value HDF5 cannot store fails the save after the file exists, so the
removal of a new file, the truncation of an existing one, and the removal of a
symlink's new target are tested on Windows too, where the file-size-limit tests
are skipped (admiral, 2026-11-03).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…spath

On Windows ispath is true for a symlink whose target does not exist yet, so a
failed save through such a link emptied the new target instead of removing it
(found by the hosted Windows run). h5_landing_path follows each link's text from
the link's directory; whether that file existed decides removal or truncation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kalidke
kalidke merged commit 5075b9a into main Nov 4, 2026
5 checks passed
@kalidke
kalidke deleted the fixes-0.2.7 branch November 4, 2026 03:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant