Repository navigation
Add validation to MultiRecord.wrsamp for segment cohesion - #559
Conversation
1257006 to
51faaf0
Compare
There was a problem hiding this comment.
Thanks Brian, checking all the segments before writing anything is a good change. On main, a segment whose length doesn't match seg_len gets written without any error, and a bad segment partway through leaves half-written files. This PR fixes both.
I tried a few cases locally, and two writes that work on main are now rejected:
- The expanded check runs even when
expanded=False. A segment with a validd_signaland an out-of-datee_d_signalnow fails, even though onlyd_signalis written. The check also falls back toe_p_signal, which is never written. Could we passexpandedinto_check_segment_cohesionand only checke_d_signalwhen it'sTrue? samps_per_frame=[None]is rejected.Record.wrsamp(expanded=True)treats a missing value as 1 (check_sig_cohesionin_signal.py), so a segment that writes fine on its own now fails inside a multi-segment record. It would be good to treat it the same way here.
Both of these come from having the expanded-length rule in two places. Another option is to drop the new block and run each segment's existing checks (its header field checks plus check_sig_cohesion) in a first pass, then write everything in a second pass. That keeps the "check before writing" behaviour with a single rule.
A few smaller things:
- Gap segments (
None, e.g. from reading a variable-layout record withsampto) now fail with'NoneType' object has no attribute 'fs'. That isn't worse than before, but a clear error saying gap segments can't be written yet would help. - There are no tests for
MultiRecord.wrsampyet. It would be good to add:- a round trip of
041s - a
seg_lenmismatch - a bad segment after a good one, checking that no files are written
- the two cases above that should be accepted
- a round trip of
- Could you add a changelog entry to
docs/? - A few of the new error message lines are over 88 characters.
51faaf0 to
05e6d7e
Compare
05e6d7e to
06bd1a3
Compare
|
Thanks for suggesting a first pass that runs each segment’s existing header and signal checks before writing. I didn’t take that approach in this PR because those checks are currently part of I’ve updated the PR to:
Other validators may check additional properties or parameters later in the write process, so this doesn’t catch every possible validation error before files are written. If we find a specific case where a later validation causes partial output, we can add the relevant pre-write check to |
There was a problem hiding this comment.
Thanks, this looks good! Two small things:
- I don't see the changelog fragment. Please could you add it?
- Because
_check_segment_cohesion()now runs before the header field checks, aseg_lenthat is shorter thann_seggivesIndexError: list index out of range. On main it givesValueError: The length of field: seg_len does not match field n_seg. Perhaps we could add a length check next to then_segone?:
if len(self.seg_len) != self.n_seg:
raise ValueError("Length of seg_len must match the 'n_seg' field")
|
Good catch @tompollard . This has been updated and a test has been added. |
|
Thanks, this looks great! Happy to merge. |
As mentioned in #557, we should add segment validation when writing multi-segment records. This PR adds pre-write validation to
MultiRecord.wrsamp()by activating the previously unused_check_segment_cohesion()method, which is now called before any files are written.The validation includes checks for basic segment cohesion (matching
n_seg, sampling frequencies, and segment lengths) as well as new validation for expanded signals. For segments withe_d_signalore_p_signal, the method now verifies thatsamps_per_frameis properly set for all channels and that expanded signal lengths match expected values (samps_per_frame[ch] * sig_len). This provides clear, descriptive error messages when segments are misconfigured, preventing invalid multi-segment records from being written.