Repository navigation
feat(query,store): "ab", "vor", "nach", since/until/before/after, from … to and between … and are windows, a month at a window's end includes it, a long window gives its cyclic dimensions no weight, and "ſeptember" starts a month name (#463) - #480
Merged
Merged
Conversation
…luded, with the questions of #463's remaining points Step 6a of HandleQuery moves unchanged into applyTemporalGravity, so the ranking test runs the stage the handler runs instead of a copy of its linear path. The seeded blocks get their dimension rows through store.RefreshTemporal (content anchors only, so the order does not depend on the day the test runs). New questions for the month end of a window, the rules for ab, vor, nach, since, until, before, after, from … to, between/zwischen and the spaced hyphen, "ab Projektstart bis …", and windows longer than the cycle of their cyclic dimension (keepsOrder). They fail on this commit; the following commits of the wave make them pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…" does, and the read side reads a month behind quotation marks, brackets and no-break spaces The write side's month forms took the start of a month name from \b, which knows ASCII only: the long s, which (?i) lets spell the first letter of "september", counted as a non-word rune, so "ſeptember 2026" yielded no date at the start of a text or behind a space, a quotation mark, a bracket or a no-break space, while "xſeptember 2026" yielded September 2026. A month name now starts where no ASCII word character stands before it, whatever rune spells its first letter; for every other month that is the \b it was. The change has its own class in the differential test against the frozen reference (#463/1), and the reference of the current extraction carries it. The read side reads the forms the write side reads around a month name: quotation marks and brackets around the phrase after "seit", "bis" or "von … bis" ("seit „März 2024“"), and the two no-break spaces the write side takes for date spaces, also between a preposition and its date. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ate outside a window counts by its day A window's end named by a month was that month's 1st: "von März 2024 bis April 2024" ended on 2024-04-01 and "bis März 2024" on 2024-03-01, so the month the question names lay outside its window. A phrase now reports whether it names a whole month (phraseEdge), and a month at a window's end stands for its last day — also when two fixed edges in the wrong order swap and when the end is counted again from the start. "bis" decides between the time up to its end and a deadline by the month's 1st, as before. A start named by a month stays its 1st. The linear gravity measured a date after a window from the end day's 00:00 and a date before it from the start's 00:00 by the hour: a time an hour before the start was half a day away, a time an hour after the end day more than a day. A window's edges are days, so a date outside it now counts by its day on both sides; a date at 00:00 lies as far away as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fore", "after", "from … to", "between … and" and two day dates with a spaced hyphen are windows The rules read "ab X" as the time from X on and "nach X" as the time from the day after X on, both open towards the future (DirFrom), and "vor X" as the time up to the day before X (DirUntil); a month as X is the whole month. "since" and "until" are "seit" and "bis", "after" and "before" are "nach" and "vor", "from X to Y" (also "until", "through") is "von X bis Y", and "ab X bis Y" with a date X is that window too. Two day dates make a window with "zwischen"/"between" before them and "und"/"and" between them, and with a hyphen between spaces. Before, these went to the LLM fallback, gave two single days, the single day of the date, or nothing. "ab", "vor", "nach", "before" and "after" are words of their own far more often than words before a date, so their rules take a phrase only when it starts with a date spelt as such — a day date, a month, a weekday or a day keyword, not a word the keyword matching forgives as a typo — and of several such words the first whose phrase is one. A negative corpus of 54 questions holds that none of them gets a window. "ab Projektstart bis 05.03.2026" stays the time up to that day, pinned by a test. The linear gravity reads a DirFrom date as the window from it on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dimension no weight
The dimension weights of windows keep a cyclic share anchored on one edge
("seit"/"bis": month 0.2, "von … bis": weekday 0.4). The ranking test, now
running the whole Step 6a, measured what that does to a window longer than
the cycle: "seit März 2025" ranked the Marches of both years ahead of every
other block in the window, "von März 2024 bis Mai 2025" the Fridays, and
"bis März 2015", whose window no content date reaches, reordered 196 of 200
blocks by their month alone. Such a window holds every phase of the cycle,
so the phase of its edge says nothing about where in it a block lies.
effectiveDimWeights drops, for the first date of a question, every cyclic
dimension whose cycle (rrf.DimensionCycle) its window is longer than — an
open window is longer than every cycle — and gives its weight to the linear
share, so the boost budget stays what the rules gave. Step 6a and
activated_dim_weights both read it. A single day and a window within the
cycle keep the weights the rules gave, the same map.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…a past end as the time up to it, as the rules read it Rule 3 of the LLM fallback's prompt asked for every "bis"/"until"/"by" as a range from today, so a question the rules leave to the model got the window from today back to a past end that the rules stopped yielding in #469. The rule now asks for an end before today as the time up to it (dir "until", no end), a later end as the deadline from today, and a month as the end for its last day; the answer format offers "until". The prompt is a new version (2026-11-21) with its body pin moved along. The answer parsing takes either form; the query consumer reads "until" as the window up to the date and an old-form inverted window as a point, as before. No model ran: what the model makes of the rule is for the evals. The architecture doc describes the dimension weights of long windows and the prompt rule. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e naming its own day, a time unit before them makes a distance, and "seit X bis Y" is a window
From the review of the wave. The open-edge rules guessed the year of a bare
month and the week of a bare weekday from the tense, which reads the German
"Was" as the English "was": "Was muss vor März erledigt werden?" became the
time up to last February, and a month that is a word of its own opened a
window ("nach August", "after may I ask"). These words now take only a date
that names its own day — a day date, a month with its year, a day keyword,
a weekday or month after "nächsten"/"letzten"; a bare weekday or month stays
what it was before the wave. A count of a time unit before "vor", "nach",
"before" or "after" ("zwei Tage vor dem 15.03.2024") makes the word a
distance: the date stays its day. "seit X bis Y" and "since X until Y" are
the window from X to Y instead of the time since X with the end dropped.
Tests cover two fixed edges that swap with a month on one side only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… they get Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 52 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
… the open edges take no year first
From round 1 of the PR review. The phrase form "year, then month name" read
"Logs after 2025 may be deleted" as June 2025 on, "Tests before 2024 may
fail" as up to April 2024 and "Requests since 2024 may time out" as the time
since May 2024. A month name that is a word of its own (the modal "may", the
verb "march", the adjective "august") after a year is no month now, for every
rule; the open edges (ab, vor, nach, before, after) take no phrase that
starts with a year at all, a form without a known use. Month before year
("May 2024") reads as before.
The test file also carries the tests of the round's next points, which the
following commits make pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r "august" is no month
The sweep of round 1's first point: the English kin of "seit" and "bis"
that this branch added read a bare "may", "march" or "august" as a month
("Wait until may I ask a question" became the deadline up to May 2027,
"since march the troops" the time since March). After "since", "until" and
"from" such a bare name is the modal, the verb or the adjective far more
often than the month, so those rules take it for none; with its year it is
one ("since May 2024"), the other month names stay months ("since June"),
and the German rules keep "seit August" and "bis August".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fore "vor", "nach", "before" or "after" keeps the day From round 1 of the PR review: "kurz vor dem 24.12.2026", "the night before 2024-12-24", "am Morgen nach dem 01.01.2026", "das Wochenende vor dem 15.03.2026" and "ein Quartal nach dem 01.01.2026" opened a window towards the past or the future in which every older or newer block earned the full gravity; the question names a time near that day. A word of nearness (kurz, knapp, direkt, unmittelbar, gleich, shortly, right, just, immediately, soon) or a time unit or time of day (Sekunde … Jahr, Nacht, Abend, Morgen, Mittag, Wochenende, Quartal and their English kin) right before the word now keeps the day the question names, as before the branch; only the word right before counts. The architecture doc names round 1's first two points. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From round 1 of the PR review: "Was muss ich vor dem 01.12.2026 erledigen?" was the time up to 2026-11-30 open towards the past, so blocks from 2024 earned the gravity of the days the question asks about, while "bis zum 30.11.2026" is the deadline from today. "vor X" and "before X" with X after today are now the window from today to the day before X, the distinction "bis" makes at its end; with X today or before, the time up to the day before X as before. "nach X" with a past X stays open towards the future: for contents up to today that is the window up to today, and a block dated after today answers it as well. The ranking test has a question for the deadline (R-precision 0/2 on the previous commit, 2/2 now). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… today, and an unreadable "from" keeps its "until"
From round 1 of the PR review: "bis"/"until" stepped aside whenever "from …
to/until" matched, also when the "from" phrase was no date, so "Which
tickets stay open from now until Friday?" and "Move the tickets from backlog
to done until Friday" were the point Friday. A range form whose start is
"jetzt" or "now" now starts today; a "from … until" whose start the rules
cannot read leaves "until" its window ("from the start until 2026-03-05" is
the time up to that day). "von"/"vom" with an unreadable start keeps the day
date the day it names, as #469 pinned. A test also pins that "seit" before
"bis" keeps the typo tolerance "seit" has alone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… cycle, are pinned The 158 questions an independent reviewer of the pull request ran against the new rules become test data with the answers after round 1: words of the new rules without a date behind them, distances before them, modal verbs after a year, and the forms that are windows on purpose. Every question that is no window at the reference Saturday is none on any weekday as today. On the head the reviewer read, 83 checks of the test fail. The day of the month cycles in a calendar month, not in 31 days (a window from February 1 to March 1 outlasts it); no producer puts the monthday on a window today, so the cycle is pinned at effectiveDimWeights. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Write side (
store.ExtractDates). A month name starts where no ASCII letter, digit or underscore stands before it, whatever rune spells its first letter. The long s — the one rune beyond ASCII that(?i)lets spell the first letter of a month name, the s ofseptember— counted for the ASCII\bas a non-word rune, soſeptember 2026yielded no date at the start of a text or behind a space, a quotation mark, a bracket or a no-break space, whilexſeptember 2026yielded September 2026. For every other month the check is the\bit was.Read side (
llm.NormalizeTemporalRules).seit „März 2024“) and with U+00A0 / U+202F wherever a space stands, also between the preposition and its date, read as the write side reads them.von März 2024 bis April 2024ends on 2024-04-30,bis März 2024on 2024-03-31 (also after a swap of fixed edges and when the end is counted again from the start). A month at the start stays its 1st.ab X(from X on, open towards the future, newDirfrom),nach X/after X(from the day after X on),vor X/before X(up to the day before X,Diruntil),since=seit,until=bis,from X to/until/through Y=von X bis Y,ab X bis Ywith a date X,seit X bis Y/since X until Y, two day dates withzwischen/between…und/and, and two day dates with a hyphen between spaces.vor X/before Xwith X after today is the deadline from today to the day before X.ab,vor,nach,before,aftertake only a phrase that starts with a date naming its own day (day date, month with its year, day keyword, weekday or month afternächsten/letzten; no typo-tolerant keyword match, no bare weekday or month, no year first), not after a time unit, a time of day or a word of nearness right before them (zwei Tage vor dem 15.03.2024,am Abend vor dem 24.12.2026,kurz vor dem 24.12.2026stay the day), and of several such words the first whose phrase is one. A year beforemay,marchoraugustis no month for any rule (Logs after 2025 may be deleted), and aftersince,untilandfroma baremay,marchoraugustis no month (Wait until may I ask). A range form starts today when its start isjetzt/now(from now until Friday), and afromwhose start the rules cannot read keeps itsuntil.ab Projektstart bis 05.03.2026stays the time up to that day (pinned).Query stage (
handler/query.go,rrf/gravity.go).applyTemporalGravity, so the ranking test runs the stage the handler runs, cyclic path included.effectiveDimWeights: a window longer than the cycle of a cyclic dimension (rrf.DimensionCycle: week for weekday, month for monthday, day for daily, year for month/quarter/week/seasonal), and every open window, gives that dimension no weight; its weight goes to the linear share. Step 6a andactivated_dim_weightsboth read it. A single day and a window within the cycle keep the rules' map.linearGravityParamsreadsDirfromas the window from the date on.LLM fallback prompt. Rule 3 asks for
bis/untilwith an end before today as the time up to it (diruntil, no end), a later end as a deadline from today, a month as the end for its last day; the format offersuntil. Prompt version 2026-11-21, body pin moved along.Why
Closes #463 (point 5, and the points of the last comment's "Found on the way" and "Left as they are").
Measured before the cyclic change: the ranking test, running the whole Step 6a, showed that a window longer than its dimension's cycle ranked the start's phase first in every year (
seit März 2025: the Marches of both years ahead of every other block in the window;von März 2024 bis Mai 2025: the Fridays) and thatbis März 2015, which no content date reaches, reordered 196 of 200 blocks by month alone.Rejected alternatives: for the long-s fix, a Unicode word boundary on both sides (it would also change every month directly behind a non-ASCII letter such as
ümärz 2026, which matched before and is outside the issue); for the window edges, measuring after the window from the end of the end day by the hour (a date-only block on the day after the end then sits at distance 0 and takes the full boost — a mutant of exactly this turns the ranking test red); for long windows, dropping the cyclic weight without moving it to the linear share (a month windowvon März 2024 bis April 2024would then boost by 18 % while the same window written as day dates boosts by 30 %); forab Xwith a past X, the window up to today likeseit(abnames only where the time begins; a block dated after today answers "was gilt ab März 2024" as well).Behaviour changes (intended)
ExtractDates: contents with a long s as the first letter ofseptemberbefore a year: behind no ASCII word character they now yield the month, behind one (xſeptember 2026) no longer. Blocks get these dates on their next content write; the backfill of existing blocks is separate (store: ExtractDates follow-ups of #437 — cost on number-heavy text, the long s, and what the wider year window also matches #451 point 4).bis März 2015→ 2015-03-31, FTS expansion2015-03-31 OR 2015-03).Seit Montag bis heute(anyseit X bis Y) now gets the weights ofvon … bis, weekday 0.4, whicheffectiveDimWeightskeeps for a window within a week (before: month 0.2 via theseitrule).nach 2024-03-152024-03-15 OR 2024-03→2024-03-16 OR 2024-03;nach dem 31.03.2024→2024-04-01 OR 2024-04(the month term moves);vor März 2024→2024-02-29 OR 2024-02;vor dem 01.12.2026(deadline) →2026-11-21 OR 2026-11 OR 2026-11-30. New forvor/nach/before/afterwith a day date; with a month the base had no rule (LLM fallback). Its effect on the full-text arm is not measured: the ranking test has no FTS arm.seit X bis Y/since X until Yis the window from X to Y with the weights ofvon … bis(before: the time since X up to today, the end dropped;seit 2026-03-01 bis heutekeeps its window, its reported weights become{linear: 1}).activated_dim_weightsdrops cyclic dimensions for long and open windows (e.g.seit März 2025:{linear: 0.8, month: 0.2}→{linear: 1}); windows within the cycle and single days report as before.Consumers of the changed contracts
store.ExtractDates(long s)blocks.go,upsert_batch.go,block_edits.go,sources.go,supersedes_declare.go,temporal.go),dream/validate_temporal.go,ingest/obsidian.go(ExtractDateStrings)#463/1, frozen-reference tests, fuzz,TestExtractDates_LongS; integration runs of store, dream, ingestllm.NormalizeTemporalRuleshandler/query.go(Step 6a, FTS expansion, synthesis date context)TemporalDate.Dirfrom(new)linearGravityParams,effectiveDimWeights,TemporalToFTSExpansion(start only),Synthesize(rendersref = date)TestLinearGravityParams,TestEffectiveDimWeights, ranking testrrf.ComputeGravitywindow distancerrf.ApplyGravityBoost← Step 6aTestComputeGravity_WindowEdgesCountWholeDays, window tests, ranking testactivated_dim_weights/api/queryresponse,eval-cyclic.shdim_weight assertTestActivatedDimWeights,TestEffectiveDimWeightsllm.temporalPromptTemplateNormalizeTemporal(fallback), goldbench temporal-query axisinternal/prompts, prompt-rule test, parse tests of both answer formsCost
ExtractDates: one byte comparison replaces the\bcheck at a month match's start; benchmark below. Rules: one more regular-expression scan per question (FindAllStringSubmatchof the five words) plus two cheap ones; questions are short.effectiveDimWeights: a pass over at most eight weight keys per query. No schema change, no new query.Verification
Red before, green after (each red run on the code without the change, logs kept):
TestExtractDates_LongS(13 contents wrong on the base:ſeptember 2026behind a space, quotation mark, bracket, no-break space yielded nothing;xſeptember 2026,2025ſeptember 2026yielded September), the class witness of#463/1, the reference comparison of the seed corpus andTestScanDates_EveryMonthSpellingMatches.TestMonthForms_ReadWhereTheWriteSideReadsThem— 1,738 of the read-side checks failed before (quotation marks, brackets, no-break space after the preposition); the write-side checks of the same test passed on both.TestRules_AMonthAtTheEndIncludesTheMonthand the updated window tests (17 checks red before).TestComputeGravity_WindowEdgesCountWholeDays(an hour before the start 2.8272 vs an hour after the end day 0.9292 before; 1.0000 both after).TestRules_OpenWindowsBeforeADate,TestRules_WindowsBetweenTwoDates(43 cases red before); the review round's cases (bare month/weekday, time unit,seit … bis) red on the commit before them.TestTemporalPrompt_BisWithAPastEndIsTheTimeUpToItred with the old template; theinternal/promptsbody pin turned red on the edit and moved with the new version.ab Projektstart bis 05.03.2026, and the parsing of both LLM answer forms; mutants show they bite.Ranking test (
TestQueryTemporalRanking, 200 blocks, 36 questions, whole Step 6a), before → after:vor März 20240/12 → 12/12,ab März 202617/53 → 53/53,von Februar 2025 bis April 202512/17 → 17/17).bis März 2015moved 196 of 200 positions → 0.Other runs:
FuzzExtractDates(write side against the reference with the classes),FuzzDayDatesAgreeWithTheWriteSide(read against write side),FuzzNormalizeTemporalRules— all pass.-count=5with and without-race: pass.go test -tags=integration -p 1on util, llm, rrf, prompts, dream, ingest, store, handler: pass (after round 1 again on handler, llm, prompts: pass) (store'sTestFullTextVectorsUpdateCostfailed once under host load 30+ in a run that also hit the 10-minute default timeout, and passed in the full rerun with-timeout 28m).ExtractDates(few/manyat 2/20/50/400 KB), base and head interleaved, 8 rounds each, medians under host load ~15: within −22 % … +3 % of the base (e.g. 20 KB: 24.7 → 23.7 µs few, 208.0 → 200.2 µs many); the absolute values are about twice the idle numbers of perf(store): ExtractDates costs 8 ms per 20 KB of content, most of it in two case-insensitive month patterns #434 for both sides alike.deadcode.sh/deadcode.sh testonly: ok.bash go/gates.sh wave --finalafter round 1: rc 0 (build, vet, lint, doccheck, short, census-tests, race, integration 6 of 6 packages, deadcode, deadcode-testonly). The run before round 1 had stopped at one load-sensitive latency gate ininternal/handler(TestModelAuthSemaphore_Integration), which passed alone and passes in this run.Round 1 of the review (an independent reviewer ran 158 questions and its own mutants against the head): every false positive it executed (a modal
mayafter a year; a nearness word, time unit or time of day beforevor/nach;vora day to come as an open window;from now until Fridayas the point Friday) is fixed, each reproduced red on the head first. Its 158 questions are test data now (testdata/temporal_corpus_463.tsv, 83 checks red on that head, every non-window answer checked on all seven weekdays); 12 more mutants (47 in all) each turn a test red, the two that survived its run among them (unit word anchored, GermanWoche) and the monthday cycle. The ranking test has a question for the deadline (vor dem 01.12.2026: R-precision 0/2 on that head, 2/2 now); single-day questions unchanged. Fuzz 65 s per target after the round: pass.A fresh reviewer agent read the diff; what it found on the open edges (a bare month after
vortook its year from a tense guess that reads German "Was" as English "was"; month names that are words; a distance beforevor/nach;since … until) is fixed with tests.Not verified
… ab Mitte Mai 2026, now the window from 2026-05-15).septemberis not counted; the repository's own text files outside the tests of this change carry none.🤖 Generated with Claude Code