Skip to content

merge: migrate main changes into v3 - #693

Open
korikuzma wants to merge 6 commits into
v3from
merge-main-changes
Open

korikuzma wants to merge 6 commits into
v3from
merge-main-changes

Conversation

@korikuzma

Copy link
Copy Markdown
Contributor

Will do a merge commit for this

korikuzma and others added 6 commits November 3, 2026 07:34
close #623

* `translate_to` and `translate_from` will be renamed in v3
_get_allele_location_pos picked the defined bound of an indefinite Range
with `pos.root[0] or pos.root[1]`, so a bound of 0 (e.g. Range([0, None]))
was treated as missing and val became None, which made normalize() raise
a TypeError. Pick the defined side explicitly instead.
`normalize()` raises a `TypeError` for an allele whose location has an
indefinite range with 0 as its defined bound, e.g. `start=Range([0,
None])` or `end=Range([0, None])`:

```
TypeError: '>' not supported between instances of 'NoneType' and 'int'
```

`_get_allele_location_pos` chose the defined bound with `pos.root[0] or
pos.root[1]`. Because 0 is falsy, `Range([0, None])` gave `val=None`,
which then reached bioutils' `normalize`. `Range([1, None])` and
`Range([None, 0])` worked, so only a 0 lower bound on an open-upper
range was affected.

The fix picks the defined side explicitly: `pos.root[1] if pos0_is_none
else pos.root[0]`.

### Tests
Two cases added to `normalize_location_in_bounds_cases` (local test
SeqRepo, no cassettes):
- `start=[0, None], end=1` substitution
- `start=[None, 0], end=[0, None]` insertion

Both fail on `main` with the TypeError above and pass with this change.
`ruff check` and `ruff format --check` are clean.
Tests no longer connect to UTA, NCBI, or any other live service.
Library code is unchanged.

- Serve hgvs data provider lookups from tests/data/hgvs_cache.json in
  hgvs's cache run mode. The cache is committed as JSON rather than
  pickle; tests/hgvs_cache_json.py converts between the two.
- Record lookups that found no data too, and replay them as the same
  HGVSDataNotAvailableError. A lookup that was never recorded fails
  the test.
- Default to --vcr-record=none, and fail any test that opens a live
  socket or PostgreSQL connection.
- Add `make record-hgvs-cache` (needs UTA_DB_URL and a
  seqrepo-rest-service), and re-record hgvs-related cassettes without
  NCBI requests.

Closes #659
@korikuzma korikuzma self-assigned this Nov 11, 2026
@korikuzma korikuzma added the priority:medium Medium priority label Nov 11, 2026
@korikuzma
korikuzma requested review from a team as code owners November 11, 2026 12:14
@korikuzma

Copy link
Copy Markdown
Contributor Author

There are more changes to be included, but mainly want to get Kyle's work in #667 in

@larrybabb larrybabb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing this sync. The merge looks clean. Running the full suite on this branch with v3's own dependencies (hgvs 2.0.0a0) in offline replay mode gave 315 passed, and ruff lint and format both pass. Every entry in tests/data/hgvs_cache.json is used by v3's tests, the new get_seq entries cover the v3-only uncertain-range tests, and the normalize fix (pos.root[1] if pos0_is_none else pos.root[0]) is correct on v3. I didn't find anything blocking.

One thing worth doing before merge: main has moved on since this branch was last updated. Two commits landed on main after the PR head:

  • #682 feat(annotator): let subclasses abort VCF annotation
  • #660 feat: make the shared RLE sequence limit configurable

#660 in particular reorganizes the same test infrastructure this PR brings over. It moves _wrap_cached_method from tests/conftest.py into a new tests/hgvs_cache_provider.py, moves fixtures into tests/extras/conftest.py, and edits tests/test_hgvs_cache.py, tests/hgvs_cache_json.py and tests/cassettes/test_normalize_allele.yaml. Merging as-is leaves v3 with the older layout and without GA4GH_VRS_RLE_SEQ_LIMIT or the annotator abort hook, and the next sync will conflict with nearly every file touched here. Pulling those two commits into this PR first would make the sync complete and the follow-up much easier.

Comment thread tests/conftest.py
yield blocked


def _wrap_cached_method(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On current main (after #660), this helper lives in tests/hgvs_cache_provider.py and test_hgvs_cache.py imports it from there instead of from conftest import. Bringing #660 into this PR would keep the two branches' test layouts aligned (see the summary).

@larrybabb larrybabb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few additional optional notes. They don't block the PR. Some of this code may change anyway once #660 is pulled in, so take them as things to keep an eye on.

Comment thread tests/hgvs_cache_json.py
cache: dict = pickle.load(f) # noqa: S301 # hgvs cache key -> cached result
entries = []
for key in cache:
*args, marker, func = key

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional: this unpacking assumes cache keys never include kwargs, and json_to_pickle always replays with {}. hgvs's _make_key appends sorted kwargs after the function name, so a provider call like hdp.get_tx_info(ac, alt_ac, alt_aln_method="splign") would produce a key that hits the "unexpected hgvs cache key layout" error, after a full live recording run has finished. No current call site uses kwargs, so this is only future-proofing. A kwargs field in the JSON (or a clear note that kwargs aren't supported) would prevent a confusing failure later.

Comment thread tests/conftest.py
def wrapped(*args: object, **kwargs: object) -> Any:
try:
result = method(*args, **kwargs)
except HGVSDataNotAvailableError as exc:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional: the replay cache depends on private hgvs internals (_make_key, __wrapped__, and connect() skipping the DB in "run" mode), and v3 is on an hgvs 2.0 alpha. If a later pre-release changes the key layout, every lookup misses here and each test fails with "Missing HGVS recording … Run make record-hgvs-cache", which sends people to re-record instead of the real cause. Pinning hgvs more tightly in the test extra, or a quick session-start check that a known JSON entry still resolves, would make that failure obvious.

Comment thread docs/development.md
path when running or recording tests, as CI does:

```shell
SEQREPO_ROOT_DIR=tests/data/seqrepo/latest python3 -m pytest --vcr-record=none

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor docs overlap: this paragraph and the new "Recorded test data" section right below both say to set SEQREPO_ROOT_DIR, and this example still passes --vcr-record=none, which is now the default from pyproject.toml addopts (the CI step in .github/workflows/python-cqa.yml passes it too). Merging the two paragraphs and dropping the redundant flag would keep the docs and CI from drifting from pyproject.

Comment thread docs/development.md

```shell
# hgvs cache: needs UTA and seqrepo-rest-service; does not touch cassettes
export UTA_DB_URL=postgresql://anonymous@localhost:5432/uta/uta_20241220

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: this repeats service setup already covered in docs/data-services.md (linked from "Set up SeqRepo and UTA" above), and the URL differs slightly. data-services.md uses postgresql://anonymous:anonymous@localhost:5432/uta/uta_20241220 and documents alternate ports (5433/5001). Linking there instead would keep one source of truth and avoid recording against the wrong port for someone who remapped it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority:medium Medium priority

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants