Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 20 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,10 @@ cd Lverage
```

### Python
Lverage requires Python3.8 at minimum to run. The tool was developed using Python3.10.12. To obtain python and the required libraries, we advise using Conda.
The legacy workflow requires Python 3.8 at minimum and was developed with
Python 3.10.12. The V2 package requires Python 3.11 or newer; see the V2 CLI
section below. To obtain Python and the required libraries, we advise using
Conda.

#### Downloading Conda
To install Conda, follow the instructions on their [website user guide](https://conda.io/projects/conda/en/latest/user-guide/install/index.html "https://conda.io/projects/conda/en/latest/user-guide/install/index.html"). We suggest downloading Miniconda.
Expand Down Expand Up @@ -95,7 +98,22 @@ the
[API documentation](docs/api/index.rst) for how to pass them to the V2
pipeline.

## How to Use
## V2 Command-Line Interface

The V2 CLI is separate from the legacy workflow below. Run the installed
`lverage` command or use `PYTHONPATH=src python -m lverage.cli` from a source
checkout. It requires Python 3.11+, local BLAST+ and PfamScan tools/data, and a
local protein database; full motif lookup uses the online JASPAR2024 service.
See the [V2 FASTA and CLI guide](docs/user_guide.rst) for input mapping,
portable examples (including local runs stopped after alignment), output
formats, selection behavior, and status codes. See the local BLAST database
instructions above for database setup.

## Legacy Command-Line Workflow

This older workflow is separate from the V2 package and CLI. The options and
examples in this section refer only to the legacy implementation.

We warn against moving any file within the directory anywhere else as this will create errors. If you wish to access from other places, we suggest appending the directory to your PATH environment variable, creating an alias, or creating a shortcut.

To call Lverage, ensure that the requirements above are all met. Within the directory is a file called *lverage.py*. All calls should be made with this file.
Expand Down
18 changes: 12 additions & 6 deletions docs/api/blast.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,13 @@ BLAST searchers
===============

BLAST+ is an external prerequisite and is not installed as a Python package.
On the SCC, ``blast+/2.12.0`` and the ``human_mouse_nr`` database are useful
development examples; applications must still pass their own executable and
database locations.
Configure the executable and database locations for the environment where
Lverage is run.

Local databases must be built with ``makeblastdb -parse_seqids`` because
specific complete-sequence retrieval through ``blastdbcmd`` requires a parsed
sequence-ID index. The existing SCC ``human_mouse_nr`` database was built
without that index and must be rebuilt before it can support an end-to-end
``LocalBlastSearcher`` run.
sequence-ID index. Rebuild databases that lack this index before using them
with ``LocalBlastSearcher``.

``LocalBlastSearcher`` returns :class:`lverage.ortholog_searcher.OrthologRecord`
objects.
Expand All @@ -19,6 +17,14 @@ excluded, unresolved, or unretrievable hits are skipped and logged; executable,
database, subprocess, timeout, and whole-result parsing failures are raised to
the caller.

The existing ``get_orthologs(sequence)`` call remains unchanged. The shared
pipeline may additionally pass an evidence collector; in that mode each
retrieved ortholog and each failed/skipped sequence-retrieval attempt is
reported as it completes. This lets the CLI retain earlier hits if a later
retrieval fails and distinguishes a BLAST result with no alignments, alignments
filtered from retrieval, and retained hits whose sequences could not be
retrieved. The optional reporting mode does not rerun BLAST.

``LocalBlastSearcher`` validates a database prefix with ``blastdbcmd -info``
and retrieves complete subject proteins from that database.

Expand Down
4 changes: 2 additions & 2 deletions docs/api/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@ API reference
=============

This reference documents the current version 2 core interfaces. Package-level
exports and a command-line interface are intentionally deferred, so import
classes from the modules shown below.
exports are intentionally deferred, so import classes from the modules shown
below. The V2 command-line interface is documented in the user guide.

.. toctree::
:maxdepth: 1
Expand Down
11 changes: 8 additions & 3 deletions docs/api/pipeline.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,14 @@ local configuration and do not contact remote services.

Each run deduplicates ORFs, selects the first longest ORF with an accepted
domain, compares matching base PFAM accessions, and searches motif databases
for domain pairs meeting the identity threshold. Intermediate evidence remains
local to the run; the instance retains its copied configuration and the latest
:class:`lverage.pipeline.LverageCode`.
for domain pairs meeting the identity threshold. Intermediate evidence is not
retained by default. Callers may pass an optional ``evidence_collector`` to
``Lverage.run()`` to receive completed stage events as they occur; this supports
incremental reporting but is not checkpoint or crash recovery. The instance
retains its copied configuration and latest :class:`lverage.pipeline.LverageCode`.

``stop_after="alignment"`` deliberately returns before motif-database searches
while preserving full-pipeline behavior when the option is omitted.

Valid biological no-hit outcomes return an empty list and set a deterministic
code. Adapter and service failures remain exceptions.
Expand Down
139 changes: 130 additions & 9 deletions docs/user_guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,128 @@ Use nucleotide FASTA input to provide the gene or transcription-factor (TF)
sequence from which Lverage infers a protein and its DNA-binding domain.
Prepare one FASTA file per gene/TF and name the file after the gene, for
example ``geneX.fa``. Keep records for different genes in separate files. The
command-line workflow requires this layout; for the Python API it is a
convention, since the caller reads the file.
V2 command-line interface derives the gene identifier from each filename
stem; for the Python API, the caller reads files and supplies any identifier.

V2 command-line interface
-------------------------

The V2 CLI uses the current modular pipeline and is distinct from the legacy
command-line workflow described at the end of this guide. It can be invoked as
an installed ``lverage`` command or, from a source checkout, as
``PYTHONPATH=src python -m lverage.cli``. ``--help`` only displays arguments;
it does not construct adapters, inspect external tools, or access a network.

Prerequisites are the Python package dependencies, local BLAST+ executables
(``blastp`` and ``blastdbcmd``), a usable local protein database, and a local
PfamScan installation/database. JASPAR2024 is queried through its online
service, so a full run that reaches motif lookup requires network access. A
local run stopped after alignment does not contact JASPAR and can run without
network access. The local BLAST database must support complete protein
retrieval with ``blastdbcmd``. See the README's V2 local BLAST database
instructions for building one.

Create a JSON species-name-to-NCBI-tax-ID map. Species names must match the
bracketed names in BLAST descriptions (case-insensitive), and tax IDs must be
positive integers:

.. code-block:: json

{
"Homo sapiens": 9606,
"Mus musculus": 10090
}

Example using portable placeholder paths:

.. code-block:: console

lverage --input-dir ./inputs --blast-db ./db/proteins \
--species-map ./config/species-taxids.json --pfam-db ./db/pfam \
--output-dir ./results

For a local pre-JASPAR run that stops after domain alignment:

.. code-block:: console

lverage --input-dir ./inputs --blast-db ./db/proteins \
--species-map ./config/species-taxids.json --pfam-db ./db/pfam \
--output-dir ./alignment-results --stop-after alignment

Required options are ``--input-dir``, ``--blast-db``, ``--species-map``,
``--pfam-db``, and ``--output-dir``. The output directory must be new. It
contains three files: ``results.tsv`` contains typed pipeline evidence,
``orfs.fasta`` stores each unique candidate protein sequence once, and
``metadata.json`` records run settings and summaries. Optional executable
overrides are ``--blastp`` (``blastp``),
``--blastdbcmd`` (``blastdbcmd``), and ``--pfamscan`` (``pfam_scan.pl``).
``--identity-threshold`` defaults to ``0.7``. Repeat ``--valid-pfam`` to
restrict accepted domains; if it is omitted, every detected Pfam domain is
accepted. The pipeline selects at most one query ORF per gene: candidates from
the input fragments are deduplicated and sorted longest-first, then the first
candidate with an accepted domain is selected for downstream analysis.
JASPAR options default to 10 hits and an inference E-value threshold of
``1e-6``. Other component options retain their V2 defaults: ORF minimum length
75 nucleotides; BLAST E-value ``1e-6``, at most 20 hits, and the standard
description filters.

The CLI discovers ``.fa``, ``.fasta``, and ``.fna`` files nonrecursively,
case-insensitively, in filename order. The filename stem labels results and
must be unique ignoring case. Every FASTA record in a file is passed as an
independent sequence fragment to the same gene run. The CLI does not assemble
fragments, and it cannot determine from record headers whether they really
belong to the same gene. Empty files/records and unsupported DNA symbols are
rejected before external-tool adapters are constructed.

The TSV has explicit ``Row Type``, ``Stage``, ``Status``, ``Error``, and
``Outcome Detail`` fields. ``Error`` is reserved for execution failures;
biological reasons and not-reached explanations use ``Outcome Detail``.
Row types include ``input``, ``orf``, ``query_domain``, ``ortholog``,
``ortholog_retrieval``, ``ortholog_domain``, ``domain_comparison``, ``motif``,
and ``gene_error``.
``stage_status`` rows record biological no-hits, below-threshold completion,
intentional stops, and stages not reached.
``ortholog_retrieval`` rows retain failed retrieval attempts and skipped hits,
including available native hit IDs/accessions, the attempted identifier, and a
diagnostic reason. A BLAST no-hit status is distinct from retrieval failure.
``results.tsv`` has multiple evidence row types; a row is not necessarily a
separate ORF or a final motif result. For example, an ORF may have candidate,
domain, selection, ortholog, comparison, and stage-status rows, while a motif
row is emitted only for a returned motif. Each motif has one row identifying
its input gene, query ORF, ortholog, and matching query/ortholog domains.
Multiple hits, domain occurrences, and motifs remain distinct. ORF rows retain
candidates even when no accepted domain exists, and an input row records a
gene with no ORF. ``orfs.fasta`` stores each unique discovered candidate
protein once under a deterministic full-sequence SHA-256 identifier
referenced by the TSV; gene associations and selection status remain in the
TSV. Candidate export does not run downstream analysis on unselected
candidates.

BLAST identity is the identity ratio from the HSP with the lowest E-value and
is distinct from domain-pair identity, which is the exact-match ratio across
every column of a global amino-acid alignment of matching Pfam domains
(including gaps).
The TSV reports BLAST identity and query coverage as ratios and percentages,
and records every computed domain-pair identity, including values below the
threshold. Domain start and end coordinates in exported rows are zero-based,
end-exclusive amino-acid positions. The domain-alignment identity score is
exported as a ratio and percentage, but the aligned sequences themselves are
not currently exported.

``metadata.json`` records run timestamps, settings, input paths/checksums,
sequence-ID conventions, known tool versions (unknown versions are reported as
``unknown``), and completion/failure summaries. Results and metadata are
flushed as pipeline evidence completes. This is incremental evidence retention,
not checkpoint/resume or recovery from arbitrary crashes. After execution
begins, the bundle is preserved on failure. Execution failures are caught per
gene and processing continues to other genes, retaining completed evidence if
a later stage fails.

Use ``--stop-after alignment`` for pre-JASPAR validation. This intentional stop
does not call JASPAR and is not a failure. Exit status ``0`` means completed or
intentionally stopped, including biological no-hits; ``2`` means preflight
validation failure; ``1`` means one or more execution failures despite
preserved partial results. Diagnostics go to standard error.

FASTA format
------------
Expand Down Expand Up @@ -34,9 +154,9 @@ same gene/TF, for example several scaffolds or contigs:
>geneX_fragment_2
GCTAGCATCGATCGTA

The record headers identify fragments of the same gene in that one file. Do
not place unrelated genes or TFs together in a multi-record input file: use a
separate FASTA file for each one.
Treat all records in the file as intended fragments of the same gene. The
headers themselves are not validated as evidence that fragments belong to
that gene. Keep unrelated genes or TFs in separate files.

The V2 ORF adapter accepts nonempty, case-insensitive DNA strings using the
IUPAC symbols ``A C G T R Y S W K M B D H V N``. It does not accept RNA ``U``,
Expand Down Expand Up @@ -91,13 +211,14 @@ BLASTP. Do not use those protein records as the gene DNA input to motif
inference. The README's V2 local BLAST database section describes the protein
FASTA used to build that database.

Legacy command-line workflow versus V2 API
------------------------------------------
Legacy command-line workflow and V2 Python API
----------------------------------------------

The legacy command-line workflow is separate from the V2 Python API. Its
``-f/--fasta`` option takes a directory containing one file per gene/TF; a
file may contain one record or multiple records for fragments of that same
gene. The legacy implementation reads all FASTA records in each file.

For V2, configure the Python pipeline and call ``Lverage.run()`` with a DNA
string or list of DNA strings as shown above.
The V2 CLI is documented above. For the V2 Python API, configure the pipeline
and call ``Lverage.run()`` with a DNA string or list of DNA strings as shown
above.
3 changes: 3 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ dependencies = [
"requests>=2.32",
]

[project.scripts]
lverage = "lverage.cli:main"

[project.optional-dependencies]
docs = [
"numpydoc>=1.10",
Expand Down
Loading
Loading