Evolving viral populations often diverge substantially from available reference genomes, potentially reducing read mapping accuracy and biasing variant calls. consiter produces an updated reference genome that more closely matches the viral population by iteratively mapping reads to the reference and generating a new consensus sequence.
In each iteration, paired-end reads are mapped to the current reference with Bowtie2, duplicates are removed, variants are called (with GATK4 HaplotypeCaller and hard filters on QD, FS, MQ, ReadPosRankSum and depth, or with bcftools from the pileup), and the passing variants are applied to make the next reference.
This continues until convergence (see below) or the maximum number of consensus rounds is reached. There are two convergence tests which can be chosen via the --stop flag. The rate test stops when the overall alignment rate no longer improves over the previous iteration, and the consensus test stops when the caller applies no new variant or returns a consensus seen in an earlier round (a cycle). Both are reported in <name>.iterations.tsv whichever is selected.
The latest consensus is then written out together with the reads mapped to it, and bases covered by fewer than --min-cov reads are masked with N.
Two read depth filters are used: a variant needs --call-min-cov reads to be applied to the next reference, while the output is masked at --min-cov.
Create the conda environment, which installs the external tools and consiter itself (as an editable install):
conda env create -f environment.yml
conda activate consiter
To install into an existing environment instead, make sure Bowtie2, Samtools, BedTools and GATK4 are on your PATH, then run pip install . from this directory. BCFtools is additionally required for --caller bcftools.
environment.yml pins versions that have been tested.
consiter -r reference.fa -n SAMPLE -1 reads_R1.fastq.gz -2 reads_R2.fastq.gz -o outdir
Run consiter --help for all options:
| Option | Default | Description |
|---|---|---|
-r, --ref |
required | Reference genome FASTA (may be gzipped) |
-n, --name |
required | Sample name, used for read groups and output file names |
-1, --left / -2, --right |
required | Paired-end reads in FASTQ format |
-o, --outdir |
. |
Output directory |
-i, --max-iter |
10 | Maximum number of consensus rounds (the loop usually stops earlier) |
-t, --threads |
16 | Threads for Bowtie2 and Samtools |
-p, --ploidy |
1 | Ploidy for the variant caller (bcftools supports 1 or 2) |
-c, --min-cov |
20 | Minimum read depth for a base not to be masked in the final consensus; also the depth at which breadth is reported |
--call-min-cov |
2 | Minimum read depth for a variant to be applied to the next reference |
--caller |
gatk |
Variant caller for the consensus rounds: gatk or bcftools |
--stop |
both |
Which convergence test ends the loop: rate, consensus or both |
--no-dedup |
off | Keep duplicate reads; suggested for amplicon libraries |
--xmx |
50 | Maximum Java heap size for GATK, in GB |
--keep-tmp |
off | Keep the working directory <outdir>/tmp, including per-step logs. consiter refuses to start if <outdir>/tmp already exists |
--no-indel |
off | Only apply SNPs, so the updated reference keeps the original coordinates |
By default, insertions and deletions are applied to the updated reference, so its coordinates can differ from the original reference.
This tool only works with paired-end (PE) reads.
<name>.consensus.fa: updated reference with low-coverage bases masked; sequence names match the input reference.<name>.consensus.sample_name.fa: the same sequences, named<name>|<reference sequence name>.<name>.bt2.rmdup.bam(and.bai): the reads aligned to the updated reference, with duplicates removed unless--no-dedupwas given (the file name does not change).<name>.iterations.tsv: one row per iteration — alignment rate, duplicate fraction, mean depth, coverage breadth at--min-cov, bases below--min-cov, a hash of the reference, whether the consensus was changed this round, and why the loop stopped. It is rewritten after every iteration, so it is there to inspect even if a later step fails.
Note-- Consensus sequences that are mostly N will still have a high alignment rate. consiter warns when coverage is too thin to iterate from, when the duplicate fraction suggests an amplicon library, and when more than half the output is masked.
If a step fails, constier stops with the failing program's error, and <outdir>/tmp is kept so the logs in <outdir>/tmp/logs can be inspected.
pytest # unit tests
pytest -m integration # end-to-end runs on simulated data (needs the conda environment)
ruff check . && ruff format --check .
By using this software, you agree this software is to be used for research purposes only. Any presentation of data analysis using the software will acknowledge the software according to the guidelines below.
Primary author(s): David B. Stern
A review of this code has been conducted, no critical errors exist, and to the best of the authors knowledge, there are no problematic file paths, no local system configuration details, and no passwords or keys included in this code. This open source software comes as is with absolutely no warranty.