Skip to content
trypsynthPublic

About

The fastest and most accurate code counter around: code, comment, blank, and shebang lines by language, as text, JSON, CSV, TSV, Markdown, or HTML

Topics

Resources

Stars

15 stars

Watchers

1 watching

Forks

Repository files navigation

codestats (cs)

Crates.io License MSRV

The fastest and most accurate code counter around. Think that's wrong? Open an issue with numbers or a miscounted file and I'll change this line.

codestats counts the code, comment, blank, and shebang lines in a codebase by language, and reports them as text, JSON, CSV, TSV, Markdown, or HTML. Use it for audits, comparisons, scripts, or to see how much code is in a directory.

Features

  • Counts files, lines, and bytes by language.
  • Counts code embedded in other files by that code's own language rules, including:
    • Scripts and styles in HTML, Vue, Svelte, Astro, and HTML templates.
    • The HTML around <?php ... ?> blocks in PHP files.
    • Code blocks in Markdown, AsciiDoc, Org, and reStructuredText.
  • Respects .gitignore and .ignore files.
  • Skips generated files, such as lockfiles and files marked DO NOT EDIT.
  • Reads the linguist attributes in .gitattributes files, the same way GitHub does.
  • Filters by language, glob pattern, and directory depth.
  • Shows counts per file or per directory.
  • Recognizes more than 770 languages.
  • Handles UTF-8, UTF-16, and legacy text encodings.

Install

Download a binary

Download the binary for your platform from GitHub Releases:

Platform Download
Linux x86_64 cs-x86_64-unknown-linux-gnu
Linux ARM64 cs-aarch64-unknown-linux-gnu
Linux x86_64, static (musl) cs-x86_64-unknown-linux-musl
Linux ARM64, static (musl) cs-aarch64-unknown-linux-musl
macOS Intel cs-x86_64-apple-darwin
macOS Apple Silicon cs-aarch64-apple-darwin
Windows x86_64 cs-x86_64-pc-windows-gnu.exe
Windows ARM64 cs-aarch64-pc-windows-gnullvm.exe

The standard Linux binaries need glibc 2.17 or later and run faster than the static ones. Use the static binaries on systems without glibc, such as Alpine.

On Linux and macOS, make the binary executable and move it to a directory on your PATH:

chmod +x cs-*
sudo mv cs-* /usr/local/bin/cs

Install with Cargo

cargo install codestats

This requires Rust 1.88 or later. Release binaries are built with profile-guided optimization, so they run faster than a build from cargo install. The minimum Rust version only increases in minor releases, never in patch releases.

Build from source

git clone https://github.com/trypsynth/codestats
cd codestats
cargo install --path .

Quick start

Count the current directory:

cs

Count several paths together:

cs src tests build.rs

Files are listed by the path you gave, such as src/main.rs. If you give the same path twice, or a path inside another one you gave, its files are counted once.

Count a list of files from standard input:

git ls-files -z | cs -
fd -e rs | cs -

The list can have one path per line or NUL-separated paths. Directories in the list are walked. The usual filters apply, except .gitignore, so hidden files are skipped unless you pass -H. A symbolic link to a file counts as that file. A symbolic link to a directory is followed only with --symlinks.

Show per-file counts for src/ as JSON:

cs -v src -o json

Compare with an earlier release:

cs diff v1.0.0               # v1.0.0 against the working tree
cs diff v1.0.0 --to v2.0.0   # two revisions

For each language, cs diff shows the files, lines, code, comments, and blanks in both versions, and the change. Both sides count the files git tracks, with the same filters, so only real changes appear.

More examples:

cs -t 10                                         # only the 10 largest languages
cs -L rust                                       # only Rust
cs -e 'test_*' -e '*.md'                         # exclude glob patterns
cs --exclude-lang markdown --exclude-lang toml   # exclude languages
cs --symlinks -H                                 # follow symbolic links, include hidden files
cs --no-gitignore                                # include files .gitignore excludes
cs -D                                            # counts by directory
cs langs                                         # list supported languages
cs completions bash                              # shell completions

Output formats

  • human (default) for terminals.
  • json or json-compact for scripts.
  • csv or tsv for spreadsheets.
  • markdown or html for documents and dashboards.

JSON

Every level of a JSON report counts lines the same way. paths lists the paths that were counted. files appears with --verbose, and directories with --by-dir. cs diff -o json reports its from and to counts in the same shape.

A language or file that contains code in another language, such as Rust in a Markdown file, has an embedded list with each embedded language and its lines. Those lines are also included in the language's or file's own lines.

"embedded": [{ "name": "Rust", "lines": { "total": 120, "code": 96, "comment": 14, "blank": 10, "shebang": 0 } }]
{
	"schema_version": 2,
	"paths": ["src"],
	"totals": {
		"file_count": 30,
		"lines": { "total": 5956, "code": 5083, "comment": 331, "blank": 542, "shebang": 0 },
		"bytes": 195221
	},
	"languages": [
		{
			"name": "Rust",
			"file_count": 30,
			"lines": { "total": 5956, "code": 5083, "comment": 331, "blank": 542, "shebang": 0 },
			"bytes": 195221,
			"files": [{ "path": "cli.rs", "lines": { "total": 328, "code": 264, "comment": 39, "blank": 25, "shebang": 0 }, "bytes": 12873 }]
		}
	],
	"hidden_languages": 0
}

Stability

Starting with 1.0, JSON fields, CSV and TSV columns, and configuration file keys only change in a new major version.

schema_version increases whenever a JSON field is renamed, removed, or changes meaning. New fields can appear without a version change, so your scripts should ignore fields they don't recognize.

Command-line options

Usage: cs [OPTIONS] [PATH]...

If you don't give a path, codestats counts the current directory. If the path is -, it reads the paths to count from standard input.

  • -q, --quiet: show only the totals.
  • -v, --verbose: also show the counts for each file.
  • --no-gitignore: ignore .gitignore files.
  • --no-gitattributes: ignore the linguist attributes in .gitattributes files. See Linguist attributes.
  • --include-generated: count generated files. See Generated files.
  • --git: count only the files that git tracks, including in submodules. .gitignore doesn't apply to tracked files.
  • --max-depth <N>: descend at most N directory levels.
  • --min-lines <N>: hide languages with fewer than N lines.
  • -H, --hidden: include hidden files and directories. The .git directory is always skipped.
  • --symlinks: follow symbolic links. A link to one of its own parent directories is skipped and reported as an error.
  • -e, --exclude <PATTERN>: exclude files and directories that match a glob pattern. Repeatable.
  • -L, --lang <LANGUAGE>: count only this language. Repeatable, or comma-separated. Can't be combined with --exclude-lang.
  • --exclude-lang <LANGUAGE>: don't count this language. Repeatable, or comma-separated. Can't be combined with --lang.
  • --ext <EXT>: count only files with this extension, such as rs or py. Repeatable, or comma-separated.
  • -n, --number-style <plain|comma|underscore|space>: how to format numbers. Default: plain.
  • -u, --size-units <binary|decimal>: the units for sizes. Default: binary.
  • -p, --precision <0-6>: decimal places in percentages. Default: 1.
  • -s, --sort-by <lines|code|comments|blanks|files|size|name>: how to sort languages and files. Default: lines.
  • -d, --sort-direction <asc|desc>: the sort direction. Default: desc.
  • -t, --top-languages <N>: show only the N largest languages. N must be at least 1; to show only the totals, use --quiet.
  • -D, --by-dir: show counts by directory instead of by language.
  • -o, --output <human|json|json-compact|csv|tsv|markdown|html>: the output format. Default: human.
  • --indent <tab|1-8>: the indentation of human-readable output, as a tab or a number of spaces. Default: tab.
  • --fail-on-error: exit with an error if any file couldn't be read.
  • -c, --config <PATH>: use this configuration file.
  • -h, --help: show help.
  • -V, --version: show the version.

Exit status

  • 0: success.
  • 1: an error, including a file that couldn't be read when you pass --fail-on-error.
  • 2: invalid arguments.

Subcommands

  • diff <FROM> [PATH]: compare counts by language between a git revision and the working tree, or another revision with --to <REV>.
  • langs: list the supported languages.
  • completions <shell>: generate shell completions, for example for bash, zsh, or fish.
  • init: write a configuration file that lists every setting.

Configuration

codestats can read settings from a TOML file. Options on the command line override the file. An unknown key in the file is an error, so typos don't go unnoticed.

codestats uses the first of these files that exists:

  1. The file you pass with --config <path>. It's an error if this file doesn't exist.
  2. ./.codestats.toml
  3. ./codestats.toml
  4. ~/.config/codestats/config.toml
  5. ~/.codestats.toml

Example configuration

paths = ["src", "tests"]  # Relative to this file.

[analysis]
verbosity = "verbose"
respect_gitignore = true
respect_gitattributes = true  # Use the linguist attributes in .gitattributes files.
git_tracked_only = false  # true counts only the files git tracks.
include_hidden = true
follow_symlinks = false
include_generated = false
max_depth = 5
exclude_patterns = ["languages.json5", "*.tmp", "node_modules/*"]  # File names or glob patterns.
include_languages = ["rust", "python"]  # Count only these languages.
# exclude_languages = ["markdown", "toml"]  # Or skip these. You can't set both.
include_extensions = ["rs", "py"]  # Count only these extensions.
fail_on_error = false

[display]
number_style = "comma"
size_units = "decimal"
precision = 4
sort_by = "files"
sort_direction = "desc"
output = "human"
top_languages = 10
min_lines = 100
by_dir = false
indent = "tab"

Generated files

By default, codestats skips generated files. To count them, pass --include-generated.

A file counts as generated in any of these cases:

  • Its linguist attribute says so. See Linguist attributes.
  • Its name says so. Lockfiles, such as Cargo.lock and package-lock.json, and minified files, such as *.min.js and *.min.css.
  • Its header says so. codestats reads the comments at the start of each file, up to the first line of code, and looks for a note that a tool wrote the file. For example:
    • // Code generated by stringer; DO NOT EDIT. (Go)
    • // Generated by the protocol buffer compiler. DO NOT EDIT! (protobuf)
    • // <auto-generated /> (C#)
    • // @generated by pegen, or any other @generated
    • /* automatically generated by rust-bindgen */
    • # This makefile was generated by bakefile. followed closely by # Do not modify

A comment that only mentions generated code, such as // Retrieves the auto-generated key, doesn't make a file generated.

Linguist attributes

GitHub uses the linguist attributes in .gitattributes files for its language statistics, and codestats uses them the same way:

Attribute Effect
linguist-generated The file is generated, so it's skipped unless you pass --include-generated.
-linguist-generated or linguist-generated=false The file isn't generated, even if its name or header looks like it is.
linguist-vendored The file is skipped.
linguist-documentation The file is skipped.
linguist-language=Name The file is counted as Name, with the exceptions below.

codestats ignores linguist-language when:

  • It doesn't recognize Name.
  • The file's name already identifies exactly one language. For example, *.bgt files stay BGT and *.nvgt files stay NVGT, even though many projects map them to AngelScript because GitHub doesn't recognize BGT or NVGT.

When the file's name matches no language or several, such as *.h or *.pp, Name decides.

For example:

**/zz_generated.*.go linguist-generated
lib/libc/**          linguist-vendored
*.fixed              linguist-language=Rust

codestats reads the .gitattributes files in the directories it counts, and in the directories above them up to the repository root. A file in a deeper directory overrides one higher up, and within a file, later lines override earlier ones. Macros, such as [attr]generated linguist-generated, are expanded.

With --git and cs diff, the working tree's .gitattributes files apply to every revision, so both sides of a comparison follow the same rules.

To ignore linguist attributes, pass --no-gitattributes.

Technical notes

Memory-mapped files

codestats memory-maps files of 64 KB or larger, which makes reading them faster. Don't modify files while codestats is counting them.

Threads

codestats counts files on several threads, each keeping its own totals, and combines them at the end. You can run several codestats processes on different directories at once, but don't run more than one on the same directory.

Accuracy

Every change is checked in CI against LineJudge, a collection of tricky files: comment markers in strings, raw strings, heredocs, languages embedded in other languages, and more. LineJudge checks codestats against the counting rules declared in .linejudge.

Each tool is judged against its own declared rules, so a failure means that the tool breaks its own rules. Results with LineJudge 0.3.0:

Tool Cases that agree, out of 95
codestats 95
mezura 3.3.0 89
scc 4.1.0 71
tokei 15.0.0 45

codestats' answers are recorded in .linejudge/recorded/codestats.toml, and CI fails if any of them changes.

To run LineJudge locally:

  1. Install LineJudge.
  2. Run cargo build --release.
  3. Run linejudge check --counter codestats.

Benchmarks

All times are for files already in the file cache, on an AMD Ryzen 7 PRO 5850U (8 cores, 16 threads) with 32 GB of RAM. They use the x86-64 Linux glibc binary that CI built at commit b84c4f0, which is the same build a release ships.

Equal work

In these tables, every tool counts the same files. linebench pins each repository's commit, turns off each tool's filters, and checks that each tool's file and line counts are within 1% of the repository's.

Mean time:

Repository Files Lines codestats mezura scc tokei
cpython 3,554 2.2M 18 ms 24 ms 51 ms 107 ms
jdk 61,420 13M 128 ms 154 ms 345 ms 881 ms
linux 63,738 36M 217 ms 238 ms 626 ms 1.11 s

Peak memory:

Repository codestats mezura scc tokei
cpython 12 MB 19 MB 47 MB 18 MB
jdk 19 MB 28 MB 95 MB 57 MB
linux 81 MB 110 MB 347 MB 245 MB

To reproduce these tables, run the following command with linebench 0.3.1, after running linebench fetch --counters all --corpus all:

linebench run cpython,linux,jdk --counters scc,tokei,mezura,codestats@dev --given codestats@dev=path/to/cs --definition codestats@dev=.linebench/codestats.toml --add .linebench

Each tool ran 30 times, half of them in reverse order. These tables measure codestats, mezura 3.3.0, scc 4.1.0, and tokei 15.0.0.

Default settings

In these tables, every tool runs with no flags, which is how most people use it. The tools don't count the same files, so compare these times with care. For example, mezura counts 253K lines in nvda, where codestats counts 1.6M, because it leaves out Gettext translation files. The file and line counts in the table are from codestats.

Mean time:

Repository Files Lines codestats mezura tokei scc gocloc
ripgrep 154 65K 3.8 ms 7.3 ms 9.6 ms 11 ms 38 ms
cli 1,102 312K 7.6 ms 11 ms 19 ms 18 ms 112 ms
nvda 1,158 1.6M 13 ms 9.5 ms 41 ms 23 ms 136 ms
wxWidgets 4,491 2.3M 29 ms 25 ms 79 ms 103 ms 450 ms
cpython 4,947 2.5M 25 ms 30 ms 130 ms 78 ms 670 ms
kubernetes 22,693 5.2M 66 ms 67 ms 271 ms 218 ms 2.06 s
rust 44,334 5.1M 93 ms 94 ms 381 ms 230 ms 2.48 s
linux 92,811 44M 300 ms 323 ms 1.39 s 1.00 s 11.2 s

Peak memory:

Repository codestats mezura tokei scc gocloc
ripgrep 7.4 MB 8.6 MB 7.9 MB 27 MB 29 MB
cli 8.3 MB 9.7 MB 8.3 MB 32 MB 36 MB
nvda 19 MB 9.9 MB 27 MB 38 MB 38 MB
wxWidgets 19 MB 20 MB 24 MB 74 MB 40 MB
cpython 14 MB 22 MB 25 MB 66 MB 44 MB
kubernetes 19 MB 24 MB 64 MB 96 MB 53 MB
rust 33 MB 43 MB 92 MB 129 MB 64 MB
linux 80 MB 179 MB 285 MB 385 MB 111 MB

To reproduce these tables, run CS=path/to/cs scripts/bench.sh, which pins each repository's commit. It uses hyperfine 2.0.0 and these tools: mezura 3.3.0, tokei 15.0.0, scc 4.1.0, and gocloc 0.7.0.

Windows

These times are from a different machine than the tables above, so don't compare the two. It has an AMD Ryzen 9 5950X (16 cores, 32 threads) and 128 GB of RAM, and runs Windows 11 with Microsoft Defender real-time protection on. They use the x86-64 Windows binary that CI built at commit 6234d74, compared with mezura 3.3.0. Each time is a median of 10 to 40 runs for files already in the file cache, and includes about 14 ms to start a process.

Repository Files Lines Settings codestats mezura
cpython 3,626 2.3M Equal work 46 ms 56 ms
kubernetes 18,399 5.6M Equal work 156 ms 168 ms
ripgrep 154 65K Default 20 ms 33 ms
cli 1,102 312K Default 44 ms 53 ms
cpython 4,947 2.5M Default 71 ms 64 ms
kubernetes 22,697 5.2M Default 238 ms 221 ms

At default settings, the tools don't count the same files. For example, mezura counts 17,108 files in kubernetes, where codestats counts 22,697. The file and line counts in the table are from codestats.

To reproduce this table, run powershell -ExecutionPolicy Bypass -File scripts/bench-windows.ps1.

License

codestats is licensed under the MIT License.

About

The fastest and most accurate code counter around: code, comment, blank, and shebang lines by language, as text, JSON, CSV, TSV, Markdown, or HTML

Topics

Resources

Stars

15 stars

Watchers

1 watching

Forks

Releases

Used by

Contributors

Languages