Unicode spoofing and confusable detection for Python, following Unicode Technical Standard #39. Pure Python, no runtime dependencies.
Status: alpha. Everything version 1 sets out to do is in place and passes its conformance tests; names and signatures may still change before the beta. The alphas are on PyPI:
pip install confusable==0.1.0a3(pip skips pre-releases unless the version is pinned or--preis given).
Not to be confused with confusables
(1.2.0, from 2021), which matches words that can be confused for each other, or
with confusable_homoglyphs
(3.3.1), whose API this package provides as confusable.compat.homoglyphs. Every
release of confusable is published from this repository's own workflow and
carries a provenance attestation on PyPI.
Which checks to apply to usernames, display names and filenames, and how to refuse a name that looks like one you already have, is in docs/accepting-names.md. The examples below show each check on its own.
import confusable
confusable.skeleton("p\u0430ypal") # 'paypal' (with CYRILLIC SMALL LETTER A)
confusable.match("paypal", "p\u0430ypal") # Match.MIXED_SCRIPT
confusable.match("rn", "m") # Match.SINGLE_SCRIPT
confusable.match("paypal", "paypa1x") # None: not confusable
# The skeleton follows display order, so a right-to-left override is seen
# the way a reader sees it:
confusable.skeleton("invoice\u202egpj.exe") # 'invoiceexe.jpg'Restriction levels (UTS #39 section 5.2) rank how safely a string mixes scripts,
from ASCII_ONLY to UNRESTRICTED, using the General Security Profile:
confusable.restriction_level("caf\u00e9") # RestrictionLevel.SINGLE_SCRIPT
confusable.restriction_level("abc\u3042") # HIGHLY_RESTRICTIVE: Latin with Japanese
confusable.restriction_level(
"p\u0430ypal"
) # MINIMALLY_RESTRICTIVE: Latin with Cyrillic
confusable.is_mixed_script("p\u0430ypal") # True
# Levels are ordered, most restrictive first:
confusable.restriction_level(name) <= confusable.RestrictionLevel.HIGHLY_RESTRICTIVEThe identifier checks follow UTS #39's General Security Profile, under canonical equivalence, so the answer is the same however the text is encoded:
confusable.is_allowed("paypal") # True
confusable.is_allowed("pay pal") # False: a space is Restricted
confusable.is_allowed("\u0114") # True: U+0114 is Restricted, E + U+0306 is not
confusable.identifier_types("\u200d") # {IdentifierType.DEFAULT_IGNORABLE}
confusable.isxidstart("_"), confusable.isxidcontinue("_") # (False, True)
# Persian, Malayalam and Sinhala need ZERO WIDTH (NON-)JOINER in some words. The
# profile excludes both; joining_controls=True allows them only where UTS #39
# section 3.1.1.1 says they change how the word looks:
confusable.is_allowed("\u0dc1\u0dca\u200d\u0dbb\u0dd3", joining_controls=True) # True
confusable.is_allowed("pay\u200dpal", joining_controls=True) # FalseThree more checks from UTS #39 sections 5.3 and 5.4 catch digits from different number systems and combining marks that pile up or hide:
confusable.has_mixed_numbers("1\u0661") # True: ASCII and Arabic-Indic digits
confusable.has_unlikely_marks("e\u0301\u0301") # True: the same accent twice
confusable.has_hidden_overlay("i\u0307") # True: a dot above, lost in i's own dotInvisible and control characters, for text that is not held to the identifier profile, such as display names and file names:
confusable.find_invisible("pay\u200bpal") # [(3, Invisible.DEFAULT_IGNORABLE)]: ZWSP
confusable.find_invisible("invoice\u202egpj.exe") # [(7, Invisible.BIDI_CONTROL)]
confusable.find_invisible("hi\U000e0068\U000e0069") # two Invisible.TAG: hidden "hi"
confusable.has_invisible(
"\U0001f468\u200d\U0001f469\u200d\U0001f467"
) # False: an emojiGrapheme clusters, the characters a reader sees, and truncation that never cuts one apart. A cluster can be any size, so pair a grapheme limit with a size limit:
[
str(g) for g in confusable.iter_graphemes("e\u0301\U0001f1fa\U0001f1f8")
] # ['e' + accent, a flag]
confusable.truncate(name, 32) # at most 32 graphemes
confusable.truncate(bio, 300, max_utf8_bytes=3000) # and at most 3000 bytesskeleton(text, direction) takes the paragraph direction text is shown in:
"ltr" (the default, UTS #39's skeleton), "rtl", "auto", or None for
UTS #39's internalSkeleton, which ignores display order. Skeletons are for
comparison, not display, and change with the Unicode version
(confusable.unicode_version), so recompute any you store when it changes.
confusable.compat.homoglyphs has the API of confusable_homoglyphs 3.3.1, so
existing code switches by changing one import:
from confusable.compat.homoglyphs import categories, confusables
confusables.is_dangerous("p\u0430ypal") # TrueIt returns what that package returns, with current Unicode data. It also finds
Arabic and Hebrew look-alikes, which that package's data misses: its generator
copies the invisible LEFT-TO-RIGHT MARKs that confusables.txt puts around
right-to-left characters in its comments. For new code, prefer match and
restriction_level, which follow UTS #39.
Code that cannot change its import can give confusable_homoglyphs itself this
data: python -m confusable.compat.homoglyphs export DIR writes its two data
files, and the unchanged package loads them when the CONFUSABLE_DATA
environment variable names DIR before the interpreter starts. Where an
environment variable is not an option, confusable.compat.homoglyphs.install(),
called before anything imports confusable_homoglyphs, makes that import
resolve to this module instead. The data bug, how far it reaches and the three
ways around it are explained in
docs/homoglyphs-data.md.
confusable decides whether user-supplied text is safe to accept as an identity: a
username, display name, project name, slug or filename. Version 1 covers:
- Look-alike detection with the UTS #39 skeleton algorithm, so that
pаypal(with a Cyrillicа) andpaypalcompare as confusable. - Restriction levels and mixed-script checks from UTS #39 section 5, built on the
Unicode
ScriptsandScript_Extensionsproperties. - Identifier status and type from the UTS #39 General Security Profile.
- Mixed numbers and suspicious combining marks from UTS #39 sections 5.3 and 5.4: digits from different decimal systems, repeated or stacked marks, and a dot above hidden by the letter under it.
- Invisible and control characters: default-ignorable code points, zero-width characters, bidirectional controls, and Tag-block "ASCII smuggling", while still accepting valid emoji tag sequences such as subdivision flags.
- UAX #31 identifier checks equivalent to Python 3.15's
unicodedata.isxidstart()andunicodedata.isxidcontinue(), on Python 3.10 and later. - Grapheme clusters: iteration and truncation that never split a user-perceived
character, equivalent to Python 3.15's
unicodedata.iter_graphemes(). - A compatibility module,
confusable.compat.homoglyphs, with the same API asconfusable_homoglyphs3.3.1, so existing code can switch by changing one import, checked against that package itself.
The Unicode data is pinned to a single Unicode version (18.0.0) and generated into the package, so installing needs no network access and no compiler.
A look-alike that slips through is a security bug, so the tests come first. Every
operation is checked against two independent implementations of UTS #39, ICU and
the Rust unicode-security crate, over every Unicode code point and a corpus of
known attacks and generated strings. Both oracles run at the pinned Unicode
version, and every place where they differ is explained by a specific rule. See
tools/oracles/README.md.
uv sync
uv run pre-commit install
uv run pytest
uv run pre-commit run --all-filesCHARTER.md says what the project is for and the principles every change is held to. AGENTS.md has the conventions, for people and coding agents alike.
To report a vulnerability, see SECURITY.md.
MIT. See LICENSE. The Unicode data the package and its test fixtures are generated from is under the Unicode License v3; see LICENSES/Unicode-3.0.txt.