Repository navigation
Python support: transpiled Python on every target, Python servers, Pillow and font-from-data - #6128
Merged
Merged
Conversation
Python 3 sources transpile to Java 17 (maven/python-transpiler) and run on maven/python-runtime on every Codename One target. Generators, async, match, dataclasses, the builtins and a stdlib wave are covered by a conformance suite diffed against CPython on the JVM and natively through ParparVM. Constructs a translated program cannot run (eval, exec, compile, __import__, frame hooks) are build errors naming file and line. ParparVM: a frameless method returning a value after several try blocks that end on one label ran a single END_TRY and left the other blocks on the thread's try stack, so a later throw longjmp'd into a returned frame. The folded return paths now unwind the stack (FramelessTryReturnIntegrationTest). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
from java.<package> import <Class> resolves at build time against the compile class path; a generated JavaInterop bridge dispatches with instanceof and direct calls (no reflection), converts values both ways, adapts Python callables to functional interfaces and exposes Java collections to len/in/iteration/indexing. cn1:import-python-project and cn1:transpile-python (Maven), the transpilePython task (Gradle), and the archetype's python profile build src/main/python. runMain returns the exit status, honouring SystemExit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…vice library from java.java.* resolves against the class path's own class library (java-runtime, JavaAPI) when it carries one, so an API the device lacks is a build error rather than a bridge that fails to link. python-project-test.sh imports scripts/python-samples/hello-ui (a Codename One UI written in Python), builds it with Maven and Gradle and stages every target; python-project.yml runs it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PyHttp is the client the packages are written over: ConnectionRequest in an application, installable like PyFileSystem. requests (verbs, Session, auth, params/json/form bodies, cookies, exceptions) and httpx (verbs, Client, AsyncClient, URL, Headers, HTTPStatusError) are our own Python over it; http_requests and http_httpx match the real packages' output against the same test server. BaseException.__new__ ignores keyword arguments, as CPython's does, so an exception subclass whose __init__ takes keywords can be raised. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…_type_hints Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lice.indices The HTTP client is installed by the generated registry only when the program bundles requests or httpx, so other programs do not link the network stack. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ule __dict__ Fixes found while porting attrs and pydantic: an annotation built by a call (Annotated[int, Field(gt=0)], conint(ge=2)) was stored as None; a class __delattr__ was bypassed by del; classes had no __hash__ attribute; and type(name, bases, ns) gave __module__ 'builtins'. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
_cn1server.serve(dispatch, host, port) bridges a Python dispatch callable to the backend HttpServer when the backend is on the transpile class path; scripts/python-backend/build.sh builds such an app as a native binary. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Clean-room site packages flask, werkzeug.exceptions, fastapi, starlette (responses, requests, testclient, status) and uvicorn. Every app exposes dispatch(method, path, query, headers, body) -> (status, headers, body); app.run / uvicorn.run hand it to _cn1server.serve. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ions inspect.isgeneratorfunction/iscoroutinefunction/isasyncgenfunction now answer from __code__ as CPython's do; FastAPI's yield dependencies use it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pter Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…indings Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… helper returns CommonWorkloads ported to Python (scripts/python-bench/common) with a Java hot-spot variant, and a comparison across CPython, PyPy, GraalPy and Codename One with peak RSS per workload. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mance chapter The developer guide's Python chapter compares CPython, PyPy, GraalPy and Codename One on startup, kernels, Flask and the CommonWorkloads suite, with peak memory in every table, two charts, and a section on moving a measured hot spot into Java. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
IntRanges runs interval analysis over each typed function: loop counters are bounded by their range, accumulators by trip count times their per-pass growth, and masked values by the mask. A local whose every value fits in 64 bits becomes a Java long, and an int expression whose result fits is computed with wrapping long arithmetic, which equals the exact result however large the intermediates are. Expression helpers such as i32/i64 are inlined, module int constants are folded, and >> of a value masked into [0, 2^64) becomes >>>. Builtins without a typed lowering (print, sorted, ...) are now called generically from typed code instead of leaving the whole function untyped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nd //, lowered loop tests Locals that only ever hold a freshly constructed object whose __init__ copies its arguments into fields, and are only read through fields or getters, become one local per field. while tests lower their arithmetic and compares like any statement. for c in str walks code points, with ord(c) the code point itself. l.extend([v] * n) fills in place. Lone surrogates index and repr as CPython does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ped builtins over BOT, dict clear/len intrinsics Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… int results held bare, taken in the slow path Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t stores per instance, try-free dict key hashing Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-top arithmetic over impure calls Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ct clear; direct dict len Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… box the list Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ll dicts scan A virtual thread's call-stack, try-block and pending-allocation buffers now come from one packed mmap that is unmapped when the retired state is freed, and each retirement charges the BiBOP trigger so thousands of dead states cannot pile up between cycles (Flask under load held ~4,000 of them, ~300 MB resident). PyDict allocates nothing until first insert and keeps no index table at or below eight entries, scanning keys linearly instead. Words.next keeps its loop conditions free of stack juggling. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The semantic analyzer's merge compared values with SimdAllocaValue.equals, which refuses ASM's own BasicValue constants, so a long pushed by LCONST_1 meeting the checker's long at a join fell to BasicInterpreter.merge and became a one-slot UNINITIALIZED_VALUE; the next DUP2 failed "Illegal use of DUP2" on generated Python that ASM's plain analyzer accepts. Merge now compares types. The typed float path for ** emitted Math.pow, which CLDC lacks; more stdlib functions are typed now, and the demos' compliance check caught 30 such calls. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pinning the GC JFR on the JVM build of the Flask sample (escape analysis off) put the request path at 17.7 KB allocated per request; it is now 12.7 KB on the JVM, and the native binary's peak RSS under ab -c 8 fell from 765-816 MB to 219-220 MB with no GC tuning. - Flask shim: Headers keeps folded names beside the pairs; routing skips a rule whose literal prefix does not match before running its regex, and makes the 405 set only when needed; prebuilt JSON encoders; lazy flask.g; no copies of the header list or the blueprint chain per request. - json: unescaped strings quote without a char list; sort_keys sorts pairs without a lambda; the circular check is an identity stack, not an id() dict. - runtime: regex scratch (text, registers, stacks) reused per program via an atomic hand-off; str.startswith/endswith intrinsics and in-place affix; str < str compares without wrapping; str iteration and indexing share Latin-1 one-char strings; dict literals presized, first growth 4. - backend: a connection's deferred-JSON sink and target memo table are made on first need, so a one-request connection allocates neither. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…66% survival The collector sat at its peak footprint for 30 seconds after load stopped, because only allocation or the long idle starts a cycle. The first quiet pass after a busy one now idles 1s, so a quiet cycle runs the major sweep, page trim and malloc relief. The backend build grows the trigger only at 66% survival instead of 25%, so the heap settles near 1.5x the live set rather than 4x. Flask sample, 3 interleaved runs: footprint after load 137-191MB -> 75-98MB, 5s later 101-139MB -> 68-70MB, at 27.7-29.2k -> 25.7-25.9k req/s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…duty cycle A fixed 66% cost the Flask server 9% of its throughput. The threshold now moves from CN1_BIBOP_TIGHT_SURVIVAL_PERCENT (backend: 66) when the collector is idle for 90% of wall time or more, to the VM's 25% when it is busy 40% or more. Default builds set both equal and are unchanged. Full speed: 29.1-29.7k req/s vs 29.6-30.1k default and 26.0-26.9k fixed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cales with a small budget Under a per-process budget the survival threshold is the tight one regardless of the collector's duty cycle. The 64MB admission margin becomes a quarter of the budget below 256MB: under 128MB the Flask server went from 478 to 16.3k req/s at a 75MB footprint. Budgets of 256MB or more are admitted as before. compare.py runs every server for the same 30s, reports physical footprint after load, 5s later and at peak, and runs Codename One both unconstrained and under a --budget-mb budget (default 128). render_charts.py adds the throughput-against- memory chart. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nd under a budget The comparison now runs every Codename One binary twice: as shipped, where the adaptive collector spends spare memory for throughput, and under a 128 MB per-process budget, where it keeps the heap tight. Memory is physical footprint for every runtime; Flask gets 30 s of load and a settle read. Adds the throughput-vs-footprint chart. Also routes the transpiler's float pow through Py.powd (Math.pow): MathUtil.pow had made nbody 4x slower. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…on Windows Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Per-thread pacing claims lived in __thread storage, and a virtual thread waiting for budget can resume on another host, so claims leaked until no allocation was admitted and a budgeted server crawled. The claim is now one global value stamped with the GC epoch, expiring after two cycles. Retired virtual-thread stacks are released under a budget. The comparison runs the 128 MB budget only: Flask under 64 MB cannot hold the cap, since its unpaced virtual-thread stacks and malloc swing past it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # docs/developer-guide/languagetool-accept.txt
Collaborator
Author
|
Compared 193 screenshots: 193 matched. Benchmark Results
Build and Run Timing
Detailed Performance Metrics
|
…docs: Vale test_virtual_thread.c links cn1_virtual_thread.c without cn1_globals.m, so the new cn1ProcessHasMemoryBudget() reference failed the link. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # .github/workflows/pr.yml
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
viewSafeAreaInsetsDidChange calls it on Mac Catalyst and tvOS too, but it was defined only in the iPhone layout branch, so the Metal Catalyst build failed with -Wobjc-method-access. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # Ports/iOSPort/nativeSources/CodenameOne_GLViewController.m # scripts/developer-guide/check-guide-api-names.py
…e fairly CPython and PyPy served the same ~690 requests a second under app.run() although PyPy runs Flask twice as fast: Werkzeug's development server drains the socket for 10 ms after every response, and ab counts a request done when the socket closes. The guide keeps that chart, marks it as suspicious, tells how the cause was found, and adds the fair comparison: compare.py now also serves each interpreter behind gunicorn with 8 workers, footprint summed over every process. flask_cost.py times Flask alone, without a server. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ide's accept list Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The base row is a single run; the branch's VM runtime measures identical to master's on the self-hosted translator (interleaved A/B, reason in the file). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ture The multi-window default scenario was captured before its first paint (skin and form solid black) because the menu bar alone passes the flat-image check; single-window mode already waits that out, multi mode had no equivalent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The iOS accessibility thread contract (on IOSNative.m changes) and the Xcode leg of the Gradle/Maven parity check queued on hosted macos-15 for every matching pull request, and were dropped unrun when that pool was short. Both now run nightly, plus on a pull request touching their own tree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
|
Compared 193 screenshots: 193 matched. Benchmark Results
Build and Run Timing
Detailed Performance Metrics
|
…nutes The x64 suite allocated a fixed 12G swap file and died 'No space left on device' two minutes into the C build on a runner with little free disk; the swap now leaves 40G for the build. build-ios-tv measured 15-22 min of build plus 28-33 min of tests and was cancelled at its 60 minute limit mid-run on this and other branches. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Python support for Codename One: Python sources are transpiled to Java and run on every target, client and server.
What is in it
maven/python-transpilerandmaven/python-runtime: the transpiler, the runtime and the bundled stdlib and site packages, with ~125 conformance programs whose expected output is real CPython output.build-engine(Maven goal, Gradle task, project importer, PyPI resolution, C-extension stubs) and thepython.virtualThreadStackbuild hint /@VirtualThreadStack.com.codename1.backend.image: raster, JPEG/PNG, TrueType text); on clients over Codename One'sImage/ImageIO.Font.createTrueTypeFontFromData/isTrueTypeStreamSupportedon JavaSE, Android, iOS/Mac, JavaScript, Windows and Linux;Databasecolumn types.sslmode=verify-cawith a CA file (plus the MySQL spellings).io/gzip/InfCodes) rejected valid deflate streams whose last code ended a few bits into the final byte (about half of all gzip files, depending on the header mtime). Fuzzed againstjava.util.zipwith 160k streams fed in chunks.Verification
TrueTypeFontFromDataTestin the JS suite covers it here), the Windows/Linux C font changes (no toolchain locally) and the iOS private-API scan.🤖 Generated with Claude Code