[krassowski]

Python 3.12 and later give a DeprecationWarning when PtyProcess.spawn() forks a process with threads (pexpect/pexpect#817, pexpect/pexpect#827, #82), and the forked child can hang before exec (pexpect/pexpect#369). This PR starts a small Python helper with os.posix_spawn() in place of the fork, and the helper makes the pty its controlling terminal before exec.

References

• Closes use os.posix_spawn when available (i.e, 3.8) #53
• Closes PtyProcess.spawn (and thus pexpect) is not safe for use in multithreaded applications #43
• Closes PtyProcess.spawn (and thus pexpect) slowdown in close() loop #50
• Closes PtyProcess.spawn just got slower and can trigger problems #52
• Follows Use and prefer os.posix_spawn() when available #54: posix_spawn() cannot set a controlling terminal, so the helper does it, as GNOME/vte#219 proposes.
• Ports the helper of jupyter/terminado#250, so terminado can drop its copy.

The first commit adds two tests that fail on master:

Details

FAILED tests/test_fork_hazards.py::ForkHazardsTestCase::test_lock_held_by_another_thread_does_not_hang
AssertionError: 'timed out\nstarted\n' != 'started\n'
FAILED tests/test_fork_hazards.py::ForkHazardsTestCase::test_objects_of_other_threads_are_not_finalized
AssertionError: "False\n['row 0', 'row 1', 'row 2', 'row 0', 'row 1', 'row 2']\n" != "True\n['row 0', 'row 1', 'row 2']\n"

Code changes

• Adds _posix_spawn_helper.py, a script that calls setsid(), sets the controlling terminal with TIOCSCTTY, changes the directory, closes inherited file descriptors and calls exec.
• Adds _posix_spawn.py: it starts the script as python -I -S -X utf8 -c <script> with os.posix_spawn(), and raises the OSError of a failed chdir() or exec in the caller.
• PtyProcess.spawn() uses the helper. It still forks with a preexec_fn, with PTYPROCESS_USE_FORK=1, on Solaris and PyPy, in frozen applications, and when a test start of the helper fails.
• Adds restore_signals=False to spawn(), to reset SIGPIPE, SIGXFZ and SIGXFSZ in the child as subprocess.Popen does.

User-facing changes

Before, the forked child ran the finalizers of thread-local objects of other threads, and the at-fork handlers of libraries. In a process with threads, for example an IPython kernel that runs a ! command, this code could hang spawn() or change files that the parent still uses. After, no code of the caller runs in the child, and Python does not warn. pexpect passes a preexec_fn only when its caller gives one or sets ignore_sighup=True, so pexpect uses the helper by default.

A spawn now takes about 14 ms at any process size, and the first spawn in a process also runs a test start of the helper:

Process memory	master	This PR
14 MiB	1.2 ms	13.6 ms
1 GiB	31 ms	14.8 ms
4 GiB	94 ms	14.2 ms
8 GiB	155 ms	14.2 ms

Backwards-incompatible changes

None

AI usage

I used Opus 5.5 Max to assist with code writing and self-review.

CI

This PR also includes #88 which drops EOL Python versions and adds new ones. I'm happy to rebase once that gets merged.

I ran the CI on my fork (krassowski#1) and all checks are green. It is also all green on jupyter/terminado#250 (which uses a slightly different matrix).

[krassowski]

The tests all pass on krassowski#1.

@takluyver @Red-M would you be open to considering this PR, or should downstream (jupyter/terminado) move ahead without waiting here?

[Red-M]

Personally, I'm not really able to review this properly due to its size and use of LLMs, sorry.

[krassowski]

Sure, thank you for the reply!

[takluyver]

I'm interested in the concept, but it's not going to be a priority for me any time soon, and I also dislike wading through LLM-generated code. So I think go ahead downstream.

A couple of thoughts from a quick glance:

• From the link you posted, it looks like vte (Gnome's terminal emulator library) backed off using posix_spawn and still forks in 2026. Do we understand why (as in Chesterton's Fence :-) ?
• If I do one day have time & interest to do significant work on ptyprocess and want to go in this direction, I'm more tempted to deprecate features that absolutely require fork (preexec_fn) and maintain just one code path, rather than having both.