Skip to content

Let questions name helper modules that student code can import - #34

Draft
eliotwrobson wants to merge 3 commits into
mainfrom
claude/helper-modules
Draft

eliotwrobson wants to merge 3 commits into
mainfrom
claude/helper-modules

Conversation

@eliotwrobson

@eliotwrobson eliotwrobson commented Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator

Adds ConfigObject(helper_modules=["helper_function"]), so setup code, leading_code.py and student code can run import helper_function as hf for a file in the tests folder. Today that fails with No module named 'helper_function', even when the name is in import_whitelist.

How it works

The grader reads each named file and sends its source to the sandbox, and the sandbox's importer loads it from there. The tests folder never goes on sys.path, so anything not named still can't be imported.

Rules

  • Names: plain module identifiers only, with no dots, paths or .py. Packages aren't supported.
  • Restrictions: a named module doesn't need to be in import_whitelist. A helper module's own imports and builtins follow the same rules as student code, so it doesn't widen the sandbox.
  • Configuration errors: a missing file, a syntax error, or a name that's also in import_blacklist makes the submission ungradable, with the reason in each test's message. Students don't lose an attempt.
  • Failures while loading: an error raised while a helper module runs is an ImportError naming the module, and it does count against the student's submission.
  • Workspace mode: supported. A named module's name is reserved: a student who submits a different file or a folder with that name gets a message asking them to rename it, and it counts as an attempt. An identical copy is allowed, so instructors can hand one out. A named module keeps the restricted builtins, which the student's own modules don't have. Dotted queries such as query_function("helper_function.triple") only search the student's project, so they won't call a helper module.
  • Unset: with no setting, the importer behaves as before.

Known limit

This doesn't make the tests folder private. Students can read helper modules, and importlib isn't blacklisted by default, so student code can still load files there by path. The docs say both, and this PR doesn't change those rules.

Testing

  • uv run pytest tests -p no:prairielearn-grader: 108 passed. An earlier run also had a test_matplotlib_plots failure in a new environment, and it didn't recur.
  • uv run mypy --config-file pyproject.toml src tests: clean. pyproject.toml now excludes the new scenarios' helper and leading files, which can't be resolved outside the sandbox.
  • With the folder temporarily added to sys.path, the deny tests failed for ans, test_student, setup_code and a helper importing ans, so they do detect that case.
  • The new scenarios passed on a full image built from 1.1.2 with this branch's wheel installed, as ag, with the installed files matching the branch. They cover query, query_function, query_setup, module_sandbox, whitelist and blacklist, a tagged notebook cell, workspace_sandbox with an entry point and an identical copy of a helper, and the three configuration errors. After the name clash change I re-ran only the three workspace scenarios on a rebuilt image: 8 of 8, ungradable for a missing helper, and score 0 with the rename message for a clash. The folder and non-UTF-8 clash cases only have unit tests.

Graded on that image against a scratch copy of MarkovChains-Gambler, as .ipynb submissions:

Submission Score Message
Markov-Chains-2.ipynb 1.0
Plus a #grade cell with import helper_function as hf 1.0
The same cell on published v1.1.2, with the question's original tests 0.0 No module named 'helper_function'
Plus a #grade cell with import graph, with graph.py in the tests folder but not named 0.0 No module named 'graph'
Plus a #grade cell with import test_student 0.0 No module named 'test_student'
Helper file set to mode 600, owned by root 1.0
Answer loaded with importlib from an unnamed file in the tests folder (the known limit) 1.0

No release or tag was cut, and the images need a package release before questions can use this.

🤖 Generated with Claude Code

eliotwrobson and others added 3 commits November 6, 2026 13:36
The tests folder isn't on the sandbox's import path, and import_whitelist
only permits a name, so helper files there couldn't be imported.

The new helper_modules setting names modules in that folder. The grader
sends their source to the sandbox, so the folder stays off the import path
and other files in it, such as reference answers, still can't be imported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An instructor's helper usually isn't in the student's folder, and if it's
a graded file then students can edit it. So workspace mode needs the
setting too. Imports of a named module get the instructor's file, even if
the student submits one with the same name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In workspace mode an import of a helper module silently replaced a student
file with the same name, so the student's code ran against a module they
didn't write. Fail with a message asking them to rename it. An identical
copy is allowed, since instructors may give students one to work with.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant