Repository navigation
Let questions name helper modules that student code can import - #34
Draft
eliotwrobson wants to merge 3 commits into
Draft
eliotwrobson wants to merge 3 commits into
eliotwrobson wants to merge 3 commits into
Conversation
The tests folder isn't on the sandbox's import path, and import_whitelist only permits a name, so helper files there couldn't be imported. The new helper_modules setting names modules in that folder. The grader sends their source to the sandbox, so the folder stays off the import path and other files in it, such as reference answers, still can't be imported. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An instructor's helper usually isn't in the student's folder, and if it's a graded file then students can edit it. So workspace mode needs the setting too. Imports of a named module get the instructor's file, even if the student submits one with the same name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In workspace mode an import of a helper module silently replaced a student file with the same name, so the student's code ran against a module they didn't write. Fail with a message asking them to rename it. An identical copy is allowed, since instructors may give students one to work with. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
ConfigObject(helper_modules=["helper_function"]), so setup code,leading_code.pyand student code can runimport helper_function as hffor a file in the tests folder. Today that fails withNo module named 'helper_function', even when the name is inimport_whitelist.How it works
The grader reads each named file and sends its source to the sandbox, and the sandbox's importer loads it from there. The tests folder never goes on
sys.path, so anything not named still can't be imported.Rules
.py. Packages aren't supported.import_whitelist. A helper module's own imports and builtins follow the same rules as student code, so it doesn't widen the sandbox.import_blacklistmakes the submission ungradable, with the reason in each test's message. Students don't lose an attempt.ImportErrornaming the module, and it does count against the student's submission.query_function("helper_function.triple")only search the student's project, so they won't call a helper module.Known limit
This doesn't make the tests folder private. Students can read helper modules, and
importlibisn't blacklisted by default, so student code can still load files there by path. The docs say both, and this PR doesn't change those rules.Testing
uv run pytest tests -p no:prairielearn-grader: 108 passed. An earlier run also had atest_matplotlib_plotsfailure in a new environment, and it didn't recur.uv run mypy --config-file pyproject.toml src tests: clean.pyproject.tomlnow excludes the new scenarios' helper and leading files, which can't be resolved outside the sandbox.sys.path, the deny tests failed forans,test_student,setup_codeand a helper importingans, so they do detect that case.ag, with the installed files matching the branch. They coverquery,query_function,query_setup,module_sandbox, whitelist and blacklist, a tagged notebook cell,workspace_sandboxwith an entry point and an identical copy of a helper, and the three configuration errors. After the name clash change I re-ran only the three workspace scenarios on a rebuilt image: 8 of 8, ungradable for a missing helper, and score 0 with the rename message for a clash. The folder and non-UTF-8 clash cases only have unit tests.Graded on that image against a scratch copy of
MarkovChains-Gambler, as.ipynbsubmissions:Markov-Chains-2.ipynb#gradecell withimport helper_function as hfNo module named 'helper_function'#gradecell withimport graph, withgraph.pyin the tests folder but not namedNo module named 'graph'#gradecell withimport test_studentNo module named 'test_student'importlibfrom an unnamed file in the tests folder (the known limit)No release or tag was cut, and the images need a package release before questions can use this.
🤖 Generated with Claude Code