Repository navigation
Load data.json in the grader images and add plot-serializer - #28
Merged
Merged
Conversation
Plot grading depends on plot_serializer, which was only a dev dependency, so it failed in the full image. Also correct the docs: plots must be built with MatplotlibSerializer, because a plain Figure isn't serialized. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PrairieLearn writes /grade/data/data.json, but the data_json fixture only read data.json next to the test file and nothing put it there, so tests using data_json failed with a NoneType error and correct submissions scored 0. run.sh now passes the path with --data-json-path. The option uses the --data-json-path=VALUE form because pytest's first pass over the arguments would treat a separate value as a test path, which moves rootdir and loses the results file. Without the option, or if the file doesn't exist, the fixture reads data.json next to the test file. data.json can hold correct_answers, so run.sh makes it root-owned with owner-only modes before student code runs, and stops grading if that fails. Root ownership matters because the student uid can match the host owner of mounted files, and owner-only modes matter because student code keeps root's supplementary groups. The fixture now raises a clear error for a data.json it can't parse or read, instead of treating it like a missing file. run.sh takes the job directory as an optional argument so tests can run it, and no longer exports JOB_DIR or MERGE_DIR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
run.sh isn't Python, so it doesn't get its own test file. The optional job directory argument only existed for that test, so /grade is fixed in the script again, and protect_data_json no longer has to handle running as non-root. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Moving the lookup out of the try block meant a test module without a file raised an AssertionError instead of getting None, as it did before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
run.sh still moves a data.json from /grade/tests to that location, and the fixture still falls back to it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pytest's first pass over the arguments treats a separate value as a possible test path, so use the same form as --data-json-path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
eliotwrobson
marked this pull request as ready for review
November 5, 2026 16:36
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two grader image fixes, split out of #27. The
query_setupwork stays there.Changes
Load PrairieLearn's
data.json. Thedata_jsonfixture only looked next to the test file, so it never found/grade/data/data.json. Tests usingdata_json["params"]failed, and correct submissions scored 0.run.shnow passes--data-json-path=/grade/data/data.json. Without the option, or if that file doesn't exist, the fixture readsdata.jsonnext to the test file.data.jsoncan containcorrect_answers, sorun.shmakes it root-owned with owner-only modes before student code runs. If that fails, grading stops with a failure result.data.jsonthat can't be parsed or read now gives a clear error instead of being treated as missing.run.shno longer exportsJOB_DIRorMERGE_DIR.Add
plot-serializer==0.2.3to the full image. Plot grading needs it, and it was only a dev dependency. The docs now say plots must be built withMatplotlibSerializer.Release together: this
run.shfails on package 1.1.0 because it doesn't know the new option.Testing
mypyis clean, andruff checkshows nothing new.test_matplotlib_plotscan fail on the first run in a fresh environment, as it does onmain.aggetsPermission deniedondata.json. With a read-only data directory, student code never starts.run.sh. Docker Desktop's macOS bind mount doesn't enforce the modes. Automated tests don't cover--data-json-pathorrun.sh.Known remaining issues
agcan still read the test files, and can reachparamsin sandbox memory.drop_privilegesdoesn't callsetgroups.agcould likely modify the moved test files, includingsetup_code.py. This wasn't tried.🤖 Generated with Claude Code