Skip to content

Accept plain matplotlib Axes and Figure values as JSON - #32

Draft
eliotwrobson wants to merge 2 commits into
mainfrom
claude/plain-matplotlib
Draft

eliotwrobson wants to merge 2 commits into
mainfrom
claude/plain-matplotlib

Conversation

@eliotwrobson

@eliotwrobson eliotwrobson commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Student code can now provide a plain matplotlib Axes or Figure, such as plot = plt.gca() after plt.semilogy(x, fx). The test receives a Figure, the same type as the MatplotlibSerializer path, which still works.

The data is plain JSON with no pickling. It comes from student code, so rebuilding validates every field and treats it as untrusted. Malformed data raises a ValueError.

What's carried and rejected

Carried: lines, including semilogy, loglog and step; scatter points with sizes and colors; bars and histograms; scales, labels, titles, limits, positions and legend labels; figure size and overall title.

Rejected with a clear error: errorbar, stem, fill_between, pie, imshow, contour, vlines and hlines, axhline and axvline, axline, axvspan, step histograms, text and annotations, colormapped scatter plots, anything not in data coordinates, polar axes, inset axes, subfigures, and items added directly to a figure. The error reaches the student through the "can't be sent back to the grader" message.

Decisions:

  • An Axes value returns its whole figure, so subplots are never lost and tests always get a Figure.
  • Text and annotations are rejected rather than carried, to keep this small. A correct plot with an annotation therefore fails with a clear message.
  • Twin axes come back as separate axes in the same position.
  • Lines on a category axis come back with their category names, and dates come back as matplotlib date numbers. NaN, infinity and masked values are kept as floats.
  • Not carried, and documented as such: grid lines, tick positions and labels, separate alpha, scatter marker shapes, log base and symlog settings.

Sandbox change: it now defaults matplotlib to the Agg backend, set inside the sandbox process. Student code runs on a worker thread, and plain plt failed there with macOS's GUI backend.

test_matplotlib_plots gets a longer sandbox_timeout: importing matplotlib in the new unit tests means the sandbox has to build its own font cache under pytester's temporary home, which took about 7.5s here. The same line is in #27.

Testing

  • uv run pytest tests -p no:prairielearn-grader on Python 3.14: all 147 passed on two runs. New unit tests cover round trips, rejections and malformed data, and a new scenario covers query and query_function.
  • uv run mypy --config-file pyproject.toml src tests, the command CI runs, is clean. ruff check has three new PLC0415 lazy-import findings in json_utils.py, the same pattern used there for other optional imports.
  • Full image built from this branch's docker/ files, with this branch's wheel installed over the PyPI package. I confirmed the installed matplotlib_json.py, json_utils.py and _student_code_runner.py match the branch. It has matplotlib 3.11.2; local tests used 3.10.8.
  • Graded in that image with the real entrypoint and PrairieLearn's plots test_student.py, unchanged, with student code running as ag:
Submission Score Feedback
plt.semilogy + plt.gca() 1.0 None
plt.semilogy + plt.gcf() 1.0 None
plt.plot (wrong scale) 0.0 plot does not have the correct scale for its axes
plt.semilogy of wrong data 0.0 plot is inaccurate
plt.errorbar 0.0 Variable 'plot' has a value of type Axes, which can't be sent back to the grader: errorbar plots aren't supported
Axes from two subplots 0.0 plot should have exactly one set of axes
plot = 5 0.0 plot is not a MatplotlibSerializer object
MatplotlibSerializer initial code 1.0 None
  • In the same image as ag: the new scenario scored 5/6, where the one failure is deliberate and checks the message, including PlotTester on a rebuilt figure. test_matplotlib_plots scored 12/12.

Startup as ag: HOME stays /root, which ag can't write to, so each sandbox creates a temporary matplotlib folder in /tmp and prints a warning to its stderr. Importing matplotlib.pyplot took about 0.75s. With the question's sandbox_timeout removed, the default 1s passed 5 of 5 tries on this machine, but the margin is thin, so the docs recommend a longer timeout for plot questions.

Not tested: categorical, date, NaN and subplot values are covered by unit tests only, not graded in the image. A workspace question with a restrictive import whitelist could block the lazy import of matplotlib_json, based on reading the code.

🤖 Generated with Claude Code

eliotwrobson and others added 2 commits November 5, 2026 15:29
Student code had to build plots with MatplotlibSerializer. It can now provide a
plain Axes or Figure, which the grader receives as a Figure, sent as validated
JSON data with no pickling. An Axes brings its whole figure, so subplots aren't
lost.

Lines, scatter points, bars, histograms, scales, labels, titles, limits and
legend labels are carried. Anything else, such as errorbar plots, images or text,
is rejected with a clear error rather than losing data.

The sandbox now defaults matplotlib to the Agg backend, because student code runs
on a worker thread with no display.

Importing matplotlib in the new unit tests means the sandbox has to build its own
font cache in pytester's temporary home directory, so test_matplotlib_plots needs
a longer timeout. The same change is in #27.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
matplotlib-stubs types get_yscale as an ellipsis and leaves out
Figure.get_suptitle, so annotate those values as Any.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant