Features#

Uoabot collects homework submissions from GitHub and grades them in an isolated sandbox. The feature set is split between the three tools

All modules come with dedicated logging, that reveals only what’s needed in stdout, while it writes more important stuff in persistent log files, that are consistently rotated (Batch mode only logs to stdout).

Fetcher#

  • GitHub organization mirroring — Pulls every repository of an organization into a local tree, per homework.

  • Cherry picked paths — Only the files listed in the homework config are copied, not the whole repository.

  • Incremental diff — Patches are applied incrementally based on the previous commit hash.

  • Divergence detection — The fetcher recognizes when to do a full re-pull instead of applying a corrupt diff.

  • Per-homework configuration — Each homework carries its own fetcher_config.json.

  • Metadata — The fetcher keeps metadata about a student repository, such as pushed_at and created_at.

  • Concurrency-safe — Fetcher establishes an exclusive flock, making it so no other reads/writes can occur while fetching.

Grader#

  • Three-level test hierarchy — Tests are organized in three levels: suites, categories, and tests. A suite typically opens with a dedicated compilation category whose artifacts feed the execution categories.

  • Dependencies — Categories and tests can require earlier ones to have passed first. a failed prerequisite skips the dependent work, and sequential categories chain tests implicitly.

  • Fine I/O control — Feed or check stdin, stdout, stderr or any file descriptor, with text, files, or regex expectations. contents can be delivered as a real file or streamed through a pipe, and comparisons run in strict or loose sensitivity.

  • Artifact handling — outputs that pass a check are preserved under /artifact and handed to later tests, so one test’s binary or result feeds the next. preserved artifacts sit on PATH and can be run by bare name.

  • Visibility control — results can be public (failure details), private (pass/fail only), or grouped (one summary row per category).

  • Partial grading — --specific runs a single suite, category or test, executing only the non-suppressible dependency pre-runs.

  • Per-homework configuration — Each homework carries its own grader_config.json.

Sandboxing (Makis)#

  • Full isolation — Every test runs inside a bubblewrap sandbox with its own user, PID, network and cgroup namespaces.

  • No network access — A submission doesn’t have any access to the internet.

  • Single writable scratch area — /scratch is the only writable location, so a run can be thrown away without touching the rest of the system.

  • Least-privilege execution — Commands run as an unprivileged user (uid 1000) inside the sandbox.

Uoabot#

  • Wrapper of everything — Uoabot alone manages fetching and grading, without needing someone to separately call the python modules themselves.

  • Submission mirrors — --mirror creates a numbered copy of the repository (-mirror1, -mirror2, …). An instructor can make changes to this “repository” and still grade it. The copy is excluded from later fetches and batch grading runs. --purge_mir removes all snapshots of a repository.

  • Batch mode — One invocation fetches every repository of a homework and grades each submission in turn, skipping mirrors, then reports a summary of succeeded, failed and skipped repos.

  • Dedicated log viewer — Throught the uoabot wrapper, one can see through a cli interface, every log produced by each module seperately.