DMOJ Judge v4.0.0
This major release contains a number of features, sandboxing improvements, and support for the latest versions of many runtimes. Upgrading is strongly recommended.
Thanks to @Riolku, @kiritofeng, @int-y1 for contributing major changes in this release.
Breaking changes
- The minimum supported Linux kernel version is now 4.8. Pure
ptrace
-based sandboxing is no longer supported. - Support for running on Python 3.6 has been removed. Python 3.6 was EOL in December 2021.
- The
RUBY2
executor has been renamed toRUBY
. - All Java executors except for Java 8 have been merged into a single
JAVA
executor. - Support for CoffeeScript (
COFFEE
executor) has been removed. - Tests specified
pretest_test_cases
are now run unconditionally. See #1068 for more details. - The number of digits in submission output that interactors, etc. will be willing to handle has been dropped from 10000 to Python's new
sys.int_info.default_max_str_digits
limit. See python/cpython#95778 for more details. - The Rust edition has been upgraded from 2015 to 2021.
Deprecation notices
- Problem directories are now specified through a new
problem_storage_globs
key.problem_storage_root
is now deprecated, and will be removed in a future release. - This release adds support for batch dependencies, but their functionality will change substantially in a future release. See #1075 for more details.
Runtimes
- Added support for:
- Lean 4
- LLVM IR
- Java 17-19
- Go 1.16+
- Ruby 3
- OCaml 5.0
- @int-y1 has implemented an optimizing compiler for Brain**** using LLVM (#1091). This new compiler produces dramatically faster code than the previous implementation.
- Rust compilation has been greatly sped up by sharing build artifacts between compilations.
- Fixed Groovy compilation time being included as part of runtime.
- Tweaked sandbox to allow Go to make use of its transparent hugepage support.
- Carriage returns are now stripped for the text executor.
- The time limit for Kotlin compilation has been raised from 20 seconds to 30 seconds.
Sandbox
- Compilers are now sandboxed. Not sandboxed: renaming, various
f*
syscalls (e.g.fchmod
),exec
s. The current level of compiler sandboxing is intended to prevent e.g.#include
-ing arbitrary files that are readable by the judge user. - File access is now denied with
EACCES
rather thanEPERM
. - Java security policies are no longer used, as they were deprecated in Java 17.
- Fixed non-fatal signals occasionally being reported to the site (e.g., Haskell's ticker thread sending
SIGVTALRM
). - Added support for FreeBSD 13.
Miscellaneous
- The judge will now autodetect executors for given source files based on extension. This applies to
dmoj-cli submit
as well as generators, graders, etc. - Fixed judge handshake issues when ~9000 problems are reported at initialization.
- Fixed leaking cached executor compilation artifacts.
- Context switches (involuntary + voluntary) are now reported as part of testcase log output.
Full changelog
The full changelog is available here as well as below.
Changelog
- cptbox: remove ptrace-only tracing by @quantum5 in #900
- readme: document removal of pure ptrace sandbox by @quantum5 in #904
- cptbox: always set PR_SET_NO_NEW_PRIVS by @Xyene in #908
- cptbox: remove last vestiges of WSL 1 support by @Xyene in #907
- executors: disallow /dev/std{in,err,out} by @Xyene in #909
- Change EPERM into EACCES for file access by @Riolku in #911
- executors: add support for Java 17 by @Xyene in #914
- Drop security policies for JVM runtimes by @Xyene in #913
- cptbox: make on_return work with multiple threads by @quantum5 in #915
- executors: make go executor with with 1.16+ by @quantum5 in #916
- fix java executors VM config detection by @Riolku in #917
- testsuite: increase memory limit to avoid flaking on FreeBSD by @quantum5 in #922
- cptbox: make IsolateTracer._get*_pid work with multiprocess by @quantum5 in #924
- cptbox: remove #ifdef for PTRACE_O_EXITKILL by @quantum5 in #923
- cptbox: implement Debugger.readbytes by @quantum5 in #925
- Sandbox compilers by @Xyene in #906
- cptbox: implement proper sys_utimensat sandboxing by @quantum5 in #926
- Improve dmoj.utils.* by @quantum5 in #928
- executors: implement sys_fchmodat sandboxing for compilers by @quantum5 in #929
- ci: update upload-pypi workflow dependencies by @quantum5 in #921
- cptbox: make sys_utimensat emulation more accurate by @quantum5 in #927
- judge: fix str/bytes confusion in CompileError by @quantum5 in #930
- cptbox: ignore ESRCH from post_syscall handler by @quantum5 in #937
- cptbox: use ptrace(PT_IO) on FreeBSD by @quantum5 in #938
- cptbox: drop support for FreeBSD < 8 by @quantum5 in #935
- executors: remove FIXME about posix_fadvise and posix_fallocate by @quantum5 in #940
- commands: add mypy types by @quantum5 in #934
- cptbox: correctly handle sys_execve being allowed with seccomp by @quantum5 in #939
- utils: add types to parse_helper_file_error by @quantum5 in #932
- Implement fchmod, rename(at) sandboxing for compilers by @quantum5 in #933
- cptbox: use correct result registers for arm64 by @quantum5 in #943
- executors: improve /usr handling by @quantum5 in #944
- config: fix ConfigNode.get fallback logic by @quantum5 in #945
- ci: fix ARM64 docker test by @quantum5 in #946
- testsuite: reduce memory limit for C# MLE test by @quantum5 in #942
- executors: improve types by @quantum5 in #931
- ci: run tests on arm64 by @quantum5 in #948
- ci: allow less executors to fail by @quantum5 in #953
- executors: use shutil.which instead of deprecated distutils by @quantum5 in #949
- ci: remove useless functions from citest.py by @quantum5 in #950
- cptbox: clang-format by @quantum5 in #941
- ci: include _checker.c and setbufsize.c in clang-format by @quantum5 in #954
- ci: drop Travis by @Xyene in #955
- Add PR #941 to .git-blame-ignore-revs by @Xyene in #956
- cptbox: support
subprocess.STDOUT
by @Xyene in #961 - Update Linux syscall list by @dmoj-build in #963
- executors: fix returning
str
instead ofbytes
fromget_compile_output
by @Xyene in #960 - executors/PAS: drop -So argument by @Xyene in #959
- executors: add basic autodetection from filename by @Xyene in #968
- ci: use Python 3.10 when possible by @quantum5 in #970
- Use metaclass to auto-populate Executor.name by @quantum5 in #969
- cptbox: check faccessat2 instead of always denying by @quantum5 in #975
- cptbox: automatically generate FreeBSD syscalls by @quantum5 in #976
- Fix latest runtime failure with Racket by @quantum5 in #977
- cptbox: allow statx by default by @quantum5 in #978
- Work around pip bug by @int-y1 in #979
- Add executor for Lean 4 by @int-y1 in #982
- Update docker test by @int-y1 in #983
- cptbox: allow FreeBSD 13 syscalls by @quantum5 in #985
- ci: improve automatic syscall PR generation by @quantum5 in #987
- Use socket.sendall to send packets by @hieplpvip in #984
- Implement debug mode with debug logging level by @quantum5 in #989
- Remove Python 3.6 support by @quantum5 in #991
- cptbox: implement setting child affinity by @Xyene in #992
- cptbox: make
_file_access_check
syscall-agnostic by @Xyene in #993 - cptbox: track and report context switches by @Xyene in #994
- Exclude context switches before initial exec by @quantum5 in #995
- Isolate refactor by @Riolku in #996
- executors: add support for Ruby 3 by @Xyene in #998
- cptbox + executors: cleanup IsolateTracer signature by @Riolku in #1005
- fix syscall name in log by @Riolku in #1000
- cptbox: remove legacy PTBOX_HANDLER_STDOUTERR by @Riolku in #1006
- executors: remove all regex path checks by @Riolku in #1004
- testsuite: add test for opening format string by @Riolku in #1003
- cptbox: add typing and fix typing on rule list by @Riolku in #1008
- cptbox: strip carriage returns in TEXT by @Riolku in #1011
- executors: increase compiler TL for Pascal by @Riolku in #1018
- executors: remove .gforth-history by @Riolku in #1016
- Re-comment logic for new fallback scheme by @Riolku in #1015
- cptbox: don't allow paths that don't exist by @Riolku in #1010
- Report context switches and runtime version to bridge by @quantum5 in #1012
- executors: add compiler syscalls option to CompiledExecutor by @Riolku in #1017
- judge: add support for test case prerequisites by @Riolku in #1013
- remove goto for failing seccomp by @Riolku in #1014
- judge: rename requirements to dependencies by @Riolku in #1019
- testsuite: remove some seed2 tests by @int-y1 in #1025
- Upgrade black to 22.3 by @Riolku in #1029
- testsuite: rename
requirements
todependencies
by @Riolku in #1023 - judge: pass
result
to checkers by @Riolku in #1021 - testsuite: increase TL on java aplusb ac by @Riolku in #1030
- Update syscall lists by @dmoj-build in #986
- Allow rseq syscall by @alexwaeseperlman in #1031
- Fix Groovy slow execution by @int-y1 in #1033
- Type
setproctitle
by @kiritofeng in #1038 - executors/COFFEE: allow
capget
by @Xyene in #1043 - executors/GO: allow
setrlimit
by @Xyene in #1041 - executors/DART: allow
msync
by @Xyene in #1042 - executors/SWIFT: allow read access to
/proc/self/cmdline
by @Xyene in #1044 - executors/GO: allow
setrlimit
for the compiler by @Xyene in #1045 - executors: deprecate CoffeeScript by @Xyene in #1046
- executors/GO: allow
setrlimit
for the Go runtime too, again by @Xyene in #1047 - executors/SWIFT: EPERM
setpgid
by @Xyene in #1049 - executors/GO: allow Go to tell if it's running with THPs by @Xyene in #1048
- executors/SWIFT: don't allow
setpgid
at all by @Xyene in #1050 - Update syscall lists by @dmoj-build in #1034
- Clean up cache when worker finishes by @kiritofeng in #1040
- Drop
INT_MAX_NUMBER_DIGITS
to match Python's limit by @kiritofeng in #1052 - isolate: globally allow
clone3
by @Xyene in #1053 - Clean up README by @int-y1 in #1037
- Fix minor typo by @int-y1 in #1039
- Make
INT_MAX_NUMBER_DIGITS
backwards compatible by @kiritofeng in #1055 - Print seccomp errors to parent stderr by @quantum5 in #1054
- Use /code paths for python in arm64 CI by @quantum5 in #1056
- executors: merge Java executors by @Xyene in #1057
- isolate: allow sanitized
tkill
by @Xyene in #1058 - Print versions during autoconfig; fixes #644 by @kiritofeng in #1051
- executors/KOTLIN: increase compiler time limit by @kiritofeng in #1059
- Fix signal feedback on IR by @Xyene in #1061
- executors/RUST: Simplify TOML + Lock by @Riolku in #1065
- executors/RUST: update package versions by @Riolku in #1062
- Rust Edition & Test Program Upgrade by @Riolku in #1066
- ci: update actions by @Xyene in #1067
- grader: always run
pretest_test_cases
if available by @Xyene in #1070 - ci: test on Python 3.11 by @Xyene in #1073
- Report Python 3.11 support by @Xyene in #1074
- Fix
no_implicit_optional
mypy error by @Riolku in #1078 - executors/RUST: build rust into shared directory by @Riolku in #1077
- executors/RUST: remove DMOJ_CARGO_OFFLINE by @Riolku in #1079
- Remove
bool_env
by @kiritofeng in #1080 - executors/RUST: Use per-executor shared directory by @Riolku in #1081
- testsuite: add README for rust bridged test by @Riolku in #1082
- executors/RUBY: extend search path to cover Ruby 3.1 by @Xyene in #1085
- executors/OCAML: grant 256 MiB address space grace by @Xyene in #1086
- Implement LLVM-IR executor LLC by @quantum5 in #1087
- Update build shield to use new path; fixes #1088 by @kiritofeng in #1089
- Make LLC executor work on non-amd64 platforms by @quantum5 in #1092
- executors/BF: optimize BF using LLVM by @int-y1 in #1091
- executors/BF: clean up opt command by @int-y1 in #1094
- Fix mypy breakage due to watchdog 2.2.1 by @kiritofeng in #1095
- isolate: globally allow epoll syscalls by @Xyene in #1096
- Use problem globs to declare problem storage roots by @kiritofeng in #728
New contributors
- @hieplpvip made their first contribution in #984
- @alexwaeseperlman made their first contribution in #1031