Implement and document a new API

.github/workflows/CI.yml

@@ -23,13 +23,28 @@ jobs:
       fail-fast: false
       matrix:
         version:
-          - '1.10'
           - '1.6'
+          - '1'
           - 'nightly'
         os:
           - ubuntu-latest
+          - macos-latest
+          - windows-latest
         arch:
           - x64
+        include:
+          - version: '1'
+            os: ubuntu-latest
+            arch: x86
+          - version: '1.7'
+            os: ubuntu-latest
+            arch: x64
+          - version: '1.8'
+            os: ubuntu-latest
+            arch: x64
+          - version: '1.9'
+            os: ubuntu-latest
+            arch: x64
     steps:
       - uses: actions/checkout@v4
       - uses: julia-actions/setup-julia@v2

LICENSE

@@ -1,7 +1,7 @@
 MIT License
 
 Copyright (c) 2016-2021: Michael Hatherly, Morten Piibeleht, Fredrik Ekre, and Documenter.jl contributors
-Copyright (c) 2024 Morten Piibeleht <[email protected]> and contributors
+Copyright (c) 2024: Morten Piibeleht <[email protected]> and contributors
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

Project.toml

@@ -1,10 +1,11 @@
 name = "CodeEvaluation"
 uuid = "5a076611-96cb-4f02-9d3a-9e309f06f8ff"
-authors = ["Morten Piibeleht <morten.piibeleht@juliahub.com> and contributors"]
+authors = ["Morten Piibeleht <morten.piibeleht@gmail.com> and contributors"]
 version = "0.0.1"
 
 [deps]
 IOCapture = "b5f81e59-6552-4d32-b1f0-c071b021bf89"
+REPL = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb"
 
 [compat]
 IOCapture = "0.2"

README.md

@@ -3,14 +3,114 @@
 [![Build Status](https://github.com/JuliaDocs/CodeEvaluation.jl/actions/workflows/CI.yml/badge.svg?branch=main)](https://github.com/JuliaDocs/CodeEvaluation.jl/actions/workflows/CI.yml?query=branch%3Amain)
 [![PkgEval](https://JuliaCI.github.io/NanosoldierReports/pkgeval_badges/C/CodeEvaluation.svg)](https://JuliaCI.github.io/NanosoldierReports/pkgeval_badges/C/CodeEvaluation.html)
 
+A small utility package to emulate executing Julia code, seemingly in a clean `Main` module.
+
 > [!NOTE]
-> This package is in active development, and not yet registered.
+> This package is in active development.
+
+## API overview
+
+There are two main parts to the API:
 
-A small utility package to emulate executing Julia code in a clean `Main` module.
+1. The `Sandbox` object: provides a clean, mock Julia Main module, and the related low-level `evaluate!` function to directly evaluate Julia expressions in the sandbox.
 
-The package uses [IOCapture.jl](https://github.com/JuliaDocs/IOCapture.jl) to perform output capture of the evaluated code.
+2. Higher level functions that can be used to run code in the sandbox in different modes (`codeblock!` and `replblock!`).
+
+> [!NOTE]
+> The functions that run code in a sandbox are marked with `!` because they mutate the sandbox.
 
 > [!WARNING]
-> The code evaluation is not thread-safe.
-> This is because for each evaluation, the code has to change the Julia processe's working directory with `cd`.
-> This global change will also affect any code running in parallel in other tasks or threads.
+> The code evaluation is not thread/async-safe.
+> For each evaluation, the code has to change the Julia process' working directory with `cd`.
+> This also affects any code running in parallel in other tasks or threads.
+
+> [!NOTE]
+> This is just a high-level overview.
+> See the docstrings for more details!
+
+### Sandbox
+
+Constructing a `Sandbox` object provides you
+
+The `evaluate!` function can be used to evaluate Julia expressions within the context of the sandbox module.
+It returns a `Result` object that contains the captured return value and what was printed into the standard output and error streams.
+
+```julia-repl
+julia> sb = CodeEvaluation.Sandbox();
+
+julia> r = CodeEvaluation.evaluate!(sb, :(x = 40 + 2));
+
+julia> r.value, r.output
+(42, "")
+
+julia> r = CodeEvaluation.evaluate!(sb, :(println("x = " * string(x))));
+
+julia> r.value, r.output
+(nothing, "x = 42\n")
+```
+
+As an implementation detail, it uses the [IOCapture.jl](https://github.com/JuliaDocs/IOCapture.jl) package underneath to perform output capture of the evaluated code.
+
+As an asterisk, as the sandboxes are implemented as anonymous Julia modules, all within the same Julia process, there are limitations to to their independence (e.g. method definitions and other global state modifications can, of course, leak over).
+The goal is to be best-effort in terms of providing a seemingly independent Julia session to execute code in.
+
+> [!NOTE]
+> The notion of a sandbox can probably be abstracted.
+> While a module-based sandbox is very simple, it would be useful to have a way to execute Julia code in a clean process (e.g. to fully enforce the independence of the sandboxes, run code in a different package environment, or multi-threading settings).
+> However, ideally the high-level API would be the same, irrespective of how the sandbox is implemented.
+
+### Evaluating code
+
+Presently, there are two functions that offer a
+
+1. `codeblock!` is meant to offer a simple way to execute a block of Julia code (provided as a simple string, not a parsed expression).
+   This is roughly meant to correspond to running a Julia script.
+
+   ```julia-repl
+   julia> sb = CodeEvaluation.Sandbox();
+
+   julia> code = """
+          x = 40
+          println("x = \$x")
+          x + 2
+          """
+   "x = 40\nprintln(\"x = \$x\")\nx + 2\n"
+
+   julia> r = CodeEvaluation.codeblock!(sb, code);
+
+   julia> r.value, r.output
+   (42, "x = 40\n")
+   ```
+
+2. `replblock!` emulates a REPL session.
+   The input code is split up and evaluated as if copy-pasted into the REPL line-by-line.
+   The outputs are then captured as if they would be shown in the REPL.
+
+   ```julia-repl
+   julia> sb = CodeEvaluation.Sandbox();
+
+   julia> code = """
+          x = 40
+          println("x = \$x")
+          x + 2
+          """
+   "x = 40\nprintln(\"x = \$x\")\nx + 2\n"
+
+   julia> r = CodeEvaluation.replblock!(sb, code);
+   ```
+
+   At this point, using the `CodeEvaluation.join_to_string(r)` function, the package is able to reconstruct how the corresponding REPL session would look like.
+
+   ```julia-repl
+   julia> x = 40
+   40
+
+   julia> println("x = $x")
+   x = 40
+
+   julia> x + 2
+   42
+   ```
+
+> [!NOTE]
+> Additional code evaluation "modes" could be added as new functions --- the precise requirements differ, so it would be useful to have a library of methods available.

src/CodeEvaluation.jl

@@ -1,7 +1,10 @@
 module CodeEvaluation
-import IOCapture
+using IOCapture: IOCapture
+using REPL: REPL
 
+include("parseblock.jl")
 include("sandbox.jl")
-include("namedsandboxes.jl")
+include("codeblock.jl")
+include("replblock.jl")
 
 end

src/codeblock.jl

@@ -0,0 +1,16 @@
+"""
+    CodeEvaluation.codeblock!(sandbox::Sandbox, code::AbstractString; kwargs...) -> Result
+
+Evaluates a block of Julia code `code` in the `sandbox`, as if it is included as
+a script. Returns a [`Result`](@ref) object, containing the result of the evaluation.
+
+# Keywords
+
+- `color::Bool=true`: determines whether or not to capture colored output (i.e. controls
+  the IOContext).
+"""
+function codeblock!(sandbox::Sandbox, code::AbstractString; color::Bool=true)
+    exprs = CodeEvaluation.parseblock(code)
+    block_expr = Expr(:block, (expr.expr for expr in exprs)...)
+    return evaluate!(sandbox, block_expr; setans=true, color)
+end

src/parseblock.jl

@@ -0,0 +1,87 @@
+struct ParsedExpression
+    expr::Any
+    code::SubString{String}
+end
+
+"""
+Returns a vector of parsed expressions and their corresponding raw strings.
+
+Returns a `Vector` of tuples `(expr, code)`, where `expr` is the corresponding expression
+(e.g. a `Expr` or `Symbol` object) and `code` is the string of code the expression was
+parsed from.
+
+The keyword argument `skip = N` drops the leading `N` lines from the input string.
+
+If `raise=false` is passed, the `Meta.parse` does not raise an exception on parse errors,
+but instead returns an expression that will raise an error when evaluated. `parseblock`
+returns this expression normally and it must be handled appropriately by the caller.
+
+The `linenumbernode` can be passed as a `LineNumberNode` to give information about filename
+and starting line number of the block.
+"""
+function parseblock(
+    code::AbstractString;
+    skip=0,
+    keywords=true,
+    raise=true,
+    linenumbernode=nothing
+)
+    # Drop `skip` leading lines from the code block. Needed for deprecated `{docs}` syntax.
+    code = string(code, '\n')
+    code = last(split(code, '\n', limit=skip + 1))
+    endofstr = lastindex(code)
+    results = ParsedExpression[]
+    cursor = 1
+    while cursor < endofstr
+        # Check for keywords first since they will throw parse errors if we `parse` them.
+        line = match(r"^(.*)\r?\n"m, SubString(code, cursor)).match
+        keyword = Symbol(strip(line))
+        (ex, ncursor) = if keywords && haskey(Docs.keywords, keyword)
+            (QuoteNode(keyword), cursor + lastindex(line))
+        else
+            try
+                Meta.parse(code, cursor; raise=raise)
+            catch err
+                @error "parse error"
+                break
+            end
+        end
+        str = SubString(code, cursor, prevind(code, ncursor))
+        if !isempty(strip(str)) && ex !== nothing
+            push!(results, ParsedExpression(ex, str))
+        end
+        cursor = ncursor
+    end
+    if linenumbernode isa LineNumberNode
+        exs = Meta.parseall(code; filename=linenumbernode.file).args
+        @assert length(exs) == 2 * length(results) "Issue at $linenumbernode:\n$code"
+        for (i, ex) in enumerate(Iterators.partition(exs, 2))
+            @assert ex[1] isa LineNumberNode
+            expr = Expr(:toplevel, ex...) # LineNumberNode + expression
+            # in the REPL each evaluation is considered a new file, e.g.
+            # REPL[1], REPL[2], ..., so try to mimic that by incrementing
+            # the counter for each sub-expression in this code block
+            if linenumbernode.file === Symbol("REPL")
+                newfile = "REPL[$i]"
+                # to reset the line counter for each new "file"
+                lineshift = 1 - ex[1].line
+                _update_linenumbernodes!(expr, newfile, lineshift)
+            else
+                _update_linenumbernodes!(expr, linenumbernode.file, linenumbernode.line)
+            end
+            results[i] = ParsedExpression(expr, results[i].code)
+        end
+    end
+    results
+end
+
+function _update_linenumbernodes!(x::Expr, newfile, lineshift)
+    for i = 1:length(x.args)
+        x.args[i] = _update_linenumbernodes!(x.args[i], newfile, lineshift)
+    end
+    return x
+end
+_update_linenumbernodes!(x::Any, newfile, lineshift) = x
+function _update_linenumbernodes!(x::LineNumberNode, newfile, lineshift)
+    return LineNumberNode(x.line + lineshift, newfile)
+end