Merge pull request #530 from timholy/teh/revise3-docs

Update the docs for Revise 3, fix expression-splitting, and remove tracking on deleted files

docs/src/config.md

@@ -3,19 +3,19 @@
 ## Using Revise by default
 
 If you like Revise, you can ensure that every Julia session uses it by
-launching it from your `.julia/config/startup.jl` file.
+launching it from your `~/.julia/config/startup.jl` file.
 
-### Revise 2.6+ and Julia 1.5+
+### Julia 1.5 and higher
 
-If you're using at least version 2.6 of Revise *and* version 1.5 or higher of Julia, this can be as simple as adding
+If you're using at least Julia 1.5, this can be as simple as adding
 
 ```julia
 using Revise
 ```
 
 to your `startup.jl`.
 
-If you use different package depots and do not always have Revise available,
+If you use different package environments and do not always have Revise available,
 
 ```julia
 try
@@ -27,7 +27,7 @@ end
 
 is recommended instead.
 
-### Earlier versions of Revise and/or Julia
+### Earlier Julia versions
 
 If you sometimes use versions of Julia prior to 1.5, instead of the above add
 
@@ -56,24 +56,23 @@ catch e
 end
 ```
 
-## Optional package-specific configuration
+## Configuring the revise mode
 
-In packages, by default Revise will re-evaluate every changed expression.
-However, in packages that process a lot of "data" at toplevel, this is
-unlikely to be desirable. You can set the *mode* by defining a
-variable `__revise_mode__` in your package. The default setting corresponds to
+By default, in packages all changes are tracked, but with `includet` only method definitions are tracked.
+This behavior can be overridden by defining a variable `__revise_mode__` in the module(s) containing
+your methods and/or data. `__revise_mode__` must be a `Symbol` taking one of the following values:
 
-```
-const __revise_mode__ = :eval
-```
-
-(which re-evaluates everything) but you have other options:
-
-- `:evalmeth` will only evaluate those statements needed to redefine methods.
+- `:eval`: evaluate everything (the default for packages)
+- `:evalmeth`: evaluate changes to method definitions (the default for `includet`)
   This should work even for quite complicated method definitions, such as those that might
-  be made within an `@eval` block.
-- `:evalassign` additionally evaluates assignment statements. A top-level expression
+  be made within a `for`-loop and `@eval` block.
+- `:evalassign`: evaluate method definitions and assignment statements. A top-level expression
   `a = Int[]` would be evaluated, but `push!(a, 1)` would not because the latter is not an assignment.
+- `:sigs`: do not implement any changes, only scan method definitions for their signatures so that
+  their location can be updated as changes to the file(s) are made.
+
+If you're using `includet` from the REPL, you can enter `__revise_mode__ = :eval` to set
+it throughout `Main`. `__revise_mode__` can be set independently in each module.
 
 ## Optional global configuration
 
@@ -85,7 +84,7 @@ There are several ways to set these environment variables:
 
 - If you are [Using Revise by default](@ref) then you can include statements like
   `ENV["JULIA_REVISE"] = "manual"` in your `.julia/config/startup.jl` file prior to
-  the line `@eval using Revise`.
+  the line containing `using Revise`.
 - On Unix systems, you can set variables in your shell initialization script
   (e.g., put lines like `export JULIA_REVISE=manual` in your
   [`.bashrc` file](http://www.linuxfromscratch.org/blfs/view/svn/postlfs/profile.html)
@@ -104,9 +103,6 @@ the timing of revisions. `Revise` looks for an environment variable
 `JULIA_REVISE`, and if it is set to anything other than `"auto"` it
 will require that you manually call `revise()` to update code.
 
-Alternatively, you can omit the call to `Revise.async_steal_repl_backend()` from your
-`startup.jl` file (see [Using Revise by default](@ref)).
-
 ### User scripts: JULIA\_REVISE\_INCLUDE
 
 By default, `Revise` only tracks files that have been required as a consequence of
@@ -169,3 +165,4 @@ string `"1"` (e.g., `JULIA_REVISE_POLL=1` in a bash script).
 !!! note
     NFS stands for [Network File System](https://en.wikipedia.org/wiki/Network_File_System) and is typically only used to mount shared network drives on *Unix* file systems.
     Despite similarities in the acronym, NTFS, the standard [filesystem on Windows](https://en.wikipedia.org/wiki/NTFS), is completely different from NFS; Revise's default configuration should work fine on Windows without polling.
+    However, WSL2 users currently need polling due to [this bug](https://github.com/JuliaLang/julia/issues/37029).

docs/src/cookbook.md

@@ -28,7 +28,7 @@ Both options are described below.
     this approach might require you to do some configuration.
     (Once you get things set up, you shouldn't have to do this part ever again.)
     PkgTemplates needs you to configure your `git` user name and email.
-    Some instructions on configuration are [here](https://help.github.com/en/github/getting-started-with-github/set-up-git)
+    Some instructions on configuration are [here](https://docs.github.com/en/github/getting-started-with-github/set-up-git)
     and [here](https://git-scm.com/book/en/v2/Getting-Started-First-Time-Git-Setup).
     It's also helpful to sign up for a [GitHub account](https://github.com/)
     and set git's `github.user` variable.
@@ -173,7 +173,7 @@ control.
 ## `includet` usage
 
 The alternative to creating packages is to manually load individual source files.
-This approach is intended for quick-and-dirty development;
+This approach is intended for early stages of development;
 if you want to track multiple files and/or have some files include other files,
 you should consider switching to the package style above.
 

docs/src/debugging.md

@@ -4,7 +4,7 @@
 
 If Revise isn't behaving the way you expect it to, it can be useful to examine the
 decisions it made.
-Revise supports Julia's [Logging framework](https://docs.julialang.org/en/latest/stdlib/Logging/)
+Revise supports Julia's [Logging framework](https://docs.julialang.org/en/v1/stdlib/Logging/)
 and can optionally record its decisions in a format suitable for later inspection.
 What follows is a simple series of steps you can use to turn on logging, capture messages,
 and then submit them with a bug report.

docs/src/dev_reference.md

@@ -45,13 +45,12 @@ Revise.user_callbacks_by_key
 
 ```@docs
 Revise.RelocatableExpr
-Revise.BackEdges
 Revise.ModuleExprsSigs
 Revise.FileInfo
 Revise.PkgData
 Revise.WatchList
-Revise.SlotDep
 Revise.TaskThunk
+Revise.ReviseEvalException
 MethodSummary
 ```
 
@@ -121,7 +120,9 @@ Much of the "brains" of Revise comes from doing analysis on lowered code.
 This part of the package is not as well documented.
 
 ```@docs
-Revise.hastrackedexpr
+Revise.minimal_evaluation!
+Revise.methods_by_execution!
+Revise.CodeTrackingMethodInfo
 ```
 
 ### Modules and paths

docs/src/index.md

@@ -14,9 +14,10 @@ debugger displays the source code of what you're actually debugging.
 
 !!! note "Automatically loading Revise"
 
-    Many users automatically load Revise on startup. This is slightly more involved
-    than just adding `using Revise` to `.julia/config/startup.jl`: see [Using Revise
-    by default](@ref) for details.
+    Many users automatically load Revise on startup.
+    On versions of Julia older than 1.5, this is slightly more involved
+    than just adding `using Revise` to `.julia/config/startup.jl`: see
+    [Using Revise by default](@ref) for details.
 
 ## Installation
 
@@ -30,10 +31,17 @@ or with `using Pkg; Pkg.add("Revise")`.
 
 ## Usage example
 
+We'll make changes to Julia's "Example" package (a trivial package designed to
+illustrate the file and directory organization of typical packages).
+We have to "develop" it in order to make changes:
+
 ```julia
 (v1.0) pkg> dev Example
 [...output related to installation...]
 
+```
+Now we load Revise (if we haven't already done so) and Example:
+```julia
 julia> using Revise        # importantly, this must come before `using Example`
 
 julia> using Example
@@ -42,14 +50,14 @@ julia> hello("world")
 "Hello, world"
 ```
 
-Now we're going to test that the `Example` module lacks a function named `f`:
+Now we're going to check that the `Example` module currently lacks a function named `f`:
 
 ```julia
 julia> Example.f()
 ERROR: UndefVarError: f not defined
 ```
 
-But we really want `f`, so let's add it.
+But say we really want `f`, so let's add it.
 You can either navigate to the source code (at `.julia/dev/Example/src/Example.jl`)
 in an editor manually, or you can use Julia to open it for you:
 
@@ -65,7 +73,10 @@ julia> Example.f()
 π = 3.1415926535897...
 ```
 
-Now suppose we realize we've made a horrible mistake: that `f` method will ruin everything.
+Voila! Even though we'd loaded Example before adding this function,
+Revise noticed the change and inserted it into our running session.
+
+Now suppose we realize we've made a horrible mistake: that `f` method will mess up everything, because it's part of a more complicated dispatch process and incorrectly intercepts certain `f` calls.
 No problem, just delete `f` in your editor, save the file, and you're back to this:
 
 ```julia
@@ -74,6 +85,8 @@ ERROR: UndefVarError: f not defined
 ```
 
 all without restarting Julia.
+While you can evalued *new* methods without Revise using [inline evaluation](https://www.julia-vscode.org/docs/stable/userguide/runningcode/#Julia:-Execute-Code-Block-(AltEnter)-1) through your IDE,
+method *deletion* is just one example of a change that can only be made easily by Revise.
 
 If you need more examples, see [Revise usage: a cookbook](@ref).
 
@@ -97,17 +110,17 @@ julia> Example.f()
 ```
 
 Revise is not tied to any particular editor.
-(The [EDITOR or JULIA_EDITOR](https://docs.julialang.org/en/latest/stdlib/InteractiveUtils/#InteractiveUtils.edit-Tuple{AbstractString,Integer}) environment variables can be used to specify your preference for which editor gets launched by Julia's `edit` function.)
+(The [EDITOR or JULIA_EDITOR](https://docs.julialang.org/en/v1/stdlib/InteractiveUtils/#InteractiveUtils.edit-Tuple{AbstractString,Integer}) environment variables can be used to specify your preference for which editor gets launched by Julia's `edit` function.)
 
 If you don't want to have to remember to say `using Revise` each time you start
 Julia, see [Using Revise by default](@ref).
 
 ## What Revise can track
 
-Revise is fairly ambitious: if all is working you should be able to track changes to
+Revise is fairly ambitious: if all is working, subject to a few [Limitations](@ref) you should be able to track changes to
 
 - any package that you load with `import` or `using`
-- any script you load with [`includet`](@ref)
+- any script you load with [`includet`](@ref) (see [Configuring the revise mode](@ref) for important default restrictions on `includet`)
 - any file defining `Base` julia itself (with `Revise.track(Base)`)
 - any of Julia's standard libraries (with, e.g., `using Unicode; Revise.track(Unicode)`)
 - any file defining `Core.Compiler` (with `Revise.track(Core.Compiler)`)
@@ -180,7 +193,7 @@ particularly relevant for them.
 If Revise isn't working for you, here are some steps to try:
 
 - See [Configuration](@ref) for information on customization options.
-  In particular, some file systems (like NFS) might require special options.
+  In particular, some file systems (like [NFS](https://en.wikipedia.org/wiki/Network_File_System)) and current users of [WSL2](https://devblogs.microsoft.com/commandline/announcing-wsl-2/) might require special options.
 - Revise can't handle all kinds of code changes; for more information,
   see the section on [Limitations](@ref).
 - Try running `test Revise` from the Pkg REPL-mode.

docs/src/internals.md

@@ -201,15 +201,15 @@ init_c_library()     # library crashes if we call this twice
 Starting with version 2.3, Revise attempts to avoid interpreting any code not necessary for signature computation.
 If you are just tracking changes, Revise will skip over such blocks; if you're loading a file with `includet` for the first time, Revise will execute such blocks in compiled mode.
 
-Revise achieves this by computing [`Revise.BackEdges`](@ref), essentially a set of links encoding the dependencies among different lines of the lowered code.
+Revise achieves this by computing [backedges](https://juliadebug.github.io/LoweredCodeUtils.jl/stable/edges/), essentially a set of links encoding the dependencies among different lines of the lowered code.
 For the `floatwins` example above, the backedges would represent the fact that line 2 has one direct dependant, line 3 (which uses `%2`), that lines 3 and 4 both have line 5 as their dependents, and line 5 has line 6 as a dependent.  As a consequence, to (nearly) execute line 6, we have to execute lines 2-5, because they set up the signature. If an interdependent block doesn't contain any `:method` or related (`:struct_type`, `:eval`) expressions, then it doesn't need to interpret the block at all.
 
 As should be evident, the lowered code makes it much easier to analyze the graph of these dependencies.  There are, however, a few tricky cases.
 For example, any code inside an `@eval` might, or might not, expand into lowered code that contains a `:method` expression.  Because Revise can't reliably predict what it will look like after expansion, Revise will execute any code in (or needed for) an `@eval` block. As a consequence, even after version 2.3 Revise may sometimes interpret more code than is strictly necessary.
 
 !!! note
 
-    If Revise executes code that still shouldn't be run twice, one good solution is to put all initialization inside your module's [`__init__` function](https://docs.julialang.org/en/latest/manual/modules/#Module-initialization-and-precompilation-1).
+    If Revise executes code that still shouldn't be run twice, one good solution is to put all initialization inside your module's [`__init__` function](https://docs.julialang.org/en/v1/manual/modules/#Module-initialization-and-precompilation-1).
     For files that you track with `includet`, you can also split "code that defines methods" into a separate file from "code that does work," and have Revise track only the method-defining file.
     However, starting with version 2.3 Revise should be fairly good at doing this on its own; such manual interventions should not be necessary in most cases.
 

src/Revise.jl

@@ -637,11 +637,21 @@ function handle_deletions(pkgdata, file)
     mexsold = fi.modexsigs
     filep = normpath(joinpath(basedir(pkgdata), file))
     topmod = first(keys(mexsold))
-    mexsnew = file_exists(filep) ? parse_source(filep, topmod) :
-              (@warn("$filep no longer exists, deleting all methods"); ModuleExprsSigs(topmod))
+    fileok = file_exists(filep)
+    mexsnew = fileok ? parse_source(filep, topmod) : ModuleExprsSigs(topmod)
     if mexsnew !== nothing
         delete_missing!(mexsold, mexsnew)
     end
+    if !fileok
+        @warn("$filep no longer exists, deleted all methods")
+        idx = fileindex(pkgdata, file)
+        deleteat!(pkgdata.fileinfos, idx)
+        deleteat!(pkgdata.info.files, idx)
+        wl = get(watched_files, basedir(pkgdata), nothing)
+        if isa(wl, WatchList)
+            delete!(wl.trackedfiles, file)
+        end
+    end
     return mexsnew, mexsold
 end
 
@@ -729,6 +739,7 @@ function revise(; throw=false)
     for ((pkgdata, file), mexsnew) in zip(finished, mexsnews)
         defaultmode = PkgId(pkgdata).name == "Main" ? :evalmeth : :eval
         i = fileindex(pkgdata, file)
+        i === nothing && continue   # file was deleted by `handle_deletions`
         fi = fileinfo(pkgdata, i)
         try
             for (mod, exsnew) in mexsnew
@@ -882,17 +893,59 @@ end
 """
     includet(filename)
 
-Load `filename` and track any future changes to it. `includet` is essentially shorthand for
+Load `filename` and track future changes. `includet` is intended for quick "user scripts"; larger or more
+established projects are encouraged to put the code in one or more packages loaded with `using`
+or `import` instead of using `includet`. See https://timholy.github.io/Revise.jl/stable/cookbook/
+for tips about setting up the package workflow.
+
+By default, `includet` only tracks modifications to *methods*, not *data*. See the extended help for details.
+Note that this differs from packages, which evaluate all changes by default.
+This default behavior can be overridden; see [Configuring the revise mode](@ref).
+
+# Extended help
+
+## Behavior and justification for the default revision mode (`:evalmeth`)
+
+`includet` uses a default `__revise_mode__ = :evalmeth`. The consequence is that if you change
+
+```
+a = [1]
+f() = 1
+```
+to
+```
+a = [2]
+f() = 2
+```
+then Revise will update `f` but not `a`.
+
+This is the default choice for `includet` because such files typically mix method definitions and data-handling.
+Data often has many untracked dependencies; later in the same file you might `push!(a, 22)`, but Revise cannot
+determine whether you wish it to re-run that line after redefining `a`.
+Consequently, the safest default choice is to leave the user in charge of data.
+
+## Workflow tips
+
+If you have a series of computations that you want to run when you redefine your methods, consider separating
+your method definitions from your computations:
+
+- method definitions go in a package, or a file that you `includet` *once*
+- the computations go in a separate file, that you re-`include` (no "t" at the end) each time you want to rerun
+  your computations.
+
+This can be automated using [`entr`](@ref).
+
+## Internals
+
+`includet` is essentially shorthand for
 
     Revise.track(Main, filename; mode=:eval, skip_include=false)
 
-`includet` is intended for "user scripts," e.g., a file you use locally for a specific
-purpose such as loading a specific data set or performing a particular analysis.
 Do *not* use `includet` for packages, as those should be handled by `using` or `import`.
-(If you're working with code in Base or one of Julia's standard libraries, use
-`Revise.track(mod)` instead, where `mod` is the module.)
 If `using` and `import` aren't working, you may have packages in a non-standard location;
 try fixing it with something like `push!(LOAD_PATH, "/path/to/my/private/repos")`.
+(If you're working with code in Base or one of Julia's standard libraries, use
+`Revise.track(mod)` instead, where `mod` is the module.)
 
 `includet` is deliberately non-recursive, so if `filename` loads any other files,
 they will not be automatically tracked.

src/logging.jl

@@ -73,7 +73,7 @@ const _debug_logger = ReviseLogger()
 """
     logger = Revise.debug_logger(; min_level=Debug)
 
-Turn on [debug logging](https://docs.julialang.org/en/latest/stdlib/Logging/)
+Turn on [debug logging](https://docs.julialang.org/en/v1/stdlib/Logging/)
 (if `min_level` is set to `Debug` or better) and return the logger object.
 `logger.logs` contains a list of the logged events. The items in this list are of type `Revise.LogRecord`,
 with the following relevant fields:

src/parsing.jl

@@ -59,7 +59,18 @@ function parse_source!(mod_exprs_sigs::ModuleExprsSigs, src::AbstractString, fil
         if exprs_sigs === nothing
             mod_exprs_sigs[mod] = exprs_sigs = ExprsSigs()
         end
-        pushex!(exprs_sigs, ex)
+        if ex.head === :toplevel
+            lnn = nothing
+            for a in ex.args
+                if isa(a, LineNumberNode)
+                    lnn = a
+                else
+                    pushex!(exprs_sigs, Expr(:block, lnn, a))
+                end
+            end
+        else
+            pushex!(exprs_sigs, ex)
+        end
     end
     return mod_exprs_sigs
 end