Add more information on function performance to documentation

docs/make.jl

@@ -12,7 +12,8 @@ makedocs(
       "Examples" => "Examples.md",
       "Managing Lambdas" => "Managing_lambdas.md",
       "Labels" => "Labels.md",
-      "Debugging Performance" => "Debugging_Performance.md",
+      "Function Performance" => "Function_Performance.md",
+      "Debugging Functions" => "Debugging_Functions.md",
     ],
     "API" => [
       "Functions" => "Functions.md",

docs/src/Debugging_Performance.md → docs/src/Debugging_Functions.md

@@ -1,6 +1,6 @@
-# Debugging and Performance
+# Debugging Functions
 
-### About AWS Lambda
+### Obtaining Debug Output
 Behind-the-scenes, AWS Lambda uses a heavily optimized system for running Lambda functions (some details [here](https://www.amazon.science/blog/how-awss-firecracker-virtual-machines-work)). The process is somewhat opaque and AWS makes few guarantees about exactly how your function will run. If things are not working as expected, either in terms of speed or functionality, it can be useful to add some `println` statements (or other stdout output) to your function, which will then be captured by AWS and can be recovered by Jot.
 
 The `invoke_function_with_log` function runs a Lambda function and returns a tuple of the function result, and a `LambdaFunctionInvocationLog` object that provides some diagnostics on that particular function invocation:

docs/src/Function_Performance.md

@@ -0,0 +1,38 @@
+# Function Performance
+
+### Naive vs Precompiled vs PackageCompiled
+Depending on how the `create_local_image` function is called, the resulting lambda function will be in one of three possible states:
+- If you have called `create_local_image` without either the `function_test_data` or `package_compile` parameters, your function will have been neither precompiled nor PackageCopmiled. Of the three states, this is the slowest, with invocations of the function taking the maximum possible time. This is fine for testing but not for production use.
+- If you have called `create_local_image` with `function_test_data`, but not `package_compile`, your function will be precompiled, but not PackageCompiled. Precompilation is a Julia-native concept, and it means that any compilation required by the function has been done in advance, and stored as part of the docker image.
+- If you have called `create_local_image` with `function_test_data` and with `package_compile=true`, your function will be both precompiled, and PackageCompiled. PackageCompiled means that [PackageCompiler.jl](https://github.com/JuliaLang/PackageCompiler.jl) has been used to create a Julia [system image](https://docs.julialang.org/en/v1/devdocs/sysimg/), and that this system image is part of the docker image. This will result in very fast Lambda function run times, and is highly recommended for production use.
+
+When setting the `package_compile` option to `true`, you will need to also pass a `FunctionTestData` object to the `function_test_data` parameter of `create_local_image`. This defines a sample argument to pass when testing your lambda function, and the expected response that the lambda function should return when passed that argument.
+
+So if your responder function takes a vector of integers, and increases each element by 1:
+```
+open("increment_vector.jl", "w") do f
+  write(f, "increment_vector(v::Vector{Int}) = map(x -> x + 1, v)")
+end
+increment_responder = get_responder("./increment_vector.jl", :increment_vector, Vector{Int})
+```
+
+... then your `FunctionTestData` might look like this:
+```
+function_test_data = FunctionTestData([1,2,3], [2,3,4])
+```
+where `[1,2,3]` is the argument you intend to pass to the responder, and `[2,3,4]` is the response you are expecting.
+
+... and your call to `create_local_image` might look like this:
+```
+`create_local_image(increment_responder; function_test_data=function_test_data, package_compile=true)`
+```
+
+### Hot vs warm vs cold starts
+When a lambda function is invoked, it may be in either hot, or warm, or a cold state, and this state determines how quickly the function will execute. AWS makes relatively few statements or guarantees about how this works, but from observation:
+- A function that has just been executed will be in its hot state.
+- After some time has elapsed without being invoked, the function will go from hot to warm. This amount of time appears to be variable, and anywhere between a few minutes and a few hours. This shift from hot to warm can be observed empirically by the function run-time increasing, but it is not clear what being 'warm' actually means.
+- After more time has elapsed, the function will go from warm to cold. In the cold state, the docker container has been stopped, and will be started when the function is next invoked. This further increases the function run-time.
+
+### The first execution is special
+In addition to this, the very first execution of a function that has been freshly defined in AWS Lambda appears to be special. It takes longer (and produces more debug output) than any subsequent execution.
+

docs/src/Guide.md

@@ -24,9 +24,7 @@ A `LambdaFunction` is the final stage in the process and represents a working La
 ## Best practices
 
 ### Using PackageCompiler.jl
-[PackageCompiler.jl](https://github.com/JuliaLang/PackageCompiler.jl) is a Julia package that pre-compiles methods. This can be done during the image creation process, and the `create_local_image` function has a `package_compile` option (default `false`) to indicate whether this should be used. Setting `package_compile` to `true` is **highly recommended** for production use; it eliminates almost all of the usual delay while Julia starts up, and so reduces Lambda Function [cold start-up times](https://aws.amazon.com/blogs/compute/operating-lambda-performance-optimization-part-1/) by around 75%, making it competitive with any other language used for AWS Lambda.
-
-When setting the `package_compile` option to `true`, you will need to also pass a `FunctionTestData` object to the `function_test_data` parameter of `create_local_image`. This defines a sample argument to pass when testing your lambda function, and the expected response that the lambda function should return when passed that argument. Whilst doing this test run of your responder function, Jot will record all of the precompilation that needs to be performed, and have `PackageCompiler.jl` add this precompilation to a custom julia system image.
+[PackageCompiler.jl](https://github.com/JuliaLang/PackageCompiler.jl) is a Julia package that pre-compiles methods. This can be done during the image creation process, and the `create_local_image` function has a `package_compile` option (default `false`) to indicate whether this should be used. Setting `package_compile` to `true` is **highly recommended** for production use; it eliminates almost all of the usual delay while Julia starts up, and so reduces Lambda Function [cold start-up times](https://aws.amazon.com/blogs/compute/operating-lambda-performance-optimization-part-1/) by around 75%, making it competitive with any other language used for AWS Lambda. See [function performance](Function_Performance.md) for more details.
 
 ### Working around the one-function-per-container limit
 The Lambda API limits you to one function per container. In practice, dividing up all your functions into different containers is not practical. Instead, have the responding function expect a Dict, then use one of the fields of the dict to indicate the function that should be called. The responding function can then just forward the other parameters to the appropriate function.

docs/src/index.md

@@ -46,8 +46,7 @@ lambda_components |> with_remote_image! |> with_lambda_function! |> run_test
 
 ## Package Features
 - Easily create AWS Lambda functions from Julia packages or scripts
-- Test and check for at multiple stages
-- Allows easy checking for version consistency - eg, is a given Lambda Function using the correct code?
 - PackageCompiler.jl may be optionally used to greatly speed up cold start times
 - JSON read/write and error handling is handled by Jot - you just write standard Julia
+- Allows easy checking for version consistency - ie, is a given Lambda Function using the correct code?
 

notes_on_this_branch.md

@@ -1,3 +1,7 @@
+! We should have two `create_local_image` functions: `create_local_uncompiled_image` and `create_local_compiled_image`. Perhaps initially keep `create_local_image` and point it to `create_local_uncompiled_image`. Change documentation to `create_local_uncompiled_image` and give `create_local_image` a deprecation warning. When doing so, remember that `create_local_image` is referred to `Function_Performance.md`
+! run_lambda_function_test should take a `FunctionTestData`
+! Expand the "managing lambdas" section of the documentation to include the functions like `get_all_lambda_functions`, `get_lambda_function`
+! Put all the performance-related documentation in "Performance"
 ! Not found a great way to ensure that PackageCompile is producing shorter run times - the problem is that we need to wait for a cold start to see, and the timeout for this is highly variable. The first run cannot be used as it seems to be special
 ! multi-to argument for tests is not working, nor is --full
 ! set JULIA_LOAD_PATH=@ in the nix shell so that only Jot's packages can be used.

src/Jot.jl

@@ -284,7 +284,7 @@ tested locally, or directly uploaded to an AWS ECR Repo for use as an AWS Lambda
 
 If `function_test_data` is passed, then this test data will be used to precompile both
 Jot and the responder code before adding it to the docker image. This will reduce AWS
-Lambda cold start times, but for production use, `package_compile` should be set to
+Lambda cold start times, but for production use, `package_compile` should also be set to
 `true`.
 
 `package_compile` determines whether the code is compiled use `PackageCompiler.jl`,

src/PackageCompile.jl

@@ -5,8 +5,8 @@ const SYSIMAGE_FNAME = "SysImage.so"
 
 """
     struct FunctionTestData
-        test_argument::Any
-        expected_response::Any
+      test_argument::Any
+      expected_response::Any
     end
 
 A simple data structure that contains a test argument for a given lambda function, and what return value is expected from the function if passed taht test argument. For example, if your responder function takes a `Vector{Int64}` and increments each element in the vector by 1, you might use `[1, 2]` as the `test_argument`, and `[2, 3]` as the `expected_response`.