diff --git a/docs/plugin/execution-modes.md b/docs/plugin/execution-modes.md deleted file mode 100644 index 55f3a7640a..0000000000 --- a/docs/plugin/execution-modes.md +++ /dev/null @@ -1,117 +0,0 @@ -A Thunder plugin can be configured to run in multiple different "execution modes". Each mode defines how the plugin code is executed, however it does not affect how a client will interact with the plugin. Clients will not know (and shouldn't care!) which mode a plugin is running in, and will continue to connect/communicate through the main Thunder process. - -Providing a plugin correctly implements a COM-RPC interface, there should be no additional development work in the plugin to support running in different execution modes, so the decision on which mode each plugin runs in can be an architecture decision based on platform, performance and security requirements. - -![Diagram showing plugins running in and out of process, and a client connecting to Thunder over JSON-RPC](../assets/simple_execution_modes.drawio.svg) - -## Setting the execution mode - -The execution mode of a plugin is set in its configuration file. If an execution mode is not specified a plugin will defaut to running in-process. - -To change the mode, set the `mode` value in the `configuration.root` object to: - -* Off (aka in-process) -* Local (aka out-of-process) -* Container -* Distributed - -As with other config options, this can be changed via the Controller plugin at runtime. The below example config sets the plugin mode to `Local` in order to run the plugin out-of-process - -```json -{ - "locator":"libThunderSamplePlugin.so", - "classname":"SamplePlugin", - "startmode":"Activated", - "configuration":{ - "root":{ - "mode":"Local" - } - } -} -``` - -## In Process Plugins - -This is the simplest execution mode, and the default if a mode is not set. An in-process plugin is loaded into and executed inside the main WPEFraemwork process. - -**Advantages** - -* Simple - there is no additional overhead or complexity from running additional processes -* Highest performance - no additional RPC hops are required to communicate with the plugin. All calls to the plugin from the Thunder process are just local virtual function calls. - * If an in-process plugin needs to communicate with another in-process plugin, Thunder will resolve any COM-RPC interface calls to local function calls. This removes the need to serialise/deserialise any data or cross any IPC boundary and results in the fastest performance possible - -**Disadvantages** - -* Stability - if a plugin is unstable or buggy and causes a crash then it will bring down the entire Thunder process -* Resource monitoring - since the plugin runs inside the Thunder process, it is much harder to accurately monitor the CPU/memory usage from that specific plugin -* Security - the plugin runs with the same permissions and privileges as the main Thunder process - -## Out-Of-Process (OOP) Plugins - -If a plugin is configured to run out-of-process, it will run in its own individual hosting process called `ThunderPlugin` instead of in the main Thunder process. Each out-of-process plugin will run in a separate ThunderPlugin instance. - -When the plugin is activated, Thunder will automatically spawn a ThunderPlugin instance as a child process. The ThunderPlugin host will load the plugin library and establish a COM-RPC connection between itself and the main Thunder process. This COM-RPC connection is how clients communicating with the main Thunder process are still able to invoke methods on out-of-process plugins. The ThunderPlugin process will be stopped when the plugin is deactivated. - -!!! tip - For larger, more complex out-of-process plugins, it is often useful to split a plugin into two separate libraries - see [here](../split-implementation) for more details. - -**Advantages** - -* Reliability - if a plugin crashes, it will only bring down the ThunderPlugin instance and therefore not affect any other plugin. It can then be restarted as necessary -* Monitoring - by running a plugin in its own process, it becomes easier to monitor the resource usage (memory, CPU etc) of that plugin -* Security - out of process plugins can be run in a different user/group to the main Thunder process (which might be running as root) to reduce the privileges of the plugin and increase security -* Resource control - the size of the thread pool and process priority can be set for the ThunderPlugin host, allowing more custom tuning for specific plugin requirements - -**Disadvantages** - -* Performance - since there is now an additional RPC hop over COM-RPC to invoke methods on the plugin, this can introduce some latency (although COM-RPC is efficient so this is very minimal) - * It may also take slightly longer to activate the plugin due to the overhead of starting the ThunderPlugin host -* Resource usage - potentially increased resource usage from spawning and running a new process - -## Container - -!!! note - Support for containers is not enabled by default. - - To enable, build Thunder with the `-DPROCESSCONTAINERS=ON` cmake option and select a backend with `-DPROCESSCONTAINERS_XXX=ON` where `XXX` is the backend you wish to use. - -An extension to the out-of-process mode, container mode will run the ThunderPlugin host instead a containerised environment. Thunder supports various container integrations using the `ProcessContainer` abstraction mechanism in Thunder core: - -- [LXC ](https://linuxcontainers.org/) -- [runc](https://github.com/opencontainers/runc) -- [crun](https://github.com/containers/crun) -- [Dobby](https://github.com/rdkcentral/Dobby) (RDK) -- AWC (Liberty Global) - -A requirement for running a plugin in a container is a suitable container configuration must have been defined - Thunder cannot create container configurations dynamically on-the-fly for plugins. Thunder will look for container configurations in the following locations (with the below priority): - -1. `//Container` -1. `//Container` -1. `//Container` - -For example, the config for a plugin will callsign SamplePlugin might be stored in `/opt/thunder/SamplePlugin/Container` if the persistent path is set to `/opt/thunder` in the Thunder configuration. - -**Advantages** - -* All the previous advantages of OOP plugins -* Increased security - containers have restricted access to the host system including filesystem and device access -* Improved resource limiting - containers can use cgroups to tightly control and monitor resource usage - -**Disadvantages** - -* Maintenance - need to maintain a container config and correctly punch holes in the container to access resources (e.g. device nodes) -* Startup time - plugin activation time may be increased due to the overhead of constructing the container environment - -## Distributed - -!!! danger - Distributed mode is considered experimental - -Distributed mode takes the out-of-process mode one step further by allowing plugins to run on an entirely different device than the main Thunder process. This device could even be running a different CPU with a different architecture. A COM-RPC channel is established over a TCP socket between the two devices to allow communication. - -An example use case for this could be a dual-SoC platform or for communicating with peripheral devices such as cameras. - -Note the COM-RPC protocol is not designed for untrusted channels (e.g. public internet), so this should be used with caution. Enabling tamer-resistant stubs can increase security & robustness but this should still not be considered completely secure. - - - diff --git a/docs/plugin/execution-modes/containers/container.md b/docs/plugin/execution-modes/containers/container.md new file mode 100644 index 0000000000..07e9e1cf84 --- /dev/null +++ b/docs/plugin/execution-modes/containers/container.md @@ -0,0 +1,64 @@ +# Container + +!!! note + Container support is not enabled by default. + + To enable it, build Thunder with the `-DPROCESSCONTAINERS=ON` CMake option and specify the desired backend using the `-DPROCESSCONTAINERS_XXX=ON` where `XXX` corresponds to the backend you wish to use. + +In container mode which is an extension to the out-of-process mode, ThunderPlugin host will run within a containerised environment. + +Thunder supports multiple container integrations through the `ProcessContainer` abstraction mechanism in Thunder core. + +If you have only one container integration, that will be the default one. However, if you have enabled multiple container integrations, you must specify the one you want to use. + +There are two ways to do this: + +- Specify it in the plugin configuration: + + This method allows you to specify which integration to use for the plugin. +``` +"configuration":{ + "root":{ + "mode":"Container", + "configuration" : { + "containertype":"lxc" + } + } +} +``` + +- Specify the default integration in the Thunder configuration. + + This method allows you to set a default integration for plugins that do not have a specified container type in their configuration; they will use this default integration. +``` +"processcontainers" : { + "default" : "runc" +} +``` + +## Supported Containers + +- [LXC ](https://linuxcontainers.org/) +- [runc](https://github.com/opencontainers/runc) +- [crun](https://github.com/containers/crun) +- [Dobby](https://github.com/rdkcentral/Dobby) (Maintained by RDK) +- AWC (Maintained Externally) + +To run a plugin in a container, a suitable container configuration must already exist. Thunder does not create container configurations dynamically. Thunder will search for existing container configurations in the following locations, in order of priority: + +1. `//Container` +1. `//Container` +1. `//Container` + +For instance, the configuratoin for a plugin with the callsign SamplePlugin might be stored in `/opt/thunder/SamplePlugin/Container` if the persistent path is set to `/opt/thunder` in Thunder configuration. + +**Advantages** + +* Retains all the benefits of OOP plugins. +* Enhanced security - Containers have restricted access to the host system including filesystem and device access. +* Improved resource management - Containers can use cgroups to tightly control and monitor resource usage. + +**Disadvantages** + +* Maintenance - Requires managing container configurations and appropriately configuring access to resources. (e.g. device nodes) +* Startup time - Plugin activation may take longer due to the additional overhead of setting up the container environment. \ No newline at end of file diff --git a/docs/plugin/execution-modes/containers/lxc.md b/docs/plugin/execution-modes/containers/lxc.md new file mode 100644 index 0000000000..ff628e25fe --- /dev/null +++ b/docs/plugin/execution-modes/containers/lxc.md @@ -0,0 +1,129 @@ +# What is LXC ? +LXC (Linux Containers) is a lightweight virtualization method that provides an environment similar to a complete Linux system but without the overhead of a full virtual machine. It uses a combination of Linux kernel features like namespaces and cgroups to isolate processes and manage resource allocation.. + +# Setup in Buildroot/Lithosphere + +1. Make sure kernel has all the futures needed for containerization. The easiest way to enable them is to use raspberrypi3_wpe_ml_container_defconfig. + +2. Enable containers support in Thunder. +``` +Thunder -> Extensions -> Process Containers +``` + +3. Select RunC as the backend engine for containers. +``` +Thunder -> Extensions -> Process Containers -> Containers Backend -> LXC +``` + +4. (optional) Enable containers plugin for info about running containers +``` +Thunder -> Plugins -> ProcessContainers +``` + + +# Setting Up LXC Container +For demo purposes, we will use the OCDM plugin. To run a containerized ThunderNanoService you will need to create rootfs with all of the dependencies of the nanoservice and ThunderPlugin as executable. For this tutorial rootfs.tar generated by buildroot/lithosphere/yocto will be used as reference roots. + +1) Before running the system, create a folder named 'Container' under /rootfs/usr/share/Thunder/OCDM/ + +2) Create another folder named 'rootfs' under /rootfs/usr/share/Thunder/OCDM/Container + +3) Copy the whole rootfs of the sdcard into /rootfs/usr/share/Thunder/OCDM/Container/rootfs + + +# Adjusting Configuration for Thunder + +1. Create a file named 'config' under /rootfs/usr/share/Thunder/OCDM/Container and place the below content in it. +``` +# Template used to create this container: /usr/share/lxc/templates/lxc-download +# Parameters passed to the template: --no-validate +# Template script checksum (SHA-1): 273c51343604eb85f7e294c8da0a5eb769d648f3 +# For additional config options, please look at lxc.container.conf(5) + +# Uncomment the following line to support nesting containers: +#lxc.include = /usr/share/lxc/config/nesting.conf +# (Be aware this has security implications) + +lxc.mount.entry = /tmp tmp none bind,optional 0 0 + +# Distribution configuration +lxc.include = /usr/share/lxc/config/common.conf +lxc.include = /usr/share/lxc/config/userns.conf +lxc.arch = linux32 + +# Container specific configuration +# if you see access related issue, disable user/group id settings below +lxc.idmap = u 0 100000 65536 +lxc.idmap = g 0 100000 65536 +lxc.rootfs.path = dir:/usr/share/Thunder/OCDM/Container/rootfs +lxc.uts.name = OCDM + +# Inherit envionment needed by Thunder nanoservices +lxc.environment = PATH +lxc.environment = MESSAGE_DISPATCHER_CONFIG + +# Network configuration +lxc.net.0.type = empty +#lxc.net.0.type = veth +#lxc.net.0.link = lxcbr0 +#lxc.net.0.flags = up +#lxc.net.0.hwaddr = 00:16:XX:XX:XX:XX +``` + +2. Comment out below line in /usr/share/lxc/config/common.conf: +``` +# lxc.seccomp.profile = /usr/share/lxc/config/common.seccomp +``` + +3. In plugin configuration (eg. `/etc/Thunder/plugin/OCDM.json` for OCDM) change +```"mode": Local``` +to +```"mode": "Container"``` + +4. If everything works fine, you should see OCDM working just lie an ordinary OOP plugin. + +# Mounting a shared directory between host and LXC container + +Most often, we may need to securely share files between the host machine and a container. + +Example : File logging from a container to a logging system folder located at the host machine. + +1. Create a directory in host. +``` +mkdir /testshare && chmod 7770 /testshare +``` +2. Create a directory in lxc container (eg: OCDM container) +``` +mkdir /usr/share/Thunder/OCDM/Container/rootfs/TestLogging && chmod 7770 /usr/share/Thunder/OCDM/Container/rootfs/TestLogging +``` +3. Edit container config file (eg: /usr/share/Thunder/OCDM/Container/config) +``` +lxc.mount.entry = /testshare TestLogging none bind,rw 0 0 +``` +4. UID/GID mapping +``` +# Container specific configuration +#lxc.idmap = u 0 100000 65536 +#lxc.idmap = g 0 100000 65536 + +# Container's UID/GID 0-65535 are mapped to host's 100000-165535, +# but UID/GID 1000 on the container is mapped to host's UID/GID 1000. +lxc.idmap = u 0 100000 1000 +lxc.idmap = g 0 100000 1000 +lxc.idmap = u 1000 1000 1 +lxc.idmap = g 1000 1000 1 +lxc.idmap = u 1001 101001 64535 +lxc.idmap = g 1001 101001 64535 +``` +5. Accessing contents from a shared folder and permissions . + + Use case 1 : + Host has created a domain socket within the shared folder. + Container running in non-privileged mode (not root mode), trying to write to this domain socket. + In this use case, the write operation will fail, if there is NO write permission for "other" user (eg: 775 instead of 777). + So make sure that the domain socket has been created with the required access right. + +## Good To Know + +- lxc API to kill a container is blocking, not guarenteed to work and therefore not used +- If ifconfig is not available in the container use the ip command (e.g. "ip address" to get IP address info for the device) \ No newline at end of file diff --git a/docs/plugin/execution-modes/containers/runc.md b/docs/plugin/execution-modes/containers/runc.md new file mode 100644 index 0000000000..cde93ec4b2 --- /dev/null +++ b/docs/plugin/execution-modes/containers/runc.md @@ -0,0 +1,274 @@ +# What is RunC ? +RunC is a CLI tool for spawning and running containers according to the OCI specification. The Open Container Initiative (OCI) is a standardized format for creating a container. It defines a container as a pair of rootfs folder & configuration file in JSON format. + +# Setup in Buildroot/Lithosphere + +1. Make sure kernel has all the futures needed for containerization. The easiest way to enable them is to use raspberrypi3_wpe_ml_container_defconfig. + +2. Enable containers support in Thunder +``` +Thunder -> Extensions -> Process Containers +``` + +3. Select RunC as the backend engine for containers. +``` +Thunder -> Extensions -> Process Containers -> Containers Backend -> runc +``` + +4. (optional) Enable containers plugin for info about running containers +``` +Thunder -> Plugins -> ProcessContainers +``` + + +# Setting Up runc Container +For demo purposes, we will use the OCDM plugin. To run a containerized ThunderNanoService you will need to create rootfs with all of the dependencies of the nanoservice and ThunderPlugin as executable. For this tutorial rootfs.tar generated by buildroot/lithosphere/yocto will be used as reference roots. + +1. On target platform, create `/usr/share/Thunder//Container`. This will be a directory containing all files defining a container. +``` +mkdir /usr/share/Thunder/OCDM/Container +``` + +2. Extract rootfs.tar generated eg. by buildroot into `/usr/share/Thunder//Container/rootfs`. + +3. Create a reference configuration. You can either use a reference config file presented later in the instructions or use runc cmdline tool to do it for you, by navigating to `/usr/share/Thunder//Container` and calling `runc spec`. Be careful - if you do it on the host platform, the "platform" section in config will have to be modified to match the target platform. + +# Adjusting Configuration for Thunder +Default config obtained by `runc spec` gives you a nice starting point, that you can tailor as you wish. However, there are a few things that need to be set up right to work with Thunder: + +1. The most important thing is to mount `/tmp/communicator` pipe inside the container to allow for communication between ThunderPlugin inside a container namespace and Thunder. OCDM plugin that is used in the example also creates another pipe inside /tmp folder, so for tutorial whole /tmp folder is mounted +``` +"mounts": [ + { + "source": "/tmp", + "destination": "/tmp", + "options": ["rw", "bind"] + }, + [...] +] +``` +2. Set terminal to false +``` +"process": { + "terminal": false, + [...] +} +``` +3. By design, all of the configurations should be passed to Nanoservice by arguments/runtime. However, at the moment, there are some env variables that need to be set. This is considered a temporal solution and won't be needed in future releases. +``` +"env": [ + "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", + "TERM=xterm", + "COMMUNICATOR_CONNECTOR=$COMMUNICATOR_CONNECTOR", + "MESSAGE_DISPATCHER_CONFIG=$MESSAGE_DISPATCHER_CONFIG" + ], +``` +4. Launched process will always be a Nanoservice, so container's args will be overided. You don't acctually need to do anything here, however it's good to have in mind that this option will have no effect. +``` +"args": [ + "sh", +] +``` + +is used so we will mount the whole tmp For now, you also need to set up some env variables as presented in the config, however, in future versions, this will no longer be needed. + +# Configure plugin to run as a containerized process +In plugin configuration (eg. `/etc/Thunder/plugin/OCDM.json` for OCDM) change +```"mode": Local``` +to +```"mode": "Container"``` + +If everything works fine, you should see OCDM working just lie an ordinary OOP plugin + +# Example configuration +This is an example of config generated by `runc spec` with the following changes: +1. `/tmp` folder mounted inside the container +2. Env variables set inside the container +3. terminal was set to false + +```json +{ + "ociVersion": "1.0.2-dev", + "process": { + "terminal": false, + "user": { + "uid": 0, + "gid": 0 + }, + "args": [ + "sh" + ], + "env": [ + "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", + "TERM=xterm", + "TRACE_FILENAME=/tmp/tracebuffer", + "TRACE_DOORBELL=/tmp/tracebuffer.doorbell", + "COMMUNICATOR_CONNECTOR=/tmp/communicator" + ], + "cwd": "/", + "capabilities": { + "bounding": [ + "CAP_AUDIT_WRITE", + "CAP_KILL", + "CAP_NET_BIND_SERVICE" + ], + "effective": [ + "CAP_AUDIT_WRITE", + "CAP_KILL", + "CAP_NET_BIND_SERVICE" + ], + "permitted": [ + "CAP_AUDIT_WRITE", + "CAP_KILL", + "CAP_NET_BIND_SERVICE" + ], + "ambient": [ + "CAP_AUDIT_WRITE", + "CAP_KILL", + "CAP_NET_BIND_SERVICE" + ] + }, + "rlimits": [ + { + "type": "RLIMIT_NOFILE", + "hard": 1024, + "soft": 1024 + } + ], + "noNewPrivileges": true + }, + "root": { + "path": "rootfs", + "readonly": true + }, + "hostname": "runc", + "mounts": [ + { + "source": "/tmp", + "destination": "/tmp", + "options": ["rw", "bind"] + }, + { + "destination": "/proc", + "type": "proc", + "source": "proc" + }, + { + "destination": "/dev", + "type": "tmpfs", + "source": "tmpfs", + "options": [ + "nosuid", + "strictatime", + "mode=755", + "size=65536k" + ] + }, + { + "destination": "/dev/pts", + "type": "devpts", + "source": "devpts", + "options": [ + "nosuid", + "noexec", + "newinstance", + "ptmxmode=0666", + "mode=0620", + "gid=5" + ] + }, + { + "destination": "/dev/shm", + "type": "tmpfs", + "source": "shm", + "options": [ + "nosuid", + "noexec", + "nodev", + "mode=1777", + "size=65536k" + ] + }, + { + "destination": "/dev/mqueue", + "type": "mqueue", + "source": "mqueue", + "options": [ + "nosuid", + "noexec", + "nodev" + ] + }, + { + "destination": "/sys", + "type": "sysfs", + "source": "sysfs", + "options": [ + "nosuid", + "noexec", + "nodev", + "ro" + ] + }, + { + "destination": "/sys/fs/cgroup", + "type": "cgroup", + "source": "cgroup", + "options": [ + "nosuid", + "noexec", + "nodev", + "relatime", + "ro" + ] + } + ], + "linux": { + "resources": { + "devices": [ + { + "allow": false, + "access": "rwm" + } + ] + }, + "namespaces": [ + { + "type": "pid" + }, + { + "type": "network" + }, + { + "type": "ipc" + }, + { + "type": "uts" + }, + { + "type": "mount" + } + ], + "maskedPaths": [ + "/proc/acpi", + "/proc/asound", + "/proc/kcore", + "/proc/keys", + "/proc/latency_stats", + "/proc/timer_list", + "/proc/timer_stats", + "/proc/sched_debug", + "/sys/firmware", + "/proc/scsi" + ], + "readonlyPaths": [ + "/proc/bus", + "/proc/fs", + "/proc/irq", + "/proc/sys", + "/proc/sysrq-trigger" + ] + } +} + +``` \ No newline at end of file diff --git a/docs/plugin/execution-modes/distrubuted.md b/docs/plugin/execution-modes/distrubuted.md new file mode 100644 index 0000000000..8c83e95424 --- /dev/null +++ b/docs/plugin/execution-modes/distrubuted.md @@ -0,0 +1,13 @@ +# Distributed + +!!! danger + Distributed mode is considered experimental + +Distributed mode takes the out-of-process mode one step further by allowing plugins to run on an entirely different device than the main Thunder process. This device could even be running a different CPU with a different architecture. A COM-RPC channel is established over a TCP socket between the two devices to allow communication. + +An example use case for this could be a dual-SoC platform or for communicating with peripheral devices such as cameras. + +Note the COM-RPC protocol is not designed for untrusted channels (e.g. public internet), so this should be used with caution. Enabling tamer-resistant stubs can increase security & robustness but this should still not be considered completely secure. + + + diff --git a/docs/plugin/execution-modes/execution-modes.md b/docs/plugin/execution-modes/execution-modes.md new file mode 100644 index 0000000000..fd01ff42c3 --- /dev/null +++ b/docs/plugin/execution-modes/execution-modes.md @@ -0,0 +1,31 @@ +A Thunder plugin can be configured to run in multiple different "execution modes". Each mode defines how the plugin code is executed, however it does not affect how a client will interact with the plugin. Clients will not know (and shouldn't care!) which mode a plugin is running in, and will continue to connect/communicate through the main Thunder process. + +Providing a plugin correctly implements a COM-RPC interface, there should be no additional development work in the plugin to support running in different execution modes, so the decision on which mode each plugin runs in can be an architecture decision based on platform, performance and security requirements. + +![Diagram showing plugins running in and out of process, and a client connecting to Thunder over JSON-RPC](../../assets/simple_execution_modes.drawio.svg) + +## Setting the execution mode + +The execution mode of a plugin is set in its configuration file. If an execution mode is not specified a plugin will defaut to running in-process. + +To change the mode, set the `mode` value in the `configuration.root` object to: + +* Off (aka in-process) +* Local (aka out-of-process) +* Container +* Distributed + +As with other config options, this can be changed via the Controller plugin at runtime. The below example config sets the plugin mode to `Local` in order to run the plugin out-of-process + +```json +{ + "locator":"libThunderSamplePlugin.so", + "classname":"SamplePlugin", + "startmode":"Activated", + "configuration":{ + "root":{ + "mode":"Local" + } + } +} +``` \ No newline at end of file diff --git a/docs/plugin/execution-modes/inprocess.md b/docs/plugin/execution-modes/inprocess.md new file mode 100644 index 0000000000..837267c84e --- /dev/null +++ b/docs/plugin/execution-modes/inprocess.md @@ -0,0 +1,15 @@ +# In Process Plugins + +This is the simplest execution mode, and the default if a mode is not set. An in-process plugin is loaded into and executed inside the main WPEFraemwork process. + +**Advantages** + +* Simple - there is no additional overhead or complexity from running additional processes +* Highest performance - no additional RPC hops are required to communicate with the plugin. All calls to the plugin from the Thunder process are just local virtual function calls. + * If an in-process plugin needs to communicate with another in-process plugin, Thunder will resolve any COM-RPC interface calls to local function calls. This removes the need to serialise/deserialise any data or cross any IPC boundary and results in the fastest performance possible + +**Disadvantages** + +* Stability - if a plugin is unstable or buggy and causes a crash then it will bring down the entire Thunder process +* Resource monitoring - since the plugin runs inside the Thunder process, it is much harder to accurately monitor the CPU/memory usage from that specific plugin +* Security - the plugin runs with the same permissions and privileges as the main Thunder process \ No newline at end of file diff --git a/docs/plugin/execution-modes/introduction.md b/docs/plugin/execution-modes/introduction.md new file mode 100644 index 0000000000..fd01ff42c3 --- /dev/null +++ b/docs/plugin/execution-modes/introduction.md @@ -0,0 +1,31 @@ +A Thunder plugin can be configured to run in multiple different "execution modes". Each mode defines how the plugin code is executed, however it does not affect how a client will interact with the plugin. Clients will not know (and shouldn't care!) which mode a plugin is running in, and will continue to connect/communicate through the main Thunder process. + +Providing a plugin correctly implements a COM-RPC interface, there should be no additional development work in the plugin to support running in different execution modes, so the decision on which mode each plugin runs in can be an architecture decision based on platform, performance and security requirements. + +![Diagram showing plugins running in and out of process, and a client connecting to Thunder over JSON-RPC](../../assets/simple_execution_modes.drawio.svg) + +## Setting the execution mode + +The execution mode of a plugin is set in its configuration file. If an execution mode is not specified a plugin will defaut to running in-process. + +To change the mode, set the `mode` value in the `configuration.root` object to: + +* Off (aka in-process) +* Local (aka out-of-process) +* Container +* Distributed + +As with other config options, this can be changed via the Controller plugin at runtime. The below example config sets the plugin mode to `Local` in order to run the plugin out-of-process + +```json +{ + "locator":"libThunderSamplePlugin.so", + "classname":"SamplePlugin", + "startmode":"Activated", + "configuration":{ + "root":{ + "mode":"Local" + } + } +} +``` \ No newline at end of file diff --git a/docs/plugin/execution-modes/outofprocess.md b/docs/plugin/execution-modes/outofprocess.md new file mode 100644 index 0000000000..dde4830e1a --- /dev/null +++ b/docs/plugin/execution-modes/outofprocess.md @@ -0,0 +1,21 @@ +# Out-Of-Process (OOP) Plugins + +If a plugin is configured to run out-of-process, it will run in its own individual hosting process called `ThunderPlugin` instead of in the main Thunder process. Each out-of-process plugin will run in a separate ThunderPlugin instance. + +When the plugin is activated, Thunder will automatically spawn a ThunderPlugin instance as a child process. The ThunderPlugin host will load the plugin library and establish a COM-RPC connection between itself and the main Thunder process. This COM-RPC connection is how clients communicating with the main Thunder process are still able to invoke methods on out-of-process plugins. The ThunderPlugin process will be stopped when the plugin is deactivated. + +!!! tip + For larger, more complex out-of-process plugins, it is often useful to split a plugin into two separate libraries - see [here](../split-implementation) for more details. + +**Advantages** + +* Reliability - if a plugin crashes, it will only bring down the ThunderPlugin instance and therefore not affect any other plugin. It can then be restarted as necessary +* Monitoring - by running a plugin in its own process, it becomes easier to monitor the resource usage (memory, CPU etc) of that plugin +* Security - out of process plugins can be run in a different user/group to the main Thunder process (which might be running as root) to reduce the privileges of the plugin and increase security +* Resource control - the size of the thread pool and process priority can be set for the ThunderPlugin host, allowing more custom tuning for specific plugin requirements + +**Disadvantages** + +* Performance - since there is now an additional RPC hop over COM-RPC to invoke methods on the plugin, this can introduce some latency (although COM-RPC is efficient so this is very minimal) + * It may also take slightly longer to activate the plugin due to the overhead of starting the ThunderPlugin host +* Resource usage - potentially increased resource usage from spawning and running a new process \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index 77f0e0b4f3..8c4a46d1bd 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -21,7 +21,15 @@ nav: - COM-RPC: introduction/architecture/rpc/comrpc.md - Plugin Development: - Introduction: plugin/intro.md - - Execution Modes: plugin/execution-modes.md + - Execution Modes: + - Introduction: plugin/execution-modes/execution-modes.md + - In Process: plugin/execution-modes/inprocess.md + - Out Of Process: plugin/execution-modes/outofprocess.md + - Distrubuted: plugin/execution-modes/distrubuted.md + - Containers: + - Introduction: plugin/execution-modes/containers/container.md + - Runc: plugin/execution-modes/containers/runc.md + - Lxc: plugin/execution-modes/containers/lxc.md - Plugin Lifecycle: plugin/lifecycle.md - Design Principles: plugin/principles.md - Interfaces: @@ -83,8 +91,8 @@ markdown_extensions: - toc: toc_depth: 4 - pymdownx.emoji: - emoji_index: !!python/name:materialx.emoji.twemoji - emoji_generator: !!python/name:materialx.emoji.to_svg + emoji_index: !!python/name:material.extensions.emoji.twemoji + emoji_generator: !!python/name:material.extensions.emoji.to_svg - pymdownx.superfences: custom_fences: - name: mermaid