Skip to content

Commit

Permalink
refactor: match description to diagram and make reference to glossary
Browse files Browse the repository at this point in the history
  • Loading branch information
mariajgrimaldi committed Nov 21, 2024
1 parent 8dc03c4 commit f159f3e
Show file tree
Hide file tree
Showing 3 changed files with 20 additions and 14 deletions.
Binary file modified docs/_images/openedx-filters-workflow.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
20 changes: 11 additions & 9 deletions docs/concepts/openedx-filters.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ What are Open edX Filters?

An Open edX Filter is a pipeline mechanism that executes a series of functions when configured. Each function receives input arguments, which are data used by the process in execution, and returns the same arguments, possibly modified. Given this design, filters can modify the application flow according to the specified configuration, altering or adding new behaviors during execution time.

The pipeline mechanism is implemented by a class called `OpenEdxPublicFilter`_, which provides the necessary tools to fulfill the Open edX Filters requirements, such as ordered execution, configurability, interchangeable functions, argument definition, and cumulative behavior. This enables filters to modify the flow of the application dynamically during runtime based on predefined business logic or conditions. We refer to the pipeline mechanism as the **Pipeline Tooling** throughout this document.
The pipeline mechanism is implemented by what's known as :term:`filter tooling<Filter Tooling>`, which provides the necessary tools to fulfill the Open edX Filters requirements, such as ordered execution, configurability, interchangeable functions, argument definition, and cumulative behavior. This enables filters to modify the flow of the application dynamically during runtime based on predefined business logic or conditions. We refer to the pipeline mechanism as the **Pipeline Tooling** throughout this document.

How do Open edX Filters work?
-----------------------------
Expand All @@ -26,29 +26,31 @@ Open edX Filters are implemented using an accumulative pipeline mechanism, which

The workflow of Open edX Filters is as follows:

#. An application component (caller) invokes the filter by calling the ``run_filter()`` method implemented by the filter definition.
#. An application component (caller) invokes the filter during its execution by calling the ``run_filter()`` method implemented by the :term:`filter definition<Filter Definition>`.

#. The ``run_filter`` method calls the **Pipeline Tooling** under the hood, which manages the execution of the filter's pipeline.
#. The ``run_filter`` method of the filter calls the :term:`filter tooling<Filter Tooling>` under the hood, which manages the execution of the filter's pipeline.

#. The filter's tooling retrieves the configuration from ``OPEN_EDX_FILTERS_CONFIG``, which defines a list of N functions :math:`f_1, f_2, \ldots, f_{n}` that will be executed.
#. The :term:`filter tooling<Filter Tooling>` retrieves the configuration from ``OPEN_EDX_FILTERS_CONFIG``, which defines a list of N functions :math:`f_1, f_2, \ldots, f_{n}` that will be executed.

#. The tooling then executes each function in the pipeline sequentially, starting with :math:`f_1`, which processes the input arguments and applies the developer's operations, returning potentially modified arguments.
#. The :term:`tooling <Filter Tooling>` then executes each function in the pipeline sequentially, starting with :math:`f_1`, which processes the input arguments ``kwargs`` and applies the developer's operations, returning potentially modified arguments ``kwargs_1``.

#. The next function (if there are more than one) :math:`f_2` receives the potentially modified arguments and applies further operations, returning another modified set of arguments. This process continues through the list of functions.
#. The next function (if there are more than one) :math:`f_2` receives the potentially modified arguments ``kwargs_1`` and applies further operations, returning another modified set of arguments ``kwargs_2``. This process continues through the list of functions.

#. Each subsequent function receives the output from the previous function and returns its modified output until all functions have been executed.

#. Additionally, at any point in the pipeline, a developer can halt execution by raising an exception, based on conditions defined in the processing logic, to stop the application flow.
#. Additionally, at any point in the pipeline, a developer can halt execution by raising an exception, based on conditions defined in the processing logic, to stop the application flow. In this case, the pipeline stops, and the pipeline tooling raises the exception to the caller as the final output. From there the caller can handle the exception as needed.

#. Once the final function :math:`f_{n}` has been executed, the final modified arguments are returned to the caller, which may use them for the remaining part of its execution.
#. If no exceptions are raised, the pipeline continues executing the functions until the final function :math:`f_{n}` has been executed.

#. The final modified arguments ``kwargs_n`` are returned to the caller, which may use them for the remaining part of its execution.

Each function in the pipeline has the ability to modify the input data, add new data, or halt execution based on specific conditions, such as raising exceptions if certain criteria is not met. This pipeline structure ensures that complex business logic can be applied during runtime without directly altering the application code.

Here's an example of a filter in action:

#. A user enrolls in a course, triggering the `CourseEnrollmentStarted filter`_ by calling the ``run_filter`` method with the enrollment details. This filter processes information about the user, course, and enrollment details.

#. The filter tooling executes a series of functions configured in ``OPEN_EDX_FILTERS_CONFIG``, e.g. checking user eligibility for enrollment, updating the enrollment status, and notifying the user about the enrollment.
#. The :term:`filter tooling<Filter Tooling>` executes a series of functions configured in ``OPEN_EDX_FILTERS_CONFIG``, e.g. checking user eligibility for enrollment, updating the enrollment status, and notifying the user about the enrollment.

#. Each function can modify the input data or halt the process based on business logic, e.g. denying enrollment if the user is ineligible.

Expand Down
14 changes: 9 additions & 5 deletions docs/reference/glossary.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,23 +8,26 @@ A filter has multiple components that are used to define, execute and handle fil
Pipeline
A pipeline is a list of functions executed in a specific order; these functions are known as pipeline steps. Each function in the pipeline takes the output of the previous function as its input, with the final function's output serving as the overall output of the filter. The pipeline behavior was inspired by the `Python Social Auth accumulative pipeline`_, which is described in detail in the :doc:`/decisions/0003-hooks-filter-tooling-pipeline` ADR. These pipelines are configured in the filter configuration and are executed in sequence.

Filter Tooling
The filter tooling is a set of methods that manage the execution of the filter pipeline. The tooling retrieves the filter configuration, executes the pipeline steps in the specified order, and handles exceptions raised by the pipeline steps. This tooling ensures that the pipeline steps are executed in the correct order and that the output of each step is passed to the next step in the pipeline. All this is mainly done by the `OpenEdxPublicFilter`_ class, which provides the necessary definitions to fulfill the Open edX Filters requirements.

Pipeline Step
A pipeline step is a function within a pipeline that receives, processes, and returns data. Each step may perform operations like transforming, validating, filtering, or enriching data. Pipeline steps are implemented as classes that inherit from the base class `PipelineStep`_ and define specific logic within their `run_filter`_ method, which is executed by the pipeline tooling when the filter is triggered.

Filter Definition
A filter definition is a class that inherits from `OpenEdxPublicFilter`_ that implements the ``run_filter`` method which defines the input and output behavior of the filter. This class executes the configured pipeline steps by calling the method `run_pipeline`_, passing down the input arguments, handling exceptions and returning the final output of the filter. Since the ``run_filter`` method is the entry point for the filter, the pipeline steps must have the same signature as the filter definition.
A filter definition is a class that inherits from `OpenEdxPublicFilter`_ that implements the ``run_filter`` method which defines the input and output behavior of the filter. This class executes the configured pipeline steps by calling the method `run_pipeline`_, passing down the input arguments, handling exceptions and returning the final output of the filter. Since the ``run_filter`` method is the entry point for the filter, the pipeline steps must have the same signature as the filter definition. E.g., the `CourseEnrollmentStarted filter`_ is a filter definition that processes information about the user, course, and enrollment details.

Filter Signature
The filter signature consists of the specific parameters required by a filter's ``run_filter`` method. It defines the expected input and output structure for the filter, specifying the data the filter will process. The filter signature is used to ensure that all pipeline steps have the same input and output structure, enabling interchangeability between steps.
The filter signature consists of the specific parameters required by a filter's ``run_filter`` method. It defines the expected input and output structure for the filter, specifying the data the filter will process. The filter signature is used to ensure that all pipeline steps have the same input and output structure, enabling interchangeability between steps. E.g., the `CourseEnrollmentStarted filter`_ signature might include parameters like ``user``, ``course_key``, and ``enrollment mode``.

Filter Type
The filter type is a unique identifier for the filter, following a standardized format following the :doc:`/decisions/0004-filters-naming-and-versioning`. This type is used as an index for configuring the filter pipeline and specifies which configuration settings apply to a given filter.
The filter type is a unique identifier for the filter, following a standardized format following the :doc:`/decisions/0004-filters-naming-and-versioning`. This type is used as an index for configuring the filter pipeline and specifies which configuration settings apply to a given filter. E.g., the `CourseEnrollmentStarted filter`_ has the `filter_type` ``org.openedx.learning.course.enrollment.started.v1``.

Filter Exceptions
Filters can raise exceptions to control the flow of the pipeline. If a filter raises an exception, the pipeline halts, and the exception becomes the pipeline's output. Exceptions are typically raised when certain conditions specified in the filter's logic are met, allowing the filter to control the application flow.
Filters can raise exceptions to control the flow of the pipeline. If a filter raises an exception, the pipeline halts, and the exception becomes the pipeline's output. Exceptions are typically raised when certain conditions specified in the filter's logic are met, allowing the filter to control the application flow. E.g., the `CourseEnrollmentStarted filter`_ might raise an exception if the user is ineligible for enrollment called ``PreventEnrollment``.

Filter Configuration
Filter configuration is a dictionary that defines the pipeline settings for a filter. Each filter type has its own configuration, which includes settings like whether errors should fail silently or propagate, and the sequence of pipeline steps. Configurations specify the filter type, error-handling preferences, and a list of module paths for each pipeline step to be executed.
Filter configuration is a dictionary that defines the pipeline settings for a filter. Each filter type has its own configuration, which includes settings like whether errors should fail silently or propagate, and the sequence of pipeline steps. Configurations specify the filter type, error-handling preferences, and a list of module paths for each pipeline step to be executed. E.g., the configuration for the `CourseEnrollmentStarted filter`_ might include settings like ``fail_silently: False`` and ``['my_plugin.filters.StopEnrollmentIfNotValidEmail']`` as its pipeline steps. See the :doc:`/decisions/0002-hooks-filter-config-location` for more details on the configuration format.

This glossary provides a high-level overview of the key concepts and components of the Open edX Filters library. Understanding these terms will help you implement filters in your application and leverage the filter tooling to control the flow of your application based on specific conditions. For a better illustration of these concepts, refer to the :doc:`/how-tos/using-filters` guide.

Expand All @@ -33,3 +36,4 @@ This glossary provides a high-level overview of the key concepts and components
.. _run_filter: https://github.com/openedx/openedx-filters/blob/main/openedx_filters/filters.py#L60
.. _OpenEdxPublicFilter: https://github.com/openedx/openedx-filters/blob/main/openedx_filters/tooling.py#L14
.. _run_pipeline: https://github.com/openedx/openedx-filters/blob/main/openedx_filters/tooling.py#L164
.. _CourseEnrollmentStarted filter: https://github.com/openedx/openedx-filters/blob/main/openedx_filters/learning/filters.py#L142

0 comments on commit f159f3e

Please sign in to comment.