Development: Add deployment info and best practices to playwright docs

docs/dev/playwright.rst

@@ -1,6 +1,23 @@
 E2E Testing with Playwright
 ===========================
 
+**Background**
+
+The Playwright test suite contains system tests verifying the most important features of Artemis.
+System tests test the whole system and therefore require a complete deployment of Artemis first.
+In order to prevent as many faults (bugs) as possible from being introduced into the develop branch,
+we want to execute the Playwright test suite whenever new commits are pushed to a Git branch
+(just like the unit and integration test suites).
+
+To accomplish this we need to be able to dynamically deploy multiple different instances of Artemis at the same time.
+An ideal setup would be to deploy the whole Artemis system using Kubernetes.
+However, this setup is too complex at the moment.
+The main reason for the complexity is that it is very hard to automatically setup Docker containers for
+the external services (e.g. Gitlab, Jenkins) and connect them directly with Artemis.
+
+Therefore, the current setup only dynamically deploys the Artemis server and configures it to connect to
+the prelive system, which is already properly setup in the university data center.
+
 Set up Playwright locally
 -------------------------
 
@@ -137,8 +154,72 @@ To run tests sequentially (one after another), set the ``workers`` option to ``1
 sequentially, while running test files in parallel, set the ``fullyParallel`` option to ``false``.
 
 
-Best practices for writing tests in Playwright
-----------------------------------------------
+Best practices when writing new E2E tests
+-----------------------------------------
+
+**Understanding the System and Requirements**
+
+Before writing tests, a deep understanding of the system and its requirements is crucial.
+This understanding guides determining what needs testing and what defines a successful test.
+The best way to understand is to consolidate the original system`s developer or a person actively working on this
+component.
+
+**Identify Main Test Scenarios**
+
+Identify what are the main ways the component is supposed to be used. Try
+the action with all involved user roles and test as many different inputs as
+feasible.
+
+**Identify Edge Test Scenarios**
+
+Next to the main test scenarios, there are also edge case scenarios. These
+tests include inputs/actions that are not supposed to be performed (e.g. enter
+a too-long input into a field) and test the error-handling capabilities of the
+platform.
+
+**Write Tests as Development Progresses**
+
+Rather than leaving testing until the end, write tests alongside each piece of
+functionality. This approach ensures the code remains testable and makes
+identifying and fixing issues as they arise easier.
+
+**Keep Tests Focused**
+
+Keep each test focused on one specific aspect of the code. If a test fails, it is
+easier to identify the issue when it does not check multiple functionalities at
+the same time.
+
+**Make Tests Independent**
+
+Tests should operate independently from each other and external factors like
+the current date or time. Each test should be isolated. Use API calls for unrelated tasks, such as creating a
+course, and UI interaction for the appropriate testing steps. This also involves
+setting up a clean environment for every test suite.
+
+**Use Descriptive Test Names**
+
+Ensure each test name clearly describes what the test does. This strategy
+makes the test suite easier to understand and quickly identifies which test
+has failed.
+
+**Use Similar Test Setups**
+
+Avoid using different setups for each test suit. For example, always check
+for the same HTTP response when deleting a course.
+
+**Do Not Ignore Failing Tests**
+
+If a test consistently fails, pay attention to it. Investigate as soon as possible
+and fx the issue, or update the test if the requirements have changed.
+
+**Regularly Review and Refactor Your Tests**
+
+Tests, like code, can accumulate technical debt. Regular reviews for duplication,
+unnecessary complexity, and other issues help maintain tests and enhance reliability.
+
+
+Playwright testing best practices
+---------------------------------
 
 1. **Use page objects for common interactions**:
 
@@ -230,3 +311,102 @@ Best practices for writing tests in Playwright
         Waiting for the page load state is not recommended if we are only interested in specific elements appearing on
         the page - use ``waitFor()`` function of a locator instead.
 
+
+Artemis Deployment on Bamboo Build Agent
+----------------------------------------
+Every execution of the Playwright test suite requires its own deployment of Artemis.
+The easiest way to accomplish this is to deploy Artemis locally on the build agent, which executes the Playwright tests.
+Using ``docker compose`` we can start a MySQL database and the Artemis server locally on the build agent and
+connect it to the prelive system in the university data center.
+
+.. figure:: playwright/playwright_bamboo_deployment_diagram.svg
+  :align: center
+  :alt: Artemis Deployment on Bamboo Build Agent for Playwright
+
+  Artemis Deployment on Bamboo Build Agent for Playwright
+
+In total there are three Docker containers started in the Bamboo build agent:
+
+1. MySQL
+
+  This container starts a MySQL database and exposes it on port 3306.
+  The container automatically creates a new database 'Artemis' and configures it
+  with the recommended settings for Artemis.
+  The Playwright setup reuses the already existing
+  `MySQL docker image <https://github.com/ls1intum/Artemis/blob/develop/docker/mysql.yml>`__
+  from the standard Artemis Docker setup.
+
+2. Artemis Application
+
+  The Docker image for the Artemis container is created from the already existing
+  `Dockerfile <https://github.com/ls1intum/Artemis/blob/develop/docker/artemis/Dockerfile>`__.
+  When the Bamboo build of the Playwright test suite starts, it retrieves the Artemis executable (.war file)
+  from the `Artemis build plan <https://bamboo.ase.in.tum.de/browse/ARTEMIS-WEBAPP>`_.
+  Upon creation of the Artemis Docker image the executable is copied into the image together with configuration files
+  for the Artemis server.
+
+  The main configuration of the Artemis server are contained in the
+  `Playwright environment configuration files <https://github.com/ls1intum/Artemis/tree/develop/docker/artemis/config>`__.
+  However, those files do not contain any security relevant information.
+  Security relevant settings are instead passed to the Docker container via environment variables. This information is
+  accessible to the Bamboo build agent via
+  `Bamboo plan variables <https://confluence.atlassian.com/bamboo/bamboo-variables-289277087.html>`__.
+
+  The Artemis container is also configured to
+  `depend on <https://docs.docker.com/compose/compose-file/compose-file-v2/#depends_on>`__
+  the MySQL container and uses
+  `health checks <https://docs.docker.com/compose/compose-file/compose-file-v2/#healthcheck>`__
+  to wait until the MySQL container is up and running.
+
+3. Playwright
+
+  Playwright offers a test environment `docker image <https://hub.docker.com/r/microsoft/playwright>`__
+  to execute Playwright tests.
+  The image contains Playwright browsers and browser system dependencies.
+  However, Playwright itself is not included in the image.
+  This is convenient for us because the image is smaller and the Artemis Playwright project requires
+  additional dependencies to fully function.
+  Therefore, the Artemis Playwright Docker container is configured to install all dependencies
+  (using :code:`npm ci`) upon start. This will also install Playwright itself.
+  Afterwards the Artemis Playwright test suite is executed.
+
+  The necessary configuration for the Playwright test suite is also passed in via environment variables.
+  Furthermore, the Playwright container depends on the Artemis container and is only started
+  once Artemis has been fully booted.
+
+**Bamboo webhook**
+
+The Artemis instance deployed on the build agent is not publicly available to improve the security of this setup.
+However, in order to get the build results for programming exercise submissions Artemis relies on a webhook from Bamboo
+to send POST requests to Artemis.
+To allow this, an extra rule has been added to the firewall allowing only the Bamboo instance in the prelive system
+to connect to the Artemis instance in the build agent.
+
+**Timing**
+
+As mentioned above, we want the Playwright test suite to be executed whenever new commits are pushed to a Git branch.
+This has been achieved by adding the
+`Playwright build plan <https://bamboo.ase.in.tum.de/browse/ARTEMIS-AEPTMA1132>`__
+as a `child dependency <https://confluence.atlassian.com/bamboo/setting-up-plan-build-dependencies-289276887.html>`__
+to the `Artemis Build build plan <https://bamboo.ase.in.tum.de/browse/ARTEMIS-WEBAPP>`__.
+The *Artemis Build* build plan is triggered whenever a new commit has been pushed to a branch.
+
+The Playwright build plan is only triggered after a successful build of the Artemis executable.
+This does imply a delay (about 10 minutes on average) between the push of new commits and the execution
+of the Playwright test suite, since the new Artemis executable first has to be built.
+
+**NOTE:** The Playwright test suite is only automatically executed for internal branches and pull requests
+(requires access to this GitHub repository), **not** for external ones.
+In case you need access rights, please contact the maintainer `Stephan Krusche <https://github.com/krusche>`__.
+
+Maintenance
+-----------
+The Artemis Dockerfile as well as the MySQL image are already maintained because they are used in
+other Artemis Docker setups.
+Therefore, only Playwright and the Playwright Docker image require active maintenance.
+Since the Playwright test suite simulates a real user, it makes sense to execute the test suite with
+the latest browser versions.
+The Playwright Docker image we use always has browsers with specific versions installed.
+Therefore, the
+`docker-compose file <https://github.com/ls1intum/Artemis/blob/develop/docker/playwright.yml>`__
+should be updated every month to make sure that the latest Playwright image is used.