Chapter3 - first draft

config.yaml

@@ -1,79 +1,80 @@
-#------------------------------------------------------------
-# Values for this lesson.
-#------------------------------------------------------------
-
-# Which carpentry is this (swc, dc, lc, or cp)?
-# swc: Software Carpentry
-# dc: Data Carpentry
-# lc: Library Carpentry
-# cp: Carpentries (to use for instructor training for instance)
-# incubator: The Carpentries Incubator
-carpentry: 'incubator'
-
-# Overall title for pages.
-title: 'Introduction to vantage6'
-
-# Date the lesson was created (YYYY-MM-DD, this is empty by default)
-created: 2024-03-26
-
-# Comma-separated list of keywords for the lesson
-keywords: 'federated learning, privacy enhancing technology, python'
-
-# Life cycle stage of the lesson
-# possible values: pre-alpha, alpha, beta, stable
-life_cycle: 'pre-alpha'
-
-# License of the lesson
-license: 'CC-BY 4.0'
-
-# Link to the source repository for this lesson
-source: 'https://github.com/vantage6/vantage6-workshop'
-
-# Default branch of your lesson materials (recommended CC-BY 4.0)
-branch: 'main'
-
-# Who to contact if there are any issues
-contact: '[email protected]'
-
-# Navigation ------------------------------------------------
-#
-# Use the following menu items to specify the order of
-# individual pages in each dropdown section. Leave blank to
-# include all pages in the folder.
-#
-# Example -------------
-#
-# episodes:
-# - introduction.md
-# - first-steps.md
-#
-# learners:
-# - setup.md
-#
-# instructors:
-# - instructor-notes.md
-#
-# profiles:
-# - one-learner.md
-# - another-learner.md
-
-# Order of episodes in your lesson
-episodes: 
-- introduction.md
-- 2-understanding-v6.md
-
-# Information for Learners
-learners: 
-
-# Information for Instructors
-instructors: 
-
-# Learner Profiles
-profiles: 
-
-# Customisation ---------------------------------------------
-#
-# This space below is where custom yaml items (e.g. pinning
-# sandpaper and varnish versions) should live
-
-
+#------------------------------------------------------------
+# Values for this lesson.
+#------------------------------------------------------------
+
+# Which carpentry is this (swc, dc, lc, or cp)?
+# swc: Software Carpentry
+# dc: Data Carpentry
+# lc: Library Carpentry
+# cp: Carpentries (to use for instructor training for instance)
+# incubator: The Carpentries Incubator
+carpentry: 'incubator'
+
+# Overall title for pages.
+title: 'Introduction to vantage6'
+
+# Date the lesson was created (YYYY-MM-DD, this is empty by default)
+created: 2024-03-26
+
+# Comma-separated list of keywords for the lesson
+keywords: 'federated learning, privacy enhancing technology, python'
+
+# Life cycle stage of the lesson
+# possible values: pre-alpha, alpha, beta, stable
+life_cycle: 'pre-alpha'
+
+# License of the lesson
+license: 'CC-BY 4.0'
+
+# Link to the source repository for this lesson
+source: 'https://github.com/vantage6/vantage6-workshop'
+
+# Default branch of your lesson materials (recommended CC-BY 4.0)
+branch: 'main'
+
+# Who to contact if there are any issues
+contact: '[email protected]'
+
+# Navigation ------------------------------------------------
+#
+# Use the following menu items to specify the order of
+# individual pages in each dropdown section. Leave blank to
+# include all pages in the folder.
+#
+# Example -------------
+#
+# episodes:
+# - introduction.md
+# - first-steps.md
+#
+# learners:
+# - setup.md
+#
+# instructors:
+# - instructor-notes.md
+#
+# profiles:
+# - one-learner.md
+# - another-learner.md
+
+# Order of episodes in your lesson
+episodes: 
+- introduction.md
+- 2-understanding-v6.md
+- chapter3.md
+
+# Information for Learners
+learners: 
+
+# Information for Instructors
+instructors: 
+
+# Learner Profiles
+profiles: 
+
+# Customisation ---------------------------------------------
+#
+# This space below is where custom yaml items (e.g. pinning
+# sandpaper and varnish versions) should live
+
+

episodes/chapter3.md

@@ -0,0 +1,221 @@
+---
+title: "Running a PET analysis without programming on vantage6"
+teaching: 2
+exercises: 3
+---
+
+:::::::::::::::::::::::::::::::::::::: questions
+- What is a common collaboration scenario where the same institution is involved in multiple collaborations?
+- How to check the status of a given collaboration within vantage6?
+- How to link an algorithm store to a given collaboration?
+- How to request a task based on a given algorithm through vantage6's UI?
+
+:::::::::::::::::::::::::::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::: objectives
+
+- Explore specific data analysis scenarios that further illustrate the concept of collaboration
+- Understand the concept of 'algorithm trustworthiness' in the context of a vantage6 collaboration
+- Understand v6's algorithm-store current and envisioned features
+- Understand the UI-based approach for performing a data analysis through the given scenarios 
+
+::::::::::::::::::::::::::::::::::::::::::::::::
+
+## Vantage6 user interface basics
+
+
+To navigate vantage6's UI seamlessly, it's essential to grasp the platform's fundamental concepts and their interconnections, as the UI design reflects these relationships. The following is a simplified model of vantage6 concepts, where a `1-n` relationship means that the entity on the left side of the relationship is related to one or more entities on the right side. For instance, a **collaboration** involves one or more **nodes**, but each **node** can only be linked to exactly one **collaboration**. An `n-n` relationship is a many-to-many relationship: for instance, a **collaboration** can involve multiple **organizations**, and at the same time, each **organization** can participate in multiple **collaborations**.
+
+![Vantage6 core concepts](fig/chapter3/v6_entitites_simplified.png)
+
+Given the above, the following are the most important concepts to be considered for this episode:
+
+- **Organization**: a group of users that share a common goal or interest (e.g., a consortium, an institute, etc.).
+    - An organization has zero or more **users** who can perform certain actions.
+    - An organization can have multiple **nodes**, one for each **collaboration** it is part of.
+
+- **Collaboration**: a collection of one or more **organizations** working together towards a shared objective.
+    - In a specific **collaboration**, each participating **organization** needs one **node** to compute **tasks**; In another **collaboration**, the same **organization** must have a separate **node** for this **collaboration**.
+
+- **Node**: the component with access to the organization data, which also executes algorithms on it. It represents the individual organizations' contributions to the network.
+
+- **Task**: a request, created by **users** (with a given set of **roles**) for the execution of a given **algorithm** on one or more organizations within a collaboration. These execution requests are handled by the corresponding organizations' **node**. 
+
+- **User**: a person who can perform certain actions in vantage6.
+    - A user can only belong to one **organization**.
+    - A user can have multiple **roles** and can be assigned with extra permissions not covered by the roles.
+    - A user can create **tasks** for one or more **organizations** within a **collaboration**.
+
+- **Role**: a collection of rules that define the permissions of a user (it will be further explored on Chapter 4).
+- **Result**: the output generated by the execution of an **algorithm** as part of a **task**. 
+- **Algorithm**: computational models or processes that are executed on data. Compatible algorithms are those that adhere to the Vantage6 framework, enabling them to be securely distributed to **nodes** for execution.
+
+### Where are these concepts in the UI?
+
+After logging in to the vantage6 UI, you will see the start page.
+
+![vantage6 UI start page](fig/chapter3/ui_start_page.png)
+
+There are some collaborations displayed on the start page. Clicking one of the collaborations will show the tasks of that collaboration.
+
+![vantage6 UI tasks page](fig/chapter3/ui_task_page.png)
+
+The start page also contains a button `Administration` in the top right corner. Clicking on this button will redirect you to the administration page.
+
+On the administration page, you can manage the entities of vantage6. The entities are divided into tabs: `Organizations`, `Collaborations`, `Roles`, `Users`, and `Nodes`. You can click on an entity to see more details or to edit the entity. We will get back to this later in more detail.
+
+![vantage6 UI administration page](fig/chapter3/ui_admin_page.png)
+
+::::::::::::::::::::::::::::::::::::: challenge
+
+## Challenge 1: Get familiar with the vantage6 UI
+
+Log in to the vantage6 UI and navigate to the `Administration` page. Familiarize yourself with the entities in the administration page.
+
+Can you find the `Organizations`, `Collaborations`, `Roles`, `Users`, and `Nodes` tabs?
+
+::::::::::::::::::::::::::::::::::::::::::::::::
+
+
+
+
+
+## From theory to practice: a hypothetical case study using vantage6 collaborations
+
+As previously discussed, in vantage6 a collaboration refers to an agreement between two or more parties to participate in a study or to answer a research question together. This concept is central to the Privacy Enhancing Technologies (PETs) that vantage6 supports. Each party involved in a collaboration remains autonomous, meaning they retain control over their data and can decide how much of their data to contribute to the collaboration's global model and which algorithms are allowed for execution. 
+
+To illustrate this, let's analyze a hypothetical scenario: two international research projects relying on vantage6 technology on the same server: 
+
+* The first one, __PhY2024__ (Prevalence of hypertension and its association with lifestyle), requires determining the average systolic and diastolic blood pressure levels of the population across France, Spain, and The Netherlands. 
+* The second, __GHT__ (Global Health Trends), requires determining the Average BMI across The Netherlands, Spain, and Germany. 
+
+Although both projects are unrelated and independent from each other, the data from Spain and The Netherlands -required by both- is provided by the same large-scale cohort studies, namely CANTABRIA (Spain) and LIFELINES (The Netherlands). However, legal agreements dictate that each project is granted access solely to the data essential for its intended purposes (gaining access to the whole set of variables study increases risks of inference attacks). Data from the French and German population (for __PhY2024__ and __GHT__ studies), on the other hand, will be provided by the GAZEL and GNC prospective cohort studies.
+
+Following vantage6's concepts, this scenario would involve two collaborations, one for each research project. As described in previous episodes, a vantage6 node is needed for each collaboration. Consequently, as illustrated in the diagram below, both CANTABRIA and LIFELINES organizations require two vantage6 data-node instances each. Since each data node defines its own rules for data access and algorithm usage, this ensures that analyses from various collaborations, even those involving shared organizations/datasets, will not conflict with one another.
+
+![Hypothetical collaborations scenario](fig/chapter3/orgs_n_collabs_scenario.png)
+
+
+### Algorithms trustworthiness in a federated setting
+
+While a vantage6-supported research infrastructure like the one described above offers a strong defense against many data privacy risks, there remains one crucial security aspect that falls outside the platform's scope: the validation of the code that will run on this infrastructure. For instance, the administrators of the nodes running within each organization are responsible for defining which algorithms (i.e., [which container images](https://docs.vantage6.ai/en/main/node/configure.html#all-configuration-options)) will be allowed for execution on the respective collaborations. As this is a critical and complex task that entails activities like code analysis and verification, working with algorithms from trusted sources is the primary line of defense against potential threats.
+
+Vantage6's algorithm store feature aims to enhance trustworthiness by offering a centralized platform for managing pre-registered algorithms. This serves as an alternative to using algorithms from unknown authors or those lacking transparency regarding their development process and status. The algorithm store currently allows researchers to explore which algorithms are available and how to run them. This, along with its integration with vantage6's UI, streamlines task execution requests within collaborations.
+
+As of the time of writing this tutorial, efforts are underway to integrate additional information to the algorithm metadata such as creators and code reviewers. Moreover, plans are in place to incorporate the algorithm review process into the publication procedure for any algorithms in the store.
+
+### Running a PET (privacy-enhancing technology) analysis without programming!
+
+In this episode, you will perform a PET analysis on an existing vantage6 collaboration (based on 'dummy' nodes) that resembles the two described above. For reference, the datasets of each organization can be seen here (TODO).
+
+::::::::::::::::::::::::::::::::::::: challenge
+
+## Challenge 2: understanding a simple federated algorithm
+
+First, let's take a look at one of the federated algorithms -available on the vantage6's community store- that will be used in this episode: [a federated average](https://github.com/IKNL/v6-average-py/blob/master/v6-average-py/__init__.py). 
+
+Analyze the algorithm based on the code and its comments and answer the following questions:
+
+- How are the `central_average` and `partial_average` functions related?
+- Why does the `central_average` function, unlike `partial_average`, doesn't get any data as an input?
+- Analyze and discuss the potential outcomes if a Task to execute `central_average` is initiated within a collaboration where one of the nodes is offline.
+
+::::::::::::::::::::::::::::::::::::::::::::::::
+
+
+
+::::::::::::::::::::::::::::::::::::: challenge
+
+## Challenge 3: exploring the status of existing collaborations configured on a vantage6 server
+
+Below are the administrator credential of GHT and PhY24 collaborations (passwords will be given by the instructors).
+
+| User        | Roles               | Collaboration |
+| ----------- | ------------------- | ------------- |
+| PhY24-admin | Collaboration Admin | PhY24         |
+| GHT-admin   | Collaboration Admin | GHT           |
+
+Using these credentials check the status of both collaborations. Given this and your algorithm analysis from Challenge #2 answer the following:
+
+1. Which collaborations are ready for creating a Task for the __federated average__ algorithm?
+2. If one of the collaborations is not ready, which organization you would need to contact in order to make it ready for executing the algorithm too?
+
+::::::::::::::::::::::::::::::::::::::::::::::::
+
+:::::::::::::::::::::::: solution
+
+## Solution steps
+
+To check the status of the nodes of each collaboration: 
+
+1. Log in to each one with the given credentials
+2. Click on 'Administration' on the top of the UI
+3. select 'Collaborations' on the left panel, and then select the corresponding collaboration. 
+4. If there are 'offline' nodes, click on the 'Nodes' panel on the left and check when these were seen for the last time.
+
+![Collaboration status](fig/chapter3/collab-status-offline.png)
+
+:::::::::::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::: challenge
+
+## Challenge 4: adding an algorithm store to an organization
+
+In order to execute the __average algorithm__ on a given collaboration, considering the previous discussion on algorithm trustwortiness, you need to first register an algorithm store on it first. Use the credentials given for Challenge #4 to register the 'community store', which contains the said algorithm: `https://store.cotopaxi.vantage6.ai`
+
+::::::::::::::::::::::::::::::::::::::::::::::::
+
+:::::::::::::::::::::::: solution
+
+## Solution steps
+
+You will now link the 'community-store' to the collaboration whose nodes are ready for it. 
+
+1. Login into the organization using the corresponding credentials from above. 
+2. Click on 'Administration' on the top of the UI, select 'Collaborations' on the left panel, and then select the corresponding collaboration.
+3. Click on '+ Add algorithm store'
+4. Add the vantage6's community store. Use any descrption as name, and provide community store URL: `https://store.cotopaxi.vantage6.ai`
+5. Make sure the store is now shown on the collaboration details:
+![Community store entry on the collaboration details](fig/chapter3/community-store-entry.png)
+
+:::::::::::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::: challenge
+
+## Challenge 5: your first algorithm execution as a researcher
+
+Now, you'll take on the role of the researcher within the collaboration for which you've just established the algorithm store. With this role, you will finally request the execution of the algorithm. 
+
+1. log in as a researcher using the corresponding credentials below:
+
+    |  User     |  Roles        |  Collaboration   |
+    |-----------|---------------|------------------|
+    |PhY24-rs1  | Researcher    |PhY24             |
+    |GHT-rs1    | Researcher    |GHT               |
+
+2. Select the collaboration given on the front page, and select 'Tasks' from the panel on the left.
+    ![Collaboration researcher view](fig/chapter3/collab-researcher-view.png)
+
+3. If you have set up everything correctly, the 'Average' algorithm should be now listed under the '*Select which algorithm you want to run*' dropdown menu. Select it, and provide a name and a description.
+
+    ![Algorithm selection](fig/chapter3/task-alg-selection.png)
+
+4. Now the UI will let you choose between the two functions you explored in Challenge #1. First, try to run the `partial_average` on all the nodes individually.
+
+    ![Running a function on all nodes](fig/chapter3/task-partial-on-individial-orgs.png)
+
+5. Select the 'default' database, choose any numerical column as a parameter, and then click on 'Submit'.
+
+6. The task you just requested should be listed with a 'pending' status. Once finished, explore and download the provided results:
+    ![alt text](fig/chapter3/task-results.png)
+
+
+Based on these results, answer the following:
+
+
+1. If you repeat the same exercise but with the `central_average` function (refer to Challenge #2 if needed), which organization nodes should you choose this time to actually calculate the overall (across all the datasets) average? Experiment with this and discuss the results with the instructors.
+
+2. What would happen if you select an alpha-numerical column (e.g., 'participant_pseudo_id')? Do this experiment and explore the generated error logs. Discuss with the instructors how these logs can be used to diagnose any task execution issues.
+
+::::::::::::::::::::::::::::::::::::::::::::::::
+