Docs/update mapping diagram (#115)

.github/workflows/docs.yml

@@ -8,7 +8,7 @@ jobs:
       max-parallel: 4
       matrix:
         os: [ubuntu-latest]
-        python-version: [3.9]
+        python-version: ["3.11"]
 
     runs-on: ${{ matrix.os }}
 

docs/guides/troubleshooting.md

@@ -2,6 +2,10 @@
 
 This guide aims to provide general tips on troubleshooting and restoring services to a functional state.
 
+> NOTE: 
+> Please also check questions other users have faced: [sensitive-data-archive Discussion](https://github.com/neicnordic/sensitive-data-archive/discussions?discussions_q=)
+> We encourage starting a discussion there as it will help us improve current guides.
+
 ## After deployment checklist
 
 After having deployed the SDA services in a `FederatedEGA` setup, the following steps can be followed to ensure that everything is up and running correctly.

docs/structure.md

@@ -39,7 +39,7 @@ Inter-communication between services
 There are three main ways that the system is passing on information and persist state in the system:
 
 1. through AMQP messages sent from and to micro services;
-2. changes in the database of the status of a file being processed via the the `sda-pipeline`;
+2. changes in the database of the status of a file being processed via the the `SDA - Sensitive Data Archive` components;
 3. location and state of files in either of the three file storage areas.
 
 ### AMQP messaging - Rabbit MQ

docs/submission.md

@@ -43,7 +43,7 @@ Structure of the message and its contents are described in
       SDA RabbitMQ-->>Intercept: Intercept reads message
       Intercept -->> SDA RabbitMQ: Forwards ingest message <br/> to queue
       alt Ingest is successful 
-      Intercept->>Ingest: msg: [sda][ingest] begin ingestion
+      SDA RabbitMQ->>Ingest: msg: [sda][ingest] begin ingestion
       activate Ingest 
       Ingest->>SDA Database: mark ingested
       Note over Ingest: store file in Archive
@@ -68,7 +68,7 @@ Structure of the message and its contents are described in
       Central EGA RabbitMQ-->>SDA RabbitMQ: federated msg: [from_cega][accession type]
       SDA RabbitMQ-->>Intercept: Intercept reads message
       Intercept -->> SDA RabbitMQ: Forwards accession ID message <br/> to queue
-      Intercept->>Finalize: msg: [sda][accession] map file to accession ID
+      SDA RabbitMQ->>Finalize: msg: [sda][accession] map file to accession ID
       alt Finalize is successful 
       activate Finalize
       note right of Finalize: Finalize makes the file backup
@@ -82,16 +82,38 @@ Structure of the message and its contents are described in
       SDA RabbitMQ-->>Central EGA RabbitMQ: shovel msg:[to_cega][files.completed]
       Central EGA RabbitMQ-->>SDA RabbitMQ: federated msg: [from_cega][mappings type]
       SDA RabbitMQ-->>Intercept: Intercept reads message
-      Intercept -->> SDA RabbitMQ: Forwards mapper message <br/> to queue
-      Intercept->>Mapper: msg: [sda][mappings] begin ingestion
-      alt Mapper is successful 
+      Intercept -->> SDA RabbitMQ: Forwards mapper message of type mapping <br/> to queue
+      SDA RabbitMQ->>Mapper: msg: [sda][mappings] map dataset to file accession ID
+      alt Mapper Mapper creates dataset ID to file accession ID mapping 
       activate Mapper
       Mapper->>SDA Database: map file to dataset accession ID
       Mapper->>Inbox: remove file from inbox
       else Error occurred in mapper process
       Mapper-->>SDA RabbitMQ: msg: error
       SDA RabbitMQ-->>Central EGA RabbitMQ: shovel msg:[to_cega][files.error]
       end
+      Central EGA RabbitMQ-->>SDA RabbitMQ: federated msg: [from_cega][release type]
+      SDA RabbitMQ-->>Intercept: Intercept reads message
+      Intercept -->> SDA RabbitMQ: Forwards mapper message <br/> to queue
+      SDA RabbitMQ->>Mapper: msg: [sda][mappings] release dataset
+      alt Mapper flags dataset ready for release 
+      activate Mapper
+      Mapper->>SDA Database: flag dataset ready for release
+      else Error occurred in mapper process
+      Mapper-->>SDA RabbitMQ: msg: error
+      SDA RabbitMQ-->>Central EGA RabbitMQ: shovel msg:[to_cega][files.error]
+      end
+      Central EGA RabbitMQ-->>SDA RabbitMQ: federated msg: [from_cega][deprecate type]
+      SDA RabbitMQ-->>Intercept: Intercept reads message
+      Intercept -->> SDA RabbitMQ: Forwards mapper message  <br/> to queue
+      SDA RabbitMQ->>Mapper: msg: [sda][mappings] deprecate dataset
+      alt Mapper flags dataset as deprecated 
+      activate Mapper
+      Mapper->>SDA Database: flag dataset as deprecated
+      else Error occurred in mapper process
+      Mapper-->>SDA RabbitMQ: msg: error
+      SDA RabbitMQ-->>Central EGA RabbitMQ: shovel msg:[to_cega][files.error]
+      end
       deactivate Mapper
 
 ```
@@ -135,7 +157,7 @@ At this stage, the associated decryption key is retrieved. If decryption
 completes and the checksum is valid, a message of completion is sent to
 `CentralEGA`: Ingestion completed.
 
-> **Important**
+> Important:
 > If a file disappears or is overwritten in the inbox before ingestion is completed, ingestion may not be possible.
 
 Should any of the aforementioned steps result in an error, the workflow is terminated, and the error is logged. If the error is attributed to user misuse, such as providing an incorrect checksum or tampering with the encrypted file, it is reported to `CentralEGA` for display in the Submission Interface.

readthedocs.yaml

@@ -7,9 +7,9 @@ version: 2
 
 # Set the version of Python and other tools you might need
 build:
-  os: ubuntu-20.04
+  os: ubuntu-22.04
   tools:
-    python: "3.9"
+    python: "3.11"
 
 mkdocs:
   configuration: mkdocs.yml