chore:Merge pull request #481 from welfare-state-analytics/dev

v0.14.0
welfare-state-analytics · Feb 23, 2024 · d685fed · d685fed
2 parents 01a093d + 5494fd6
commit d685fed
Show file tree

Hide file tree

Showing 63 changed files with 59,379 additions and 58,158 deletions.
diff --git a/.github/workflows/push.yml b/.github/workflows/push.yml
@@ -68,6 +68,26 @@ jobs:
       run: |
         python -m unittest test.chairs
 
+  empty-speech:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+         python-version: [3.8]
+    steps:
+    - uses: actions/checkout@v3
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v2
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install .
+        pip install PyPDF2
+    - name: Test there are no empty u or seg elements
+      run: |
+        python -m unittest test.empty-speech
+
   mp:
     runs-on: ubuntu-latest
     strategy:
@@ -189,3 +209,23 @@ jobs:
     - name: Check that README updating script works
       run: |
         PYTHONPATH="$PYTHONPATH:." python scripts/stats-dashboard/generate-markdown.py -v v1.1.1
+
+  mandates:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: [3.8]
+    steps:
+    - uses: actions/checkout@v3
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v2
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install .
+        pip install PyPDF2
+    - name: Test manually curated mandate dates do not change
+      run: |
+        python -m unittest test.mandates
diff --git a/README.md b/README.md
@@ -2,7 +2,7 @@
 [![Validate Parla-Clarin XML](https://github.com/welfare-state-analytics/riksdagen-corpus/actions/workflows/validate.yml/badge.svg)](https://github.com/welfare-state-analytics/riksdagen-corpus/actions/workflows/validate.yml)
 
 
-# Swedish parliamentary proceedings --- 1867--today --- v0.13.0
+# Swedish parliamentary proceedings --- 1867--today --- v0.13.1
 
 _Westac Project_, 2020--2024 |
 _Swerik Project_, 2023--2025
@@ -39,18 +39,18 @@ The data in the corpus is delivered as TEI XML files to follow established pract
 
 Currently, we have an extensive set of Parliamentary Records (Riksdagens Protokoll) from 1867 until now. We are in the process of preparing Motions for inclusion in the corpus and other document types will follow.
 
-|                                      |   v0.13.0  |   v0.12.0  |   v0.10.0  |
+|                                      |   v0.13.1  |   v0.13.0  |   v0.12.0  |
 |--------------------------------------|------------|------------|------------|
-|           Corpus size (GB)           |    5.48    |    5.59    |    5.58    |
+|           Corpus size (GB)           |    5.48    |    5.48    |    5.59    |
 |    Number of parliamentary records   |    17642   |    17642   |    17642   |
-|   Total parliamentary record pages*  |   1045458  |   1045458  |   1041807  |
-|  Total parliamentary record speeches |   1014214  |   1014214  |   1127027  |
-|   Total parliamentary record words   |  442634322 |  442634322 |  441525242 |
+|   Total parliamentary record pages*  |   1045458  |   1045458  |   1045458  |
+|  Total parliamentary record speeches |   1014214  |   1014214  |   1014214  |
+|   Total parliamentary record words   |  442634322 |  442634322 |  442634322 |
 |           Number of Motions          |      0     |      0     |      0     |
 |          Total motion pages          |      0     |      0     |      0     |
 |          Total motion words          |      0     |      0     |      0     |
 |     Number of people with MP role    |    5975    |    5975    |    5975    |
-|  Number of people with minister role |     546    |     535    |     535    |
+|  Number of people with minister role |     546    |     546    |     535    |
 
 \* Digital original parliamentary records for some years in the 1990s are not paginated and thus do not contribute to the page count.See also §_Number of Pages in Parliamentary Records_.
 
@@ -110,4 +110,4 @@ If you find any errors, it is possible to submit corrections to them. This is do
 <img src="scripts/stats-dashboard/figures/logos/vr.png" width="250"/>
 
 ---
-Last update: 2024-01-15, 08:56:24
+Last update: 2024-01-16, 12:52:56
diff --git a/corpus/metadata/README.md b/corpus/metadata/README.md
@@ -72,6 +72,14 @@ Individual level data such as born, gender, etc. for all persons in the metadata
 - *gender*: gender
 - *riksdagen_id*: id for riksdagen open data individual
 
+### references_map.csv
+
+This links SWERIK person_ids to references in the biography books.
+- `swerik_id`: person ID
+- `bibtex_key`: bibtex key from `../references/`
+- `wiki_id`: wiki_id of the book referenced (volume can be identified by the bibtex key)
+- `page`: page where the person identified is referenced
+
 ### speaker.csv
 
 Same as member_of_parliament.csv but for speakers.