Code
from IPython.display import display, Markdown
@@ -393,7 +393,7 @@ Criteria
Badges
-
+
Code
# Full badge names
diff --git a/evaluation/reproduction_report.html b/evaluation/reproduction_report.html
index 7da5395..8e5c140 100644
--- a/evaluation/reproduction_report.html
+++ b/evaluation/reproduction_report.html
@@ -257,11 +257,11 @@ Computationa
Evaluation against guidelines
-
+
-
+
+
+
diff --git a/logbook/logbook.html b/logbook/logbook.html
index a69c8e1..5533a40 100644
--- a/logbook/logbook.html
+++ b/logbook/logbook.html
@@ -260,7 +260,7 @@ Categories
-
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
-
+
diff --git a/logbook/posts/2024_07_31/index.html b/logbook/posts/2024_07_31/index.html
index 3af74bc..b3251bd 100644
--- a/logbook/posts/2024_07_31/index.html
+++ b/logbook/posts/2024_07_31/index.html
@@ -296,7 +296,7 @@ 16.34-16.46: Upload code
Timings
-
+
import sys
sys.path.append('../')
from timings import calculate_times
diff --git a/logbook/posts/2024_08_01/index.html b/logbook/posts/2024_08_01/index.html
index e064bb3..cb74335 100644
--- a/logbook/posts/2024_08_01/index.html
+++ b/logbook/posts/2024_08_01/index.html
@@ -321,7 +321,7 @@ 11.09-1
Timings
-
+
import sys
sys.path.append('../')
from timings import calculate_times
diff --git a/logbook/posts/2024_08_13/index.html b/logbook/posts/2024_08_13/index.html
index c0c49fb..cafb8d3 100644
--- a/logbook/posts/2024_08_13/index.html
+++ b/logbook/posts/2024_08_13/index.html
@@ -317,7 +317,7 @@ 16.01-16.41: Import epicR
Timings
-
+
import sys
sys.path.append('../')
from timings import calculate_times
diff --git a/logbook/posts/2024_08_14/index.html b/logbook/posts/2024_08_14/index.html
index 4438fa1..747bec4 100644
--- a/logbook/posts/2024_08_14/index.html
+++ b/logbook/posts/2024_08_14/index.html
@@ -283,7 +283,7 @@ 09.27-09.52, 10.20-10.21, 10.40-
Timings
-
+
import sys
sys.path.append('../')
from timings import calculate_times
diff --git a/logbook/posts/2024_09_04/index.html b/logbook/posts/2024_09_04/index.html
index 8a9a347..f71d81a 100644
--- a/logbook/posts/2024_09_04/index.html
+++ b/logbook/posts/2024_09_04/index.html
@@ -284,7 +284,7 @@ 09.19-09.41: Re
11.29-11.34, 11.48-12.00: Reviewing results from 50e+6
This ran in 1.642896 hours (1 hour 38 minutes). The result was still wrong though:
-
+
import pandas as pd
pd.read_csv('mini_analysis_50e6.csv')
@@ -393,7 +393,7 @@
+
import pandas as pd
# Import results
@@ -599,7 +599,7 @@
+
# Filter to columns not influenced by agent number
s1 = df_s1nocd[[
'SABA', 'LAMA', 'LAMALABA', 'ICSLAMALABA', 'MildPY','ModeratePY', 'SeverePY',
@@ -752,7 +752,7 @@ 15.26-15.
Then ran on remote machine. It took a bit of tweaking to correct the saving into outputs folder and get right paths, as I got the path wrong, and then it wouldn’t work before had made “outputs” folder.
Finishing this, I got five results tables (saved to csv).
-
+
pd.read_csv('sall_5e5.csv').sort_values(by='Scenario')
@@ -900,7 +900,7 @@ 15.26-15.
-
+
pd.read_csv('ceplane_5e5.csv')
@@ -1144,7 +1144,7 @@ 15.26-15.
-
+
pd.read_csv('clinicalresults_5e5.csv')
@@ -1422,7 +1422,7 @@ 15.26-15.
Timings
-
+
import sys
sys.path.append('../')
from timings import calculate_times
diff --git a/logbook/posts/2024_09_05/index.html b/logbook/posts/2024_09_05/index.html
index 470151a..950e2b1 100644
--- a/logbook/posts/2024_09_05/index.html
+++ b/logbook/posts/2024_09_05/index.html
@@ -317,7 +317,7 @@ 10.20-11.30: Resuming Tab
Add the 5 year results to Table 3 - still assuming that S0 is S1NoCD from 3 year (not from from 5 year or S2 or S3), as can’t spot an S0 anywhere.
Due to size of table, it was difficult to just look between the two tables, so I converted and simplified Table 3 from the paper into a .csv file, and then appended that so I could calculate the differences.
Although some things are similar (e.g. scenario 1 costs and QALYs), others are very different…
-
+
import pandas as pd
pd.set_option('display.max_columns', None)
@@ -792,7 +792,7 @@ 10.20-11.30: Resuming Tab
11.32-11.36, 13.08-13.10: Trying higher agent numbers
Curious is this is due to the number of base agents, I re-ran with 1e6 base_agents (1 million) instead of 5e5 (500,000). On the remote machine, this took 17.1 + 17.1 = 34.2 minutes.
-
+
pd.read_csv('sall_compare_to_paper_1e6.csv')
@@ -1265,7 +1265,7 @@
+
pd.read_csv('sall_compare_1e6_to_mine.csv')
@@ -1843,7 +1843,7 @@
+
pd.read_csv('sall_compare_1e6_to_paper.csv')
@@ -2464,7 +2464,7 @@
+
pd.read_csv('tab3_compare_to_original.csv')
@@ -3050,7 +3050,7 @@ 16.02-16.33: Looking at ICER and INMB
I tried to see if I was using the wrong ICER, or what might explain the difference.
Looking at table 3 in the paper, if I were to calculate the ICER manually from change in costs over change in QALYs, I would get:
-
+
(2438-2151)/(12.560-12.546)
20499.999999998356
@@ -3061,7 +3061,7 @@ 16.02-16.33: Look
Timings
-
+
import sys
sys.path.append('../')
from timings import calculate_times
diff --git a/logbook/posts/2024_09_06/index.html b/logbook/posts/2024_09_06/index.html
index 1840e4e..d583c6d 100644
--- a/logbook/posts/2024_09_06/index.html
+++ b/logbook/posts/2024_09_06/index.html
@@ -412,7 +412,7 @@ 13.15-13.20, 13.25-
That’s repeated a total of 8 times, for the reference case and then for the scenario analyses (so, 26 days 8 hours).
It appears we may therefore need to explore access to a HPC.
The result was:
-
+
import pandas as pd
pd.read_csv('s1NoCD_50mil.csv')
@@ -579,7 +579,7 @@ 15.10-15.40, 15
mutate(AdjDivideByPYAll = AdjDiagnosedPYs / DiagnosedpPYAll)
write.csv(diag_investigate, "../../logbook/posts/2024_09_06/diag_investigate.csv")
-
+
pd.read_csv('diag_investigate.csv')
@@ -631,7 +631,7 @@ 15.10-15.40, 15
Timings
-
+
import sys
sys.path.append('../')
from timings import calculate_times
diff --git a/logbook/posts/2024_09_09/index.html b/logbook/posts/2024_09_09/index.html
index f63755f..c098c09 100644
--- a/logbook/posts/2024_09_09/index.html
+++ b/logbook/posts/2024_09_09/index.html
@@ -302,7 +302,7 @@ 16.08-16.18, 16.23-16
Timings
-
+
import sys
sys.path.append('../')
from timings import calculate_times
diff --git a/logbook/posts/2024_09_10/index.html b/logbook/posts/2024_09_10/index.html
index 094c2e8..14ecae0 100644
--- a/logbook/posts/2024_09_10/index.html
+++ b/logbook/posts/2024_09_10/index.html
@@ -282,7 +282,7 @@ 1
Timings
-
+
import sys
sys.path.append('../')
from timings import calculate_times
diff --git a/logbook/posts/2024_09_16/index.html b/logbook/posts/2024_09_16/index.html
index 0318656..50a56d2 100644
--- a/logbook/posts/2024_09_16/index.html
+++ b/logbook/posts/2024_09_16/index.html
@@ -288,7 +288,7 @@ 09.59-10.00: Testing seeds
Timings
-
+
import sys
sys.path.append('../')
from timings import calculate_times
diff --git a/logbook/posts/2024_09_24/index.html b/logbook/posts/2024_09_24/index.html
index fd9717f..14f94ba 100644
--- a/logbook/posts/2024_09_24/index.html
+++ b/logbook/posts/2024_09_24/index.html
@@ -274,7 +274,7 @@ 09.31-09.38: Archive scope
Timings
-
+
import sys
sys.path.append('../')
from timings import calculate_times
diff --git a/logbook/posts/2024_10_11/index.html b/logbook/posts/2024_10_11/index.html
index 7e971ee..155bf25 100644
--- a/logbook/posts/2024_10_11/index.html
+++ b/logbook/posts/2024_10_11/index.html
@@ -273,7 +273,7 @@
Timings
-
+
import sys
sys.path.append('../')
from timings import calculate_times
diff --git a/logbook/posts/2024_10_14/index.html b/logbook/posts/2024_10_14/index.html
index e16693f..b608f8e 100644
--- a/logbook/posts/2024_10_14/index.html
+++ b/logbook/posts/2024_10_14/index.html
@@ -280,7 +280,7 @@
+
import pandas as pd
pd.set_option('display.max_columns', None)
@@ -461,7 +461,7 @@
+
pd.read_csv('50m_tab3_compare_to_original.csv')
@@ -1089,7 +1089,7 @@
Timings
-
+
import sys
sys.path.append('../')
from timings import calculate_times
diff --git a/logbook/posts/2024_10_15/index.html b/logbook/posts/2024_10_15/index.html
index dbea87a..6c063f3 100644
--- a/logbook/posts/2024_10_15/index.html
+++ b/logbook/posts/2024_10_15/index.html
@@ -282,7 +282,7 @@
Timings
-
+
import sys
sys.path.append('../')
from timings import calculate_times
diff --git a/logbook/posts/2024_10_16/index.html b/logbook/posts/2024_10_16/index.html
index 634a129..20889d4 100644
--- a/logbook/posts/2024_10_16/index.html
+++ b/logbook/posts/2024_10_16/index.html
@@ -293,7 +293,7 @@ 09.28-09.31
Timings
-
+
import sys
sys.path.append('../')
from timings import calculate_times
diff --git a/logbook/posts/2024_10_18/index.html b/logbook/posts/2024_10_18/index.html
index adde579..b60dc9d 100644
--- a/logbook/posts/2024_10_18/index.html
+++ b/logbook/posts/2024_10_18/index.html
@@ -240,6 +240,8 @@ On this page
- 09.15-09.20, 09.27-09.33: Review sensitivity analysis with 100 million
+ - Untimed: Consensus
+ - 10.56-11.14: Checking Figure 4 code
- Timings
@@ -261,34 +263,107 @@ On this page
-X. Total time used: X
+Review sensitivity analysis 100 million results, consensus on reproduction success, fixed issue with medication adherence scenarios and set these to re-run. Total time used: 19h 35m (49.0%).
09.15-09.20, 09.27-09.33: Review sensitivity analysis with 100 million
-Ran Process_Sensitivity_Analysis.Rmd. These are changed to be much more similar to the original, and I feel reasonably similar enough to consider it reproduced, but will certainly need to discuss this, as there are obviously some differences too, so it feels - in this case - a very subjective decision!
+Run times:
+For 3 and 5 years of each scenario:
+
+- Sen1_No_Smoking_Cessation 1.275262 days and 1.361881 days
+- Sen2_50_Med_Adherence 1.2965 days and 1.269854 days
+- Sen3_30_Med_Adherence 1.210629 days and 1.186785 days
+- Sen4_01_Treat_Util 1.291363 days and 1.415365 days
+- Sen5_0_Treat_Util 1.45615 days and 1.315001 days
+- Sen6_0_Discount 1.148049 days and 1.163011 days
+- Sen7_3_Discount 1.217285 days and 1.276366 days
+
+These were all run at once, so the total time was 1.45615 days = 34.9476 hours = 34 hours 57 minutes (or 1 day 10 hours 57 minutes)
+Model results: Ran Process_Sensitivity_Analysis.Rmd. These are changed to be much more similar to the original, and I feel reasonably similar enough to consider it reproduced, but will certainly need to discuss this, as there are obviously some differences too, so it feels - in this case - a very subjective decision!
-
-Timings
-
+
+Untimed: Consensus
+Agreed with Tom:
+
+- Table 3: Reproduced. Costs and QALYs are incredibly similar. Because ICER and INMB derive from the incremental differences, the nature of those calculations means they are very impacted by small differences. However, we are satisified that these are now looking similar enough (and note that none change about the willingness to pay threshold) that this can be considered reproduced, for our purposes.
+- Figure 3: Reproduced.
+- Figure 4: Not reproduced. All points go across really flat and don’t show the pattern of dropping down. I will check if these are implemented correctly, given Apx 7 is all correct, and given some of these look right, it might be an issue in my implementation of the scenarios.
+- Appendix 6: Not reproduced. Due to difference in SABA.
+- Appendix 7: Reproduced. Super duper similar.
+
+Hence, for the three now considered reproduced, the timings for these are:
+Table 3 and Figure 3: Reproduced as of 09.53 on Day 14.
+Timings for table 3 and figure 3:
+
import sys
sys.path.append('../')
from timings import calculate_times
# Minutes used prior to today
-used_to_date = 1146
+used_to_date = 1111
# Times from today
times = [
- ('09.15', '09.20'),
- ('09.27', '09.33')]
-
-calculate_times(used_to_date, times)
+ ('09.40', '09.53')]
+
+calculate_times(used_to_date, times, limit=False)
+
+Time spent today: 13m, or 0h 13m
+Total used to date: 1124m, or 18h 44m
+
+
+Appendix 7: Reproduced as of 09.33 today.
+Timings for appendix 7:
+
+import sys
+sys.path.append('../')
+from timings import calculate_times
+
+# Minutes used prior to today
+used_to_date = 1146
+
+# Times from today
+times = [
+ ('09.15', '09.20'),
+ ('09.27', '09.33')]
+
+calculate_times(used_to_date, times, limit=False)
Time spent today: 11m, or 0h 11m
-Total used to date: 1157m, or 19h 17m
-Time remaining: 1243m, or 20h 43m
-Used 48.2% of 40 hours max
+Total used to date: 1157m, or 19h 17m
+
+
+
+
+10.56-11.14: Checking Figure 4 code
+Comparing our figure to the paper, our primary concern is 50% medication adherence and (in particular) 30% medication adherence, as both show very little change from prior points (moreso for 30%), making me suspect it could be an issue in the implementation of these scenarios.
+I realised that, though I had changed medication_adherence to 0.5 or 0.3, there was actually another variable used in the analysis called med_adherence which I hadn’t changed, which was actually responsible for the change in the model.
+I changed medication_adherence to med_adherence in the two base case files, then amended Sensitivity_Analysis_Prep.R to change the correct variable, so the new Sensitivity Analysis .Rmd had the correct change.
+I then set the medication adherence scenarios to run again. Note: Given long run time, will have to wait for that again, so although can speed up when run in parallel, when something like this happens, have to start from scratch again.
+
+
+Timings
+
+import sys
+sys.path.append('../')
+from timings import calculate_times
+
+# Minutes used prior to today
+used_to_date = 1146
+
+# Times from today
+times = [
+ ('09.15', '09.20'),
+ ('09.27', '09.33'),
+ ('10.56', '11.14')]
+
+calculate_times(used_to_date, times)
+
+Time spent today: 29m, or 0h 29m
+Total used to date: 1175m, or 19h 35m
+Time remaining: 1225m, or 20h 25m
+Used 49.0% of 40 hours max
diff --git a/search.json b/search.json
index d1e313b..129726c 100644
--- a/search.json
+++ b/search.json
@@ -98,123 +98,207 @@
"text": "Appendix 7\nConsensus: Successfully reproduced\nReproduction:"
},
{
- "objectID": "logbook/posts/2024_10_14/index.html",
- "href": "logbook/posts/2024_10_14/index.html",
- "title": "Day 13",
+ "objectID": "logbook/posts/2024_10_18/index.html",
+ "href": "logbook/posts/2024_10_18/index.html",
+ "title": "Day 16",
"section": "",
- "text": "Note\n\n\n\nReviewed results from 50 million (as in their repository) and decided to run 100 million (as in paper). Total time used: 18h 31m (46.3%)"
+ "text": "Note\n\n\n\nReview sensitivity analysis 100 million results, consensus on reproduction success, fixed issue with medication adherence scenarios and set these to re-run. Total time used: 19h 35m (49.0%)."
},
{
- "objectID": "logbook/posts/2024_10_14/index.html#review-results-from-50-million-agents-and-set-to-run-with-100-million",
- "href": "logbook/posts/2024_10_14/index.html#review-results-from-50-million-agents-and-set-to-run-with-100-million",
- "title": "Day 13",
- "section": "09.16-09.19, 09.36-10.22: Review results from 50 million agents and set to run with 100 million",
- "text": "09.16-09.19, 09.36-10.22: Review results from 50 million agents and set to run with 100 million\nI looked over them, comparing them to the article.\nI had to remind myself of what appendix 6 v1 vs v2 were, recalling that I’d remained uncertain on whether the implementation of this figure was correct. Version 2 was the “more correct”, so I removed version 1. What had remained uncertain for version 2 was how to adjust the results. I had multiplied by 2000, and found that resulted in all bars matching except SABA, and this still remained the case.\nFor table 3, I add back in the code for comparing against the original.\nThe results from 50 million are closer to the original.\nRun times:\n\nCase_Detection_Results.md - 14.13 hours - 14 hours 8 minutes\nCase_Detection_Results_5yrs.md - 15.38 hours - 15 hours 23 minutes\n\nAlthough note, these were run in parallel to each other, so total time required (with our machine) was 15 hours 23 minutes.\nTable 3:\n\nimport pandas as pd\npd.set_option('display.max_columns', None)\n\npd.read_csv('50m_tab3.csv')\n\n\n\n\n\n\n\n\nScenario\nTesting interval\nCosts per patient\nQALYs per patient\nICER\nINMB (ranking)\n\n\n\n\n0\nS0: No case detection\nNaN\n$2149\n12.545\nNaN\nNaN\n\n\n1\n(S1a) CDQ ≥ 17 points\n3 years\n$2440\n12.562\n17669.0\n533(1)\n\n\n2\n(S1a) CDQ ≥ 17 points\n5 years\n$2355\n12.556\n19073.0\n335(2)\n\n\n3\n(S1b) Screening spirometry\n3 years\n$2362\n12.553\n26497.0\n189(5)\n\n\n4\n(S1b) Screening spirometry\n5 years\n$2298\n12.552\n22920.0\n177(8)\n\n\n5\n(S1c) CDQ + screening spirometry\n3 years\n$2387\n12.550\n46445.0\n18(16)\n\n\n6\n(S1c) CDQ + screening spirometry\n5 years\n$2314\n12.551\n27818.0\n132(9)\n\n\n7\n(S2a) Screening spirometry\n3 years\n$2286\n12.553\n18082.0\n242(4)\n\n\n8\n(S2a) Screening spirometry\n5 years\n$2241\n12.549\n27650.0\n74(11)\n\n\n9\n(S3a) CDQ ≥ 19.5 points\n3 years\n$2234\n12.548\n28714.0\n63(14)\n\n\n10\n(S3a) CDQ ≥ 19.5 points\n5 years\n$2203\n12.548\n22107.0\n68(12)\n\n\n11\n(S3b) CDQ ≥ 16.5 points\n3 years\n$2289\n12.552\n21354.0\n188(6)\n\n\n12\n(S3b) CDQ ≥ 16.5 points\n5 years\n$2245\n12.552\n13503.0\n260(3)\n\n\n13\n(S3c) Screening spirometry\n3 years\n$2253\n12.549\n28161.0\n81(10)\n\n\n14\n(S3c) Screening spirometry\n5 years\n$2220\n12.550\n14224.0\n178(7)\n\n\n15\n(S3d) CDQ + screening spirometry\n3 years\n$2262\n12.549\n33377.0\n56(15)\n\n\n16\n(S3d) CDQ + screening spirometry\n5 years\n$2223\n12.548\n26612.0\n65(13)\n\n\n\n\n\n\n\n\npd.read_csv('50m_tab3_compare_to_original.csv')\n\n\n\n\n\n\n\n\nScenario\nInterval\nCostpAgent_paper\nCostpAgentAll\nCostpAgentAll_repo\nCostMyDiff\nCostMyPerc\nCostRepoDiff\nCostRepoPerc\nQALYpAgent_paper\nQALYpAgentAll\nQALYpAgentAll_repo\nQALYMyDiff\nQALYMyPerc\nQALYRepoDiff\nQALYRepoPerc\nICER_paper\nICERAdj\nICERAdj_repo\nICERMyDiff\nICERMyPerc\nICERRepoDiff\nICERRepoPerc\nIncrementalNMB_paper\nINMB\nINMB_repo\nINMBMyDiff\nINMBMyPerc\nINMBRepoDiff\nINMBRepoPerc\n\n\n\n\n0\nS0\nNaN\n2151\n2149\n2151\n2\n0.1%\n0\n0.0%\n12.546\n12.545\n12.545\n0.001\n0%\n0.001\n0%\nNaN\nNaN\nNaN\nNaN\nNaN\nNaN\nNaN\nNaN\nNaN\n0\nNaN\nNaN\nNaN\nNaN\n\n\n1\nS1a\n3 years\n2438\n2440\n2439\n-2\n-0.1%\n-1\n0.0%\n12.560\n12.562\n12.560\n-0.002\n0%\n0.000\n0%\n19632.0\n17669.0\n19665.0\n1963.0\n11.1%\n-33.0\n-0.2%\n444.0\n533.0\n446\n-89.0\n-16.7%\n-2.0\n-0.40%\n\n\n2\nS1a\n5 years\n2356\n2355\n2355\n1\n0.0%\n1\n0.0%\n12.556\n12.556\n12.557\n0.000\n0%\n-0.001\n0%\n19847.0\n19073.0\n16600.0\n774.0\n4.1%\n3247.0\n19.6%\n312.0\n335.0\n411\n-23.0\n-6.9%\n-99.0\n-24.10%\n\n\n3\nS1b\n3 years\n2363\n2362\n2365\n1\n0.0%\n-2\n-0.1%\n12.554\n12.553\n12.552\n0.001\n0%\n0.002\n0%\n25894.0\n26497.0\n31076.0\n-603.0\n-2.3%\n-5182.0\n-16.7%\n198.0\n189.0\n131\n9.0\n4.8%\n67.0\n51.10%\n\n\n4\nS1b\n5 years\n2296\n2298\n2298\n-2\n-0.1%\n-2\n-0.1%\n12.552\n12.552\n12.551\n0.000\n0%\n0.001\n0%\n23187.0\n22920.0\n25501.0\n267.0\n1.2%\n-2314.0\n-9.1%\n168.0\n177.0\n142\n-9.0\n-5.1%\n26.0\n18.30%\n\n\n5\nS1c\n3 years\n2386\n2387\n2386\n-1\n0.0%\n0\n0.0%\n12.551\n12.550\n12.551\n0.001\n0%\n0.000\n0%\n46956.0\n46445.0\n42697.0\n511.0\n1.1%\n4259.0\n10.0%\n15.0\n18.0\n40\n-3.0\n-16.7%\n-25.0\n-62.50%\n\n\n6\nS1c\n5 years\n2313\n2314\n2311\n-1\n0.0%\n2\n0.1%\n12.550\n12.551\n12.551\n-0.001\n0%\n-0.001\n0%\n38673.0\n27818.0\n29600.0\n10855.0\n39.0%\n9073.0\n30.7%\n47.0\n132.0\n110\n-85.0\n-64.4%\n-63.0\n-57.30%\n\n\n7\nS2a\n3 years\n2286\n2286\n2287\n0\n0.0%\n-1\n0.0%\n12.553\n12.553\n12.552\n0.000\n0%\n0.001\n0%\n18908.0\n18082.0\n21184.0\n826.0\n4.6%\n-2276.0\n-10.7%\n223.0\n242.0\n186\n-19.0\n-7.9%\n37.0\n19.90%\n\n\n8\nS2a\n5 years\n2246\n2241\n2245\n5\n0.2%\n1\n0.0%\n12.551\n12.549\n12.550\n0.002\n0%\n0.001\n0%\n17514.0\n27650.0\n20394.0\n-10136.0\n-36.7%\n-2880.0\n-14.1%\n176.0\n74.0\n137\n102.0\n137.8%\n39.0\n28.50%\n\n\n9\nS3a\n3 years\n2234\n2234\n2232\n0\n0.0%\n2\n0.1%\n12.548\n12.548\n12.548\n0.000\n0%\n0.000\n0%\n30366.0\n28714.0\n32247.0\n1652.0\n5.8%\n-1881.0\n-5.8%\n54.0\n63.0\n45\n-9.0\n-14.3%\n9.0\n20.00%\n\n\n10\nS3a\n5 years\n2207\n2203\n2207\n4\n0.2%\n0\n0.0%\n12.548\n12.548\n12.548\n0.000\n0%\n0.000\n0%\n22636.0\n22107.0\n22694.0\n529.0\n2.4%\n-58.0\n-0.3%\n68.0\n68.0\n67\n0.0\n0.0%\n1.0\n1.50%\n\n\n11\nS3b\n3 years\n2292\n2289\n2289\n3\n0.1%\n3\n0.1%\n12.553\n12.552\n12.553\n0.001\n0%\n0.000\n0%\n18438.0\n21354.0\n18157.0\n-2916.0\n-13.7%\n281.0\n1.5%\n241.0\n188.0\n243\n53.0\n28.2%\n-2.0\n-0.80%\n\n\n12\nS3b\n5 years\n2250\n2245\n2248\n5\n0.2%\n2\n0.1%\n12.552\n12.552\n12.551\n0.000\n0%\n0.001\n0%\n16251.0\n13503.0\n17325.0\n2748.0\n20.4%\n-1074.0\n-6.2%\n206.0\n260.0\n183\n-54.0\n-20.8%\n23.0\n12.60%\n\n\n13\nS3c\n3 years\n2256\n2253\n2255\n3\n0.1%\n1\n0.0%\n12.550\n12.549\n12.550\n0.001\n0%\n0.000\n0%\n23972.0\n28161.0\n20700.0\n-4189.0\n-14.9%\n3272.0\n15.8%\n114.0\n81.0\n148\n33.0\n40.7%\n-34.0\n-23.00%\n\n\n14\nS3c\n5 years\n2224\n2220\n2221\n4\n0.2%\n3\n0.1%\n12.549\n12.550\n12.549\n-0.001\n0%\n0.000\n0%\n20278.0\n14224.0\n19250.0\n6054.0\n42.6%\n1028.0\n5.3%\n107.0\n178.0\n113\n-71.0\n-39.9%\n-6.0\n-5.30%\n\n\n15\nS3d\n3 years\n2263\n2262\n2261\n1\n0.0%\n2\n0.1%\n12.549\n12.549\n12.550\n0.000\n0%\n-0.001\n0%\n28245.0\n33377.0\n23462.0\n-5132.0\n-15.4%\n4783.0\n20.4%\n86.0\n56.0\n124\n30.0\n53.6%\n-38.0\n-30.60%\n\n\n16\nS3d\n5 years\n2227\n2223\n2226\n4\n0.2%\n1\n0.0%\n12.548\n12.548\n12.548\n0.000\n0%\n0.000\n0%\n27591.0\n26612.0\n26517.0\n979.0\n3.7%\n1074.0\n4.1%\n62.0\n65.0\n67\n-3.0\n-4.6%\n-5.0\n-7.50%\n\n\n\n\n\n\n\nLooking at each of the columns…\n\nCosts per patient - max difference from paper is 5 (0.2%) (e.g. $2250 vs $2245 for S3b 5 years). With lower number of agents, max difference was 23 (1%)\nQALYs per patient - max difference from paper is 0.002 (0%) (e.g. 12.562 vs 12.560). With lower agent numbers, max difference was 0.01 (0.1%)\nICER - max difference is 10,136 (36.7%) (mine 27,650 v.s. paper 17,514) for S2A 5 years. Biggest percentage difference was 42.6% (14,224 vs 20,278). With lower agent numbers, my biggest absolute difference was 93,297, and biggest percentage difference was 458%.\nINMB - max difference was 102 (137.8%) (mine 74 vs paper 176 for S2A 5 years).\n\nDue to the large difference in ICERs and INMB, I would unfortunately say this is not reproduced.\nFigure 3:\n\nThis looks fairly similar, at a glance (unlike with higher agent numbers which was wildly different). I think I would argue this to be reproduced, as the points are pretty close to where they were in the original, although given the tiny scale is this working to, this would definitely require some consensus and second opinion.\nAppendix 6:\n\nThis hasn’t changed much from running with 50 million agents v.s. lower numbers. The same issue remains - that everything is spot on except SABA - and so, unfortunately, still not reproduced. My result is very similar to the result from their repository when they ran with 50 million.\nHaving looked over these, I think it would be worth running this again with 100 million (as in the paper), just to be sure, in case that does happen to resolve the remaining differences in Table 3 (big difference) and Figure 3 (smaller difference) (although unfortunately I suspect the difference might remain for Appendix 6 as that didn’t change with agent numbers)."
+ "objectID": "logbook/posts/2024_10_18/index.html#review-sensitivity-analysis-with-100-million",
+ "href": "logbook/posts/2024_10_18/index.html#review-sensitivity-analysis-with-100-million",
+ "title": "Day 16",
+ "section": "09.15-09.20, 09.27-09.33: Review sensitivity analysis with 100 million",
+ "text": "09.15-09.20, 09.27-09.33: Review sensitivity analysis with 100 million\nRun times:\nFor 3 and 5 years of each scenario:\n\nSen1_No_Smoking_Cessation 1.275262 days and 1.361881 days\nSen2_50_Med_Adherence 1.2965 days and 1.269854 days\nSen3_30_Med_Adherence 1.210629 days and 1.186785 days\nSen4_01_Treat_Util 1.291363 days and 1.415365 days\nSen5_0_Treat_Util 1.45615 days and 1.315001 days\nSen6_0_Discount 1.148049 days and 1.163011 days\nSen7_3_Discount 1.217285 days and 1.276366 days\n\nThese were all run at once, so the total time was 1.45615 days = 34.9476 hours = 34 hours 57 minutes (or 1 day 10 hours 57 minutes)\nModel results: Ran Process_Sensitivity_Analysis.Rmd. These are changed to be much more similar to the original, and I feel reasonably similar enough to consider it reproduced, but will certainly need to discuss this, as there are obviously some differences too, so it feels - in this case - a very subjective decision!"
},
{
- "objectID": "logbook/posts/2024_10_14/index.html#timings",
- "href": "logbook/posts/2024_10_14/index.html#timings",
- "title": "Day 13",
+ "objectID": "logbook/posts/2024_10_18/index.html#untimed-consensus",
+ "href": "logbook/posts/2024_10_18/index.html#untimed-consensus",
+ "title": "Day 16",
+ "section": "Untimed: Consensus",
+ "text": "Untimed: Consensus\nAgreed with Tom:\n\nTable 3: Reproduced. Costs and QALYs are incredibly similar. Because ICER and INMB derive from the incremental differences, the nature of those calculations means they are very impacted by small differences. However, we are satisified that these are now looking similar enough (and note that none change about the willingness to pay threshold) that this can be considered reproduced, for our purposes.\nFigure 3: Reproduced.\nFigure 4: Not reproduced. All points go across really flat and don’t show the pattern of dropping down. I will check if these are implemented correctly, given Apx 7 is all correct, and given some of these look right, it might be an issue in my implementation of the scenarios.\nAppendix 6: Not reproduced. Due to difference in SABA.\nAppendix 7: Reproduced. Super duper similar.\n\nHence, for the three now considered reproduced, the timings for these are:\nTable 3 and Figure 3: Reproduced as of 09.53 on Day 14.\nTimings for table 3 and figure 3:\n\nimport sys\nsys.path.append('../')\nfrom timings import calculate_times\n\n# Minutes used prior to today\nused_to_date = 1111\n\n# Times from today\ntimes = [\n ('09.40', '09.53')]\n\ncalculate_times(used_to_date, times, limit=False)\n\nTime spent today: 13m, or 0h 13m\nTotal used to date: 1124m, or 18h 44m\n\n\nAppendix 7: Reproduced as of 09.33 today.\nTimings for appendix 7:\n\nimport sys\nsys.path.append('../')\nfrom timings import calculate_times\n\n# Minutes used prior to today\nused_to_date = 1146\n\n# Times from today\ntimes = [\n ('09.15', '09.20'),\n ('09.27', '09.33')]\n\ncalculate_times(used_to_date, times, limit=False)\n\nTime spent today: 11m, or 0h 11m\nTotal used to date: 1157m, or 19h 17m"
+ },
+ {
+ "objectID": "logbook/posts/2024_10_18/index.html#checking-figure-4-code",
+ "href": "logbook/posts/2024_10_18/index.html#checking-figure-4-code",
+ "title": "Day 16",
+ "section": "10.56-11.14: Checking Figure 4 code",
+ "text": "10.56-11.14: Checking Figure 4 code\nComparing our figure to the paper, our primary concern is 50% medication adherence and (in particular) 30% medication adherence, as both show very little change from prior points (moreso for 30%), making me suspect it could be an issue in the implementation of these scenarios.\nI realised that, though I had changed medication_adherence to 0.5 or 0.3, there was actually another variable used in the analysis called med_adherence which I hadn’t changed, which was actually responsible for the change in the model.\nI changed medication_adherence to med_adherence in the two base case files, then amended Sensitivity_Analysis_Prep.R to change the correct variable, so the new Sensitivity Analysis .Rmd had the correct change.\nI then set the medication adherence scenarios to run again. Note: Given long run time, will have to wait for that again, so although can speed up when run in parallel, when something like this happens, have to start from scratch again."
+ },
+ {
+ "objectID": "logbook/posts/2024_10_18/index.html#timings",
+ "href": "logbook/posts/2024_10_18/index.html#timings",
+ "title": "Day 16",
"section": "Timings",
- "text": "Timings\n\nimport sys\nsys.path.append('../')\nfrom timings import calculate_times\n\n# Minutes used prior to today\nused_to_date = 1062\n\n# Times from today\ntimes = [\n ('09.16', '09.19'),\n ('09.36', '10.22')]\n\ncalculate_times(used_to_date, times)\n\nTime spent today: 49m, or 0h 49m\nTotal used to date: 1111m, or 18h 31m\nTime remaining: 1289m, or 21h 29m\nUsed 46.3% of 40 hours max"
+ "text": "Timings\n\nimport sys\nsys.path.append('../')\nfrom timings import calculate_times\n\n# Minutes used prior to today\nused_to_date = 1146\n\n# Times from today\ntimes = [\n ('09.15', '09.20'),\n ('09.27', '09.33'),\n ('10.56', '11.14')]\n\ncalculate_times(used_to_date, times)\n\nTime spent today: 29m, or 0h 29m\nTotal used to date: 1175m, or 19h 35m\nTime remaining: 1225m, or 20h 25m\nUsed 49.0% of 40 hours max"
},
{
- "objectID": "evaluation/badges.html",
- "href": "evaluation/badges.html",
- "title": "Journal badges",
+ "objectID": "logbook/posts/2024_09_06/index.html",
+ "href": "logbook/posts/2024_09_06/index.html",
+ "title": "Day 7",
"section": "",
- "text": "Please note: This is a template page and has not yet been completed, so all criteria are currently set as unmet (❌)\nThis page evaluates the extent to which the author-published research artefacts meet the criteria of badges related to reproducibility from various organisations and journals.\nCaveat: Please note that these criteria are based on available information about each badge online, and that we have likely differences in our procedure (e.g. allowed troubleshooting for execution and reproduction, not under tight time pressure to complete). Moreover, we focus only on reproduction of the discrete-event simulation, and not on other aspects of the article. We cannot guarantee that the badges below would have been awarded in practice by these journals."
+ "text": "Note\n\n\n\nWorking on Table 3, Figure 3 and Appendix 6 (none yet reproduced). Total time used: 14h 8m (35.3%)."
},
{
- "objectID": "evaluation/badges.html#criteria",
- "href": "evaluation/badges.html#criteria",
- "title": "Journal badges",
- "section": "Criteria",
- "text": "Criteria\n\n\nCode\nfrom IPython.display import display, Markdown\nimport numpy as np\nimport pandas as pd\n\n# Criteria and their definitions\ncriteria = {\n 'archive': 'Stored in a permanent archive that is publicly and openly accessible',\n 'id': 'Has a persistent identifier',\n 'license': 'Includes an open license',\n 'relevant': '''Artefacts are relevant to and contribute to the article's results''',\n 'complete': 'Complete set of materials shared (as would be needed to fully reproduce article)',\n 'structure': 'Artefacts are well structured/organised (e.g. to the extent that reuse and repurposing is facilitated, adhering to norms and standards of research community)',\n 'documentation_sufficient': 'Artefacts are sufficiently documented (i.e. to understand how it works, to enable it to be run, including package versions)',\n 'documentation_careful': 'Artefacts are carefully documented (more than sufficient - i.e. to the extent that reuse and repurposing is facilitated - e.g. changing parameters, reusing for own purpose)',\n # This criteria is kept seperate to documentation_careful, as it specifically requires a README file\n 'documentation_readme': 'Artefacts are clearly documented and accompanied by a README file with step-by-step instructions on how to reproduce results in the manuscript',\n 'execute': 'Scripts can be successfully executed',\n 'regenerated': 'Independent party regenerated results using the authors research artefacts',\n 'hour': 'Reproduced within approximately one hour (excluding compute time)',\n}\n\n# Evaluation for this study\n# TODO: Complete evaluate for each criteria\neval = pd.Series({\n 'archive': 0,\n 'id': 0,\n 'license': 0,\n 'relevant': 0,\n 'complete': 0,\n 'structure': 0,\n 'documentation_sufficient': 0,\n 'documentation_careful': 0,\n 'documentation_readme': 0,\n 'execute': 0,\n 'regenerated': 0,\n 'hour': 0,\n})\n\n# Get list of criteria met (True/False) overall\neval_list = list(eval)\n\n# Define function for creating the markdown formatted list of criteria met\ndef create_criteria_list(criteria_dict):\n '''\n Creates a string which contains a Markdown formatted list with icons to\n indicate whether each criteria was met\n\n Parameters:\n -----------\n criteria_dict : dict\n Dictionary where keys are the criteria (variable name) and values are\n Boolean (True/False of whether this study met the criteria)\n\n Returns:\n --------\n formatted_list : string\n Markdown formatted list\n '''\n callout_icon = {True: '✅',\n False: '❌'}\n # Create list with...\n formatted_list = ''.join([\n '* ' +\n callout_icon[eval[key]] + # Icon based on whether it met criteria\n ' ' +\n value + # Full text description of criteria\n '\\n' for key, value in criteria_dict.items()])\n return(formatted_list)\n\n# Define groups of criteria\ncriteria_share_how = ['archive', 'id', 'license']\ncriteria_share_what = ['relevant', 'complete']\ncriteria_doc_struc = ['structure', 'documentation_sufficient', 'documentation_careful', 'documentation_readme']\ncriteria_run = ['execute', 'regenerated', 'hour']\n\n# Create text section\ndisplay(Markdown(f'''\nTo assess whether the author's materials met the requirements of each badge, a list of criteria was produced. Between each badge (and between categories of badge), there is often alot of overlap in criteria.\n\nThis study met **{sum(eval_list)} of the {len(eval_list)}** unique criteria items. These were as follows:\n\nCriteria related to how artefacts are shared -\n\n{create_criteria_list({k: criteria[k] for k in criteria_share_how})}\n\nCriteria related to what artefacts are shared -\n\n{create_criteria_list({k: criteria[k] for k in criteria_share_what})}\n\nCriteria related to the structure and documentation of the artefacts -\n\n{create_criteria_list({k: criteria[k] for k in criteria_doc_struc})}\n\nCriteria related to running and reproducing results -\n\n{create_criteria_list({k: criteria[k] for k in criteria_run})}\n'''))\n\n\nTo assess whether the author’s materials met the requirements of each badge, a list of criteria was produced. Between each badge (and between categories of badge), there is often alot of overlap in criteria.\nThis study met 0 of the 12 unique criteria items. These were as follows:\nCriteria related to how artefacts are shared -\n\n❌ Stored in a permanent archive that is publicly and openly accessible\n❌ Has a persistent identifier\n❌ Includes an open license\n\nCriteria related to what artefacts are shared -\n\n❌ Artefacts are relevant to and contribute to the article’s results\n❌ Complete set of materials shared (as would be needed to fully reproduce article)\n\nCriteria related to the structure and documentation of the artefacts -\n\n❌ Artefacts are well structured/organised (e.g. to the extent that reuse and repurposing is facilitated, adhering to norms and standards of research community)\n❌ Artefacts are sufficiently documented (i.e. to understand how it works, to enable it to be run, including package versions)\n❌ Artefacts are carefully documented (more than sufficient - i.e. to the extent that reuse and repurposing is facilitated - e.g. changing parameters, reusing for own purpose)\n❌ Artefacts are clearly documented and accompanied by a README file with step-by-step instructions on how to reproduce results in the manuscript\n\nCriteria related to running and reproducing results -\n\n❌ Scripts can be successfully executed\n❌ Independent party regenerated results using the authors research artefacts\n❌ Reproduced within approximately one hour (excluding compute time)"
+ "objectID": "logbook/posts/2024_09_06/index.html#revisiting-table-3",
+ "href": "logbook/posts/2024_09_06/index.html#revisiting-table-3",
+ "title": "Day 7",
+ "section": "09.36-09.48: Revisiting Table 3",
+ "text": "09.36-09.48: Revisiting Table 3\n\n\n\n\n\n\nReflection\n\n\n\nI’ve been finding this quite tricky/slow to get my head around, and want to acknowledge that part of the reason for that will be that this is a complex model subject and - though having studied and done a little work in health economics - I am not a health economist.\nThere are some things that might have helped though, such as having a clear indicator of which columns and outputs match up with the paper, as its still taking me a while to make sure I’m looking at the same ones, particularly as the results don’t match up.\n\n\nContinuing from yesterday, when I’d realised that incremental costs and QALYs that subjectively look really similar actually result in really different ICERs, I explored the possibility that the difference in results is actually just down to the number of agents.\nLooking in their GitHub commit history, I found results from them running with 1 million base agents (like me here):\n\nCase_Detection_Results.md\nCase_Detection_Results_5yrs.md\n\nComparing those against Table 3, they are very different from them, and from me as well. This was despite having set the seed to 333, so it appears the seed control might not be working as expected, whether that be my fault for any changes or just in general.\nHowever, their latest results (from 50 million agents) still look quite different to the original, e.g. S1b 3yr ICER $31075/QALY in their repository versus $25,894/QALY in the paper.\nHence, I’m wary that running with 50 million agents on my machine - even if I could - still might not get me the same results.\n\n\n\n\n\n\nReflection\n\n\n\nUncertain on if seed control implemented properly. Also, here, it appears that either the number of agents can’t be reduced to get same results, and that a really high number are needed for stability, or that its actually just a seed issue."
},
{
- "objectID": "evaluation/badges.html#badges",
- "href": "evaluation/badges.html#badges",
- "title": "Journal badges",
- "section": "Badges",
- "text": "Badges\n\n\nCode\n# Full badge names\nbadge_names = {\n # Open objects\n 'open_niso': 'NISO \"Open Research Objects (ORO)\"',\n 'open_niso_all': 'NISO \"Open Research Objects - All (ORO-A)\"',\n 'open_acm': 'ACM \"Artifacts Available\"',\n 'open_cos': 'COS \"Open Code\"',\n 'open_ieee': 'IEEE \"Code Available\"',\n # Object review\n 'review_acm_functional': 'ACM \"Artifacts Evaluated - Functional\"',\n 'review_acm_reusable': 'ACM \"Artifacts Evaluated - Reusable\"',\n 'review_ieee': 'IEEE \"Code Reviewed\"',\n # Results reproduced\n 'reproduce_niso': 'NISO \"Results Reproduced (ROR-R)\"',\n 'reproduce_acm': 'ACM \"Results Reproduced\"',\n 'reproduce_ieee': 'IEEE \"Code Reproducible\"',\n 'reproduce_psy': 'Psychological Science \"Computational Reproducibility\"'\n}\n\n# Criteria required by each badge\nbadges = {\n # Open objects\n 'open_niso': ['archive', 'id', 'license'],\n 'open_niso_all': ['archive', 'id', 'license', 'complete'],\n 'open_acm': ['archive', 'id'],\n 'open_cos': ['archive', 'id', 'license', 'complete', 'documentation_sufficient'],\n 'open_ieee': ['complete'],\n # Object review\n 'review_acm_functional': ['documentation_sufficient', 'relevant', 'complete', 'execute'],\n 'review_acm_reusable': ['documentation_sufficient', 'documentation_careful', 'relevant', 'complete', 'execute', 'structure'],\n 'review_ieee': ['complete', 'execute'],\n # Results reproduced\n 'reproduce_niso': ['regenerated'],\n 'reproduce_acm': ['regenerated'],\n 'reproduce_ieee': ['regenerated'],\n 'reproduce_psy': ['regenerated', 'hour', 'structure', 'documentation_readme'],\n}\n\n# Identify which badges would be awarded based on criteria\n# Get list of badges met (True/False) overall\naward = {}\nfor badge in badges:\n award[badge] = all([eval[key] == 1 for key in badges[badge]])\naward_list = list(award.values())\n\n# Write introduction\n# Get list of badges met (True/False) by category\naward_open = [v for k,v in award.items() if k.startswith('open_')]\naward_review = [v for k,v in award.items() if k.startswith('review_')]\naward_reproduce = [v for k,v in award.items() if k.startswith('reproduce_')]\n\n# Create and display text for introduction\ndisplay(Markdown(f'''\nIn total, the original study met the criteria for **{sum(award_list)} of the {len(award_list)} badges**. This included:\n\n* **{sum(award_open)} of the {len(award_open)}** “open objects” badges\n* **{sum(award_review)} of the {len(award_review)}** “object review” badges\n* **{sum(award_reproduce)} of the {len(award_reproduce)}** “reproduced” badges\n'''))\n\n# Make function that creates collapsible callouts for each badge\ndef create_badge_callout(award_dict):\n '''\n Displays Markdown callouts created for each badge in the dictionary, showing\n whether the criteria for that badge was met.\n\n Parameters:\n -----------\n award_dict : dict\n Dictionary where key is badge (as variable name), and value is Boolean\n (whether badge is awarded)\n '''\n callout_appearance = {True: 'tip',\n False: 'warning'}\n callout_icon = {True: '✅',\n False: '❌'}\n callout_text = {True: 'Meets all criteria:',\n False: 'Does not meet all criteria:'}\n\n for key, value in award_dict.items():\n # Create Markdown list with...\n criteria_list = ''.join([\n '* ' +\n callout_icon[eval[k]] + # Icon based on whether it met criteria\n ' ' +\n criteria[k] + # Full text description of criteria\n '\\n' for k in badges[key]])\n # Create the callout and display it\n display(Markdown(f'''\n::: {{.callout-{callout_appearance[value]} appearance=\"minimal\" collapse=true}}\n\n## {callout_icon[value]} {badge_names[key]}\n\n{callout_text[value]}\n\n{criteria_list}\n:::\n'''))\n\n# Create badge functions with introductions and callouts\ndisplay(Markdown('''\n### \"Open objects\" badges\n\nThese badges relate to research artefacts being made openly available.\n'''))\ncreate_badge_callout({k: v for (k, v) in award.items() if k.startswith('open_')})\n\ndisplay(Markdown('''\n### \"Object review\" badges\n\nThese badges relate to the research artefacts being reviewed against criteria of the badge issuer.\n'''))\ncreate_badge_callout({k: v for (k, v) in award.items() if k.startswith('review_')})\n\ndisplay(Markdown('''\n### \"Reproduced\" badges\n\nThese badges relate to an independent party regenerating the reuslts of the article using the author objects.\n'''))\ncreate_badge_callout({k: v for (k, v) in award.items() if k.startswith('reproduce_')})\n\n\nIn total, the original study met the criteria for 0 of the 12 badges. This included:\n\n0 of the 5 “open objects” badges\n0 of the 3 “object review” badges\n0 of the 4 “reproduced” badges\n\n\n\n“Open objects” badges\nThese badges relate to research artefacts being made openly available.\n\n\n\n\n\n\n\n\n❌ NISO “Open Research Objects (ORO)”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Stored in a permanent archive that is publicly and openly accessible\n❌ Has a persistent identifier\n❌ Includes an open license\n\n\n\n\n\n\n\n\n\n\n\n\n❌ NISO “Open Research Objects - All (ORO-A)”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Stored in a permanent archive that is publicly and openly accessible\n❌ Has a persistent identifier\n❌ Includes an open license\n❌ Complete set of materials shared (as would be needed to fully reproduce article)\n\n\n\n\n\n\n\n\n\n\n\n\n❌ ACM “Artifacts Available”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Stored in a permanent archive that is publicly and openly accessible\n❌ Has a persistent identifier\n\n\n\n\n\n\n\n\n\n\n\n\n❌ COS “Open Code”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Stored in a permanent archive that is publicly and openly accessible\n❌ Has a persistent identifier\n❌ Includes an open license\n❌ Complete set of materials shared (as would be needed to fully reproduce article)\n❌ Artefacts are sufficiently documented (i.e. to understand how it works, to enable it to be run, including package versions)\n\n\n\n\n\n\n\n\n\n\n\n\n❌ IEEE “Code Available”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Complete set of materials shared (as would be needed to fully reproduce article)\n\n\n\n\n\n\n“Object review” badges\nThese badges relate to the research artefacts being reviewed against criteria of the badge issuer.\n\n\n\n\n\n\n\n\n❌ ACM “Artifacts Evaluated - Functional”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Artefacts are sufficiently documented (i.e. to understand how it works, to enable it to be run, including package versions)\n❌ Artefacts are relevant to and contribute to the article’s results\n❌ Complete set of materials shared (as would be needed to fully reproduce article)\n❌ Scripts can be successfully executed\n\n\n\n\n\n\n\n\n\n\n\n\n❌ ACM “Artifacts Evaluated - Reusable”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Artefacts are sufficiently documented (i.e. to understand how it works, to enable it to be run, including package versions)\n❌ Artefacts are carefully documented (more than sufficient - i.e. to the extent that reuse and repurposing is facilitated - e.g. changing parameters, reusing for own purpose)\n❌ Artefacts are relevant to and contribute to the article’s results\n❌ Complete set of materials shared (as would be needed to fully reproduce article)\n❌ Scripts can be successfully executed\n❌ Artefacts are well structured/organised (e.g. to the extent that reuse and repurposing is facilitated, adhering to norms and standards of research community)\n\n\n\n\n\n\n\n\n\n\n\n\n❌ IEEE “Code Reviewed”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Complete set of materials shared (as would be needed to fully reproduce article)\n❌ Scripts can be successfully executed\n\n\n\n\n\n\n“Reproduced” badges\nThese badges relate to an independent party regenerating the reuslts of the article using the author objects.\n\n\n\n\n\n\n\n\n❌ NISO “Results Reproduced (ROR-R)”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Independent party regenerated results using the authors research artefacts\n\n\n\n\n\n\n\n\n\n\n\n\n❌ ACM “Results Reproduced”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Independent party regenerated results using the authors research artefacts\n\n\n\n\n\n\n\n\n\n\n\n\n❌ IEEE “Code Reproducible”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Independent party regenerated results using the authors research artefacts\n\n\n\n\n\n\n\n\n\n\n\n\n❌ Psychological Science “Computational Reproducibility”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Independent party regenerated results using the authors research artefacts\n❌ Reproduced within approximately one hour (excluding compute time)\n❌ Artefacts are well structured/organised (e.g. to the extent that reuse and repurposing is facilitated, adhering to norms and standards of research community)\n❌ Artefacts are clearly documented and accompanied by a README file with step-by-step instructions on how to reproduce results in the manuscript"
+ "objectID": "logbook/posts/2024_09_06/index.html#working-on-figure-3",
+ "href": "logbook/posts/2024_09_06/index.html#working-on-figure-3",
+ "title": "Day 7",
+ "section": "09.49-10.40: Working on Figure 3",
+ "text": "09.49-10.40: Working on Figure 3\nTaking a break on Table 3, I looked to Figure 3. The current model code does output two visually similar figures, and it appears I would just need to combine these to get the figure from the paper. This was very easy to get started with - just copying it over, but tweaking to add a grouping for 3 years and 5 years.\nI add the efficiency frontier based on the description in the text - looking for lowest cost-effectiveness ratio, and then ruling out strategies that were less effective with a higher ICER. I’m not 100% certain I’ve implemented it correctly, but certainly, it looks different any since the points are all different.\n\n\n\nFigure 3 Attempt\n\n\n\n\n\n\n\n\nReflection\n\n\n\nWas really helpful having partial code for figures, but full code would have been fab (as e.g. I was uncertain if I’m implementing efficiency frontier correctly)."
},
{
- "objectID": "evaluation/badges.html#sources",
- "href": "evaluation/badges.html#sources",
- "title": "Journal badges",
- "section": "Sources",
- "text": "Sources\nNational Information Standards Organisation (NISO) (NISO Reproducibility Badging and Definitions Working Group (2021))\n\n“Open Research Objects (ORO)”\n“Open Research Objects - All (ORO-A)”\n“Results Reproduced (ROR-R)”\n\nAssociation for Computing Machinery (ACM) (Association for Computing Machinery (ACM) (2020))\n\n“Artifacts Available”\n“Artifacts Evaluated - Functional”\n“Artifacts Evaluated - Resuable”\n“Results Reproduced”\n\nCenter for Open Science (COS) (Blohowiak et al. (2023))\n\n“Open Code”\n\nInstitute of Electrical and Electronics Engineers (IEEE) (Institute of Electrical and Electronics Engineers (IEEE) (n.d.))\n\n“Code Available”\n“Code Reviewed”\n“Code Reproducible”\n\nPsychological Science (Hardwicke and Vazire (2023) and Association for Psychological Science (APS) (2023))\n\n“Computational Reproducibility”"
+ "objectID": "logbook/posts/2024_09_06/index.html#troubleshooting-difference-in-results",
+ "href": "logbook/posts/2024_09_06/index.html#troubleshooting-difference-in-results",
+ "title": "Day 7",
+ "section": "11.00-11.11: Troubleshooting difference in results",
+ "text": "11.00-11.11: Troubleshooting difference in results\nI stuck with Table 3 and Figure 3, trying to figure out the reason for the differing results.\nI checked whether the incremental costs and QALYs were based on the adjusted or unadjusted results and, from trying a calculation manually, could see it definitely uses the adjusted.\nI looked through the history of some results in their GitHub, just to help clarify to myself the discrepancies I’m seeing v.s. what they see.\nICERs:\n\n\n\nScenario (3yr)\nTable 3\nMe (1m)\nThem (50m)\nThem (1m)\n\n\n\n\nS1a\n19632\n14171\n19664\n26317\n\n\nS2a\n18908\n7905\n21183\n16496\n\n\nS3a\n30366\n5438\n32247\n-36092\n\n\n\nThis reaffirms to me - again - that the problem is likely agent number, and so the best course of action would be to run it with 50 million (like their repository) and potentially even 100 million (like their paper), although I’m not sure if our computer will be capable of it."
},
{
- "objectID": "quarto_site/reproduction_readme.html",
- "href": "quarto_site/reproduction_readme.html",
- "title": "README for reproduction",
+ "objectID": "logbook/posts/2024_09_06/index.html#running-one-scenario-with-50-million",
+ "href": "logbook/posts/2024_09_06/index.html#running-one-scenario-with-50-million",
+ "title": "Day 7",
+ "section": "11.12-11.18: Running one scenario with 50 million",
+ "text": "11.12-11.18: Running one scenario with 50 million\nI remade a mini analysis file with a single scenario, so we can test how long the run takes without trying to run all at once."
+ },
+ {
+ "objectID": "logbook/posts/2024_09_06/index.html#appendix-6",
+ "href": "logbook/posts/2024_09_06/index.html#appendix-6",
+ "title": "Day 7",
+ "section": "11.24-12.15: Appendix 6",
+ "text": "11.24-12.15: Appendix 6\nAppendix 6 presents the clinical results from S1a 3 yrs. I couldn’t spot code for these figures in the closedcohort .Rmd files. I could see the package itself had plotting functions in figures.R and some of these appeared potentially relevant. This appeared to need to be called whilst running the model (rather than using the model outputs we had saved to csv).\nI made a new .Rmd file to run S1A, reduce from 1e6 to 50000 just while testing. Looking through the package, it wasn’t immediately clear to me how/when to call the export_figures() function. I tried adding it after all the S1A code. This had error:\nError in `cost_by_GOLD[, 2:5] <- (op_ex$cumul_cost_gold_ctime[, 2:5])`:\n! number of items to replace is not a multiple of replacement length\nBacktrace:\n 1. epicR::export_figures()\nI tried instead putting it before s1a’s terminate_session(), but the error persisted.\nI tried a different approach, to instead manually copy of the relevant code from export_figures() so I can altered it more closely and figure out why its breaking. Looking at the function, it was running the model itself and using:\ndata <- as.data.frame(Cget_all_events_matrix())\nop <- Cget_output()\nop_ex <- Cget_output_ex()\nI then realised that actually, the results we need might just be in the output S1.csv, and that these figures are more complicated than we require, so I instead looked to make them manually.\nThere are columns with Mild, Moderate, etc. counts but this needs to be transformed to per 1000. I wasn’t sure whether to:\n\nDivide by 1000 (as 1 million / 1000 = 1000)\nUse the “PY” results and multiply by 1000 (as that appears to be a bit like what they do in export_figures() although not super sure)\n\nEach of the two methods came out with different results. I searched in the repository but couldn’t find anything about MildPY or the others, so decided to go with the non-PY method, since I wasn’t actually sure what PY stood for, and whether it might actually be to do with “pack years” or “patient years”.\nDoing it manually, I made the first of the four figures, and found it actually looked very visually similar to the original, even if the number of exacerbations was lower. I suspected this was an issue with my adjustments. Looking at s1.csv, the Agents are actually lower than the base agents we set (apx 700k v.s. 1m). I tried using that to adjust - and that worked out much closer."
+ },
+ {
+ "objectID": "logbook/posts/2024_09_06/index.html#result-from-50-million",
+ "href": "logbook/posts/2024_09_06/index.html#result-from-50-million",
+ "title": "Day 7",
+ "section": "13.15-13.20, 13.25-13.28: Result from 50 million",
+ "text": "13.15-13.20, 13.25-13.28: Result from 50 million\nTo run S1NoCD with 50 million base agents, it took 1.644506 hours (1 hour 38 minutes)\nThere’s 24 scenarios, so running those with 50 million would take apx. 1 day 15 hours 28 minutes. With 100 million (as in the paper), it would be at least double that (so at least 3 days 7 hours).\nThat’s repeated a total of 8 times, for the reference case and then for the scenario analyses (so, 26 days 8 hours).\nIt appears we may therefore need to explore access to a HPC.\nThe result was:\n\nimport pandas as pd\n\npd.read_csv('s1NoCD_50mil.csv')\n\n\n\n\n\n\n\n\nScenario\nAgents\nPatientYears\nCopdPYs\nNCaseDetections\nDiagnosedPYs\nOverdiagnosedPYs\nSABA\nLAMA\nLAMALABA\n...\nNoCOPD\nGOLD1\nGOLD2\nGOLD3\nGOLD4\nCost\nCostpAgent\nQALY\nQALYpAgent\nNMB\n\n\n\n\n0\nS1NoCD\n37196486\n6.259536e+08\n7.111802e+07\n190966216\n1.316646e+07\n13371458\n0.017152\n0.135457\n0.151286\n...\n527427474\n28845970\n30727603\n6872520\n1191666\n7.986489e+10\n2147.108537\n4.666148e+08\n12.544592\n625082.515224\n\n\n\n\n1 rows × 29 columns"
+ },
+ {
+ "objectID": "logbook/posts/2024_09_06/index.html#returning-to-appendix-6",
+ "href": "logbook/posts/2024_09_06/index.html#returning-to-appendix-6",
+ "title": "Day 7",
+ "section": "13.30-14.42: Returning to appendix 6",
+ "text": "13.30-14.42: Returning to appendix 6\nI created the four figures. For the fourth, the transformation I’d done to make it reflect 1000 didn’t seem to make sense for that one, but I wasn’t quite sure what the data was or how to transform it.\nI tried multiplifying by 1000, and that got results about half of the original. I tried multiplifying by 2000 - although this is just a total guess at this point.\nLooking across the results, I would say that I am satisified this matches the original for all except one bar: SABA. The SABA bars for patient years on treatment are much lower than in the original figure.\n\n\n\nAppendix 6 attempt\n\n\nSABA is one of the treatments: short-acting beta-agonists.\nI went to check my results, then realised there was an adjusted table of clinical results, and that I should try using that instead of S1, which is the more “raw” model results. Notably, that table just seems to have the columns used in the appendix plot - and they are adjusted to be per agent, so I can confidently just multiply by 1000 to get results for 1000 people.\n\n\n\n\n\n\nReflection\n\n\n\nIt’s great that this was provided, and makes it much easier. This is partly my fault for not noticing that I should be using that table. However, as for the other parts of this reproduction, this might have been supported by more guidance on which tables produced which figures, or inclusion of code to produce that figure in the paper. And, I wasn’t sure what to multiply for these either in the end (see below).\n\n\nI realised that multiplying by 1000 works with pAgent columns, but not for the pPYAll. It seems I’d need to multiply by about 16,000.\nI experimented with the others, but these were just guesses of what to multiply by, and I still found SABA results to be too low.\n\n\n\nAppendix 6 attempt 2"
+ },
+ {
+ "objectID": "logbook/posts/2024_09_06/index.html#troubleshooting-appendix-6",
+ "href": "logbook/posts/2024_09_06/index.html#troubleshooting-appendix-6",
+ "title": "Day 7",
+ "section": "15.10-15.40, 15.47-16.06: Troubleshooting appendix 6",
+ "text": "15.10-15.40, 15.47-16.06: Troubleshooting appendix 6\nLooking at their results from 50 million, they also have a low SABA\nComparing to other SABA results (with LAMA for context):\n\n\n\nScenario\nMy SABA (LAMA)\nTheir 50 million SABA (LAMA)\n\n\n\n\nS1NoCDAvg\n0.019 (0.136)\n0.019 (0.135)\n\n\nS1a\n0.026 (0.159)\n0.026 (0.159)\n\n\n\nGiven how similar these results are, I’m suspecting the issue might not be with the result, but with my transformation, as I just guessed at multiplying it all by 2000 to get similar-ish results for some bars. Looking at Case_Detecetion_Results.Rmd, SABAAll was created as follows:\nProppPatientYears = ifelse(Scenario==\"S2a\" | Scenario==\"S2NoCD\" | Scenario==\"S3NoCD\" | Scenario==\"S3a\" | Scenario==\"S3b\" |\n Scenario==\"S3c\" | Scenario==\"S3d\", PatientYears/sall[sall$Scenario==\"S1NoCDAvg\",\"PatientYears\"], 1),\n # SABA\nSABAExcluded = ifelse(Scenario==\"S1NoCDAvg\" | Scenario==\"S1a\" | Scenario==\"S1b\" | Scenario==\"S1c\", 0,\n ifelse(Scenario==\"S2a\" | Scenario==\"S2NoCD\", \n (sall[sall$Scenario==\"S1NoCDAvg\",\"SABATotal\"]- sall[sall$Scenario==\"S2NoCD\",\"SABATotal\"])/\n (sall[sall$Scenario==\"S1NoCDAvg\",\"PatientYears\"]- sall[sall$Scenario==\"S2NoCD\",\"PatientYears\"]),\n ifelse(Scenario==\"S3NoCD\" | Scenario==\"S3a\" | Scenario==\"S3b\" | Scenario==\"S3c\" | Scenario==\"S3d\", \n (sall[sall$Scenario==\"S1NoCDAvg\",\"SABATotal\"]- sall[sall$Scenario==\"S3NoCD\",\"SABATotal\"])/ (sall[sall$Scenario==\"S1NoCDAvg\",\"PatientYears\"]- sall[sall$Scenario==\"S3NoCD\",\"PatientYears\"]),999))),\n SABAAll = ProppPatientYears*SABA + (1-ProppPatientYears)*SABAExcluded,\nProppPatientYears adjusts the years for scenarios 2 and 3 by dividing them by the patient years of S1NoCDAvg. For scenario 1, it just sets this value to 1. This appears to be some adjustment of scenarios 2 and 3 so they compare against a single reference population. However, it remains unclear to me quite what the units are. SABAAll works out the same for scenarios 1 as SABA.\n\n\n\n\n\n\nReflection\n\n\n\nFound these transformations tricky to work out. I think part of the difficulty is that I’m really not quite sure what these columns are, and there isn’t documentation/comments to explain it, so I then don’t know how to transform them to match up with the paper.\n\n\nI then tried looking at the COPD DiagnosedpPY results, since those are the same thing just not broken down by treatment, and since clinical gives us adjusted versions of these but we also have “raw” versions from s1.\nclin_diag <- clinical %>%\n filter(Scenario %in% c(\"S1NoCDAvg\", \"S1a\")) %>%\n select(Scenario, DiagnosedpPYAll)\n\ns1_diag <- s1 %>%\n filter(Scenario %in% c(\"S1NoCDAvg\", \"S1a\")) %>%\n select(Scenario, Agents, DiagnosedPYs) %>%\n mutate(AdjDiagnosedPYs = DiagnosedPYs / (Agents/1000))\n\ndiag_investigate <- merge(clin_diag, s1_diag) %>%\n mutate(AdjDivideByPYAll = AdjDiagnosedPYs / DiagnosedpPYAll)\n\nwrite.csv(diag_investigate, \"../../logbook/posts/2024_09_06/diag_investigate.csv\")\n\npd.read_csv('diag_investigate.csv')\n\n\n\n\n\n\n\n\nUnnamed: 0\nScenario\nDiagnosedpPYAll\nAgents\nDiagnosedPYs\nAdjDiagnosedPYs\nAdjDivideByPYAll\n\n\n\n\n0\n1\nS1a\n0.462874\n743762\n659857.061426\n887.188457\n1916.696154\n\n\n1\n2\nS1NoCDAvg\n0.186304\n743240\n264401.460862\n355.741700\n1909.470132\n\n\n\n\n\n\n\nYou can convert between the two by multiplying by about 1910, but that doesn’t work for SABA.\nInstead, I tried looking back at export_figures() and the paper each again, but struggled to spot any answers there. I then tried looking at the first EpicR paper, “Development and Validation of the Evaluation Platform in COPD (EPIC): A Population-Based Outcomes Model of COPD for Canada”, but couldn’t find anything to explain it there either.\nI then looked to the main epicR repo, and found it has some documentation in the docs/ folder. I opened and read the html files in docs/articles/, but struggled to find answers here too."
+ },
+ {
+ "objectID": "logbook/posts/2024_09_06/index.html#timings",
+ "href": "logbook/posts/2024_09_06/index.html#timings",
+ "title": "Day 7",
+ "section": "Timings",
+ "text": "Timings\n\nimport sys\nsys.path.append('../')\nfrom timings import calculate_times\n\n# Minutes used prior to today\nused_to_date = 588\n\n# Times from today\ntimes = [\n ('09.36', '09.48'),\n ('09.49', '10.40'),\n ('11.00', '11.11'),\n ('11.12', '11.18'),\n ('11.24', '12.15'),\n ('13.15', '13.20'),\n ('13.25', '13.28'),\n ('13.30', '14.42'),\n ('15.10', '15.40'),\n ('15.47', '16.06')]\n\ncalculate_times(used_to_date, times)\n\nTime spent today: 260m, or 4h 20m\nTotal used to date: 848m, or 14h 8m\nTime remaining: 1552m, or 25h 52m\nUsed 35.3% of 40 hours max"
+ },
+ {
+ "objectID": "quarto_site/show_reproduction.html",
+ "href": "quarto_site/show_reproduction.html",
+ "title": "Main analysis",
"section": "",
- "text": "Please note: This is a template README and has not yet been completed\n\n\n\nTBC\n\n\n\nTBC\n\n\n\n\n\nTBC\n\n\n\nTBC\n\n\n\nTBC\n\n\n\n\nTBC\n\n\n\n\nTBC\n\n\n\nTBC"
+ "text": "14 October, 2024\n## \n## > errors<-c(\n## + ERR_INCORRECT_SETTING_VARIABLE=-1,\n## + ERR_INCORRECT_VECTOR_SIZE=-2,\n## + ERR_INCORRECT_INPUT_VAR=-3,\n## + ERR_EVENT_STACK_FULL=-4,\n## + .... [TRUNCATED] \n## \n## > record_mode<-c(\n## + record_mode_none=0,\n## + record_mode_agent=1,\n## + record_mode_event=2,\n## + record_mode_some_event=3\n## + )\n## \n## > agent_creation_mode<-c(\n## + agent_creation_mode_one=0,\n## + agent_creation_mode_all=1,\n## + agent_creation_mode_pre=2\n## + )\n## \n## > medication_classes<-c(\n## + MED_CLASS_SABA=1,\n## + MED_CLASS_LABA=2,\n## + MED_CLASS_LAMA=4,\n## + MED_CLASS_ICS=8,\n## + MED_CLASS_MACRO=16\n## + )\n## \n## > events<-c(\n## + event_start=0,\n## + event_fixed=1,\n## + event_birthday=2,\n## + event_smoking_change=3,\n## + event_COPD=4,\n## + event_exacerbat .... [TRUNCATED]\n# CHANGE: Add a timer (will see addition below to print time too)\n# Start timer\nstart.time <- Sys.time()\n# CHANGE: Add save paths (will see later, we also add lines to save to csv)\noutputs <- \"../outputs/\"\nfiles <- list(\n s1 = \"s1.csv\",\n s2 = \"s2.csv\",\n s3 = \"s3.csv\",\n sall = \"sall.csv\",\n ceplane = \"ceplane.csv\",\n clinical = \"clinicalresults.csv\"\n)\n\n# Apply to each element in list to create path to file in outputs\npaths <- lapply(files, function(filename) paste0(outputs, filename))\n## Initializing the session\n\n## [1] 0\nGlobal inputs:\n\nMedication adherence is 0.7\nSmoking adherence is 0.7\nCost discounting: 0.015\nQALY discounting: 0.015\nTime horizon: 20\nThe WTP threshold for NMB is 50000\n\nCase detection inputs:\n\nCase detection occurs at 3 year intervals.\nAn outpatient diagnosis costs 61.18\nThe utility gain due to symptom relief from treatment is 0.0367\n\n\n\nAll patients are eligible. The cost of case detection is:\n\n\n\nNone\nCDQ17\nFlowMeter\nFlowMeter_CDQ\n\n\n\n\n0\n11.56\n30.46\n42.01\n\n\n\n\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nAgents\nPatientYears\nCopdPYs\nNCaseDetections\nDiagnosedPYs\nOverdiagnosedPYs\nSABA\nLAMA\nLAMALABA\nICSLAMALABA\nMild\nModerate\nSevere\nVerySevere\nMildPY\nModeratePY\nSeverePY\nVerySeverePY\nNoCOPD\nGOLD1\nGOLD2\nGOLD3\nGOLD4\nCost\nCostpAgent\nQALY\nQALYpAgent\nNMB\nIncrementalCosts\nIncrementalQALY\nICER\nIncrementalNMB\n\n\n\n\nS1NoCD\n74388747\n1251903576\n142277086\n381929878\n26336153\n26747209\n0.017\n0.135\n0.151\n0.080\n30809289\n5766720\n9701609\n839538.0\n0.217\n0.041\n0.068\n0.006\n1054811684\n57697202\n61465410\n13763906\n2389153\n159789813678\n2148.037\n933216569\n12.545\n625108.5\n0.000\n0.000\nNaN\n0.000\n\n\nS1NoCD2\n74388425\n1251968458\n142302175\n249329256\n26636736\n32661650\n0.021\n0.136\n0.151\n0.080\n30801067\n5759155\n9700407\n837363.0\n0.216\n0.040\n0.068\n0.006\n1054852003\n57672835\n61488796\n13791895\n2388663\n160061317736\n2151.697\n933263557\n12.546\n625139.1\n0.000\n0.000\nNaN\n0.000\n\n\nS1NoCDAvg\n74388586\n1251936017\n142289630\n315629567\n26486444\n29704430\n0.019\n0.136\n0.151\n0.080\n30805178\n5762938\n9701008\n838450.5\n0.216\n0.041\n0.068\n0.006\n1054831844\n57685018\n61477103\n13777900\n2388908\n159925565707\n2149.867\n933240063\n12.545\n625123.8\n0.000\n0.000\nNaN\n0.000\n\n\nS1a\n74380275\n1252107231\n142568770\n372100831\n66270673\n26745802\n0.026\n0.159\n0.310\n0.094\n29815042\n5625494\n9500837\n824680.0\n0.209\n0.039\n0.067\n0.006\n1054730269\n57758498\n61574433\n13835312\n2423951\n181401075128\n2438.833\n934239602\n12.560\n625576.9\n290.796\n0.015\n19152.73\n468.354\n\n\nS1b\n74383024\n1252030978\n142410670\n376764647\n47607118\n26748888\n0.022\n0.148\n0.241\n0.088\n30192660\n5683507\n9596337\n830158.0\n0.212\n0.040\n0.067\n0.006\n1054810413\n57710159\n61499603\n13827904\n2405886\n175747140584\n2362.732\n933787724\n12.554\n625326.0\n214.694\n0.009\n24838.03\n217.495\n\n\nS1c\n74393892\n1252166867\n142379318\n378416895\n41063274\n26758496\n0.020\n0.144\n0.214\n0.085\n30350312\n5702044\n9621386\n828456.0\n0.213\n0.040\n0.068\n0.006\n1054965444\n57726777\n61483651\n13803995\n2400849\n177409072797\n2384.726\n933746980\n12.551\n625184.9\n236.689\n0.006\n37796.71\n76.419\n\n\n\nTreatment rate: SABA is expressed per all patient-years, LAMA, LAMA/LABA, ICS/LAMA/LABA are per COPD patient-years Exacerbations: Total exacerbations and rate per COPD patient-year: GOLD Stage: Cumulative patient-years Cost/QALY: Total cost and QALYs NMB: Net Monetary Benefit is calculated as QALY per patient-year * Lamba - Cost per patient-year\n\n\n\n\n\nPatients with symptoms at year 1 are eligible. The cost of case detection is:\n## Initializing the session\n\n## [1] 0\n\n\n\nNone\nFlowMeter\n\n\n\n\n0\n24.33\n\n\n\n\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nAgents\nPatientYears\nCopdPYs\nNCaseDetections\nDiagnosedPYs\nOverdiagnosedPYs\nSABA\nLAMA\nLAMALABA\nICSLAMALABA\nMild\nModerate\nSevere\nVerySevere\nMildPY\nModeratePY\nSeverePY\nVerySeverePY\nNoCOPD\nGOLD1\nGOLD2\nGOLD3\nGOLD4\nCost\nCostpAgent\nQALY\nQALYpAgent\nNMB\nIncrementalCosts\nIncrementalQALY\nICER\nIncrementalNMB\n\n\n\n\nS2NoCD\n44156404\n729084497\n100330760\n221928932\n19330952\n15481473\n0.017\n0.142\n0.159\n0.087\n22917076\n4253860\n7098458\n609501\n0.228\n0.042\n0.071\n0.006\n597619551\n37246010\n44990133\n11123560\n2050468\n117605933755\n2663.395\n541859031\n12.271\n610904.5\n0.000\n0.000\nNaN\n0.000\n\n\nS2a\n44164773\n729329891\n100492049\n217625447\n37010683\n15485858\n0.023\n0.157\n0.269\n0.097\n22383331\n4181008\n7006346\n603148\n0.223\n0.042\n0.070\n0.006\n597700094\n37284225\n45068617\n11147317\n2064302\n127735308298\n2892.244\n542449119\n12.282\n611227.4\n228.849\n0.011\n20737.2\n322.935\n\n\n\nTreatment rate: SABA is expressed per all patient-years, LAMA, LAMA/LABA, ICS/LAMA/LABA are per COPD patient-years Exacerbations: Total exacerbations and rate per COPD patient-year: GOLD Stage: Cumulative patient-years Cost/QALY: Total cost and QALYs NMB: Net Monetary Benefit is calculated as QALY per patient-year * Lamba - Cost per patient-year\n\n\n\n\n\nEver smokers ≥50 years of age are eligible. The cost of case detection is:\n## Initializing the session\n\n## [1] 0\n\n\n\nNone\nCDQ195\nCDQ165\nFlowMeter\nFlowMeter_CDQ\n\n\n\n\n0\n11.56\n11.56\n24.33\n42.01\n\n\n\n\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nAgents\nPatientYears\nCopdPYs\nNCaseDetections\nDiagnosedPYs\nOverdiagnosedPYs\nSABA\nLAMA\nLAMALABA\nICSLAMALABA\nMild\nModerate\nSevere\nVerySevere\nMildPY\nModeratePY\nSeverePY\nVerySeverePY\nNoCOPD\nGOLD1\nGOLD2\nGOLD3\nGOLD4\nCost\nCostpAgent\nQALY\nQALYpAgent\nNMB\nIncrementalCosts\nIncrementalQALY\nICER\nIncrementalNMB\n\n\n\n\nS3NoCD\n34500334\n520264617\n83469550\n160191877\n15617539\n11630123\n0.018\n0.137\n0.154\n0.086\n19525292\n3631989\n6073558\n522424\n0.234\n0.044\n0.073\n0.006\n415002016\n30608834\n37422882\n9624470\n1788687\n99842281258\n2893.951\n387517176\n11.232\n558719.7\n0.000\n0.000\nNaN\n0.000\n\n\nS3a\n34507415\n520428951\n83482576\n158676520\n22030098\n11639538\n0.021\n0.144\n0.204\n0.091\n19287934\n3609226\n6045313\n521405\n0.231\n0.043\n0.072\n0.006\n415152628\n30610103\n37407716\n9641236\n1794702\n106095806862\n3074.580\n387796823\n11.238\n558829.0\n180.629\n0.006\n31147.82\n109.325\n\n\nS3b\n34499539\n520396932\n83618002\n155590263\n34428826\n11625591\n0.027\n0.156\n0.293\n0.099\n18988453\n3564625\n5980823\n516662\n0.227\n0.043\n0.072\n0.006\n414988556\n30607762\n37472866\n9688489\n1813970\n110363699543\n3198.991\n388038736\n11.248\n559183.5\n305.039\n0.015\n19837.73\n463.797\n\n\nS3c\n34496334\n520227370\n83568351\n157099275\n28171361\n11622564\n0.024\n0.150\n0.249\n0.095\n19145218\n3585036\n6010150\n518449\n0.229\n0.043\n0.072\n0.006\n414875686\n30589301\n37479876\n9661353\n1804425\n107627143927\n3119.959\n387776879\n11.241\n558935.2\n226.008\n0.009\n25592.92\n215.536\n\n\nS3d\n34507664\n520495018\n83555471\n157877292\n25364366\n11630839\n0.023\n0.147\n0.229\n0.093\n19209915\n3595849\n6028084\n520340\n0.230\n0.043\n0.072\n0.006\n415141325\n30594930\n37472224\n9653084\n1805226\n108255487965\n3137.143\n387911628\n11.241\n558928.8\n243.192\n0.009\n26887.15\n209.054\n\n\n\nTreatment rate: SABA is expressed per all patient-years, LAMA, LAMA/LABA, ICS/LAMA/LABA are per COPD patient-years Exacerbations: Total exacerbations and rate per COPD patient-year GOLD Stage: Cumulative patient-years Cost/QALY: Total cost and QALYs NMB: Net Monetary Benefit is calculated as QALY per patient-year * Lamba - Cost per patient-year\n\n\n\n\n\nOrdered by descending Net Monetary Benefit\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nAgents\nCost\nCostpAgent\nQALY\nQALYpAgent\nICER\nIncrementalNMB\n\n\n\n\nS1a\n74380275\n181401075128\n2438.833\n934239602\n12.560\n19152.73\n468.354\n\n\nS3b\n34499539\n110363699543\n3198.991\n388038736\n11.248\n19837.73\n463.797\n\n\nS2a\n44164773\n127735308298\n2892.244\n542449119\n12.282\n20737.20\n322.935\n\n\nS1b\n74383024\n175747140584\n2362.732\n933787724\n12.554\n24838.03\n217.495\n\n\nS3c\n34496334\n107627143927\n3119.959\n387776879\n11.241\n25592.92\n215.536\n\n\nS3d\n34507664\n108255487965\n3137.143\n387911628\n11.241\n26887.15\n209.054\n\n\nS3a\n34507415\n106095806862\n3074.580\n387796823\n11.238\n31147.82\n109.325\n\n\nS1c\n74393892\n177409072797\n2384.726\n933746980\n12.551\n37796.71\n76.419\n\n\nS1NoCD\n74388747\n159789813678\n2148.037\n933216569\n12.545\nNaN\n0.000\n\n\nS2NoCD\n44156404\n117605933755\n2663.395\n541859031\n12.271\nNaN\n0.000\n\n\nS3NoCD\n34500334\n99842281258\n2893.951\n387517176\n11.232\nNaN\n0.000\n\n\n\n\n\n\n\nAdjusted to the total population\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nAgents\nPropAgents\nCost\nCostpAgent\nCostpAgentExcluded\nCostpAgentAll\nQALY\nQALYpAgent\nQALYpAgentExcluded\nQALYpAgentAll\nIncrementalCosts\nIncrementalQALY\nICERAdj\nICER\nINMB\n\n\n\n\nS1NoCDAvg\n74388586\n1.0000000\n159925565707\n2149.867\n0.000\n2149.867\n933240063\n12.54547\n0.00000\n12.54547\n0.00000\n0.0000000\nNaN\nNaN\n0.00000\n\n\nS1a\n74380275\n1.0000000\n181401075128\n2438.833\n0.000\n2438.833\n934239602\n12.56031\n0.00000\n12.56031\n288.96609\n0.0148400\n19472.10\n19152.73\n453.03421\n\n\nS1b\n74383024\n1.0000000\n175747140584\n2362.732\n0.000\n2362.732\n933787724\n12.55377\n0.00000\n12.55377\n212.86487\n0.0083008\n25643.89\n24838.02\n202.17518\n\n\nS1c\n74393892\n1.0000000\n177409072797\n2384.726\n0.000\n2384.726\n933746980\n12.55139\n0.00000\n12.55139\n234.85933\n0.0059192\n39677.70\n37796.70\n61.09955\n\n\nS2NoCD\n44156404\n0.5935911\n117605933755\n2663.395\n1399.821\n2149.867\n541859031\n12.27136\n12.94584\n12.54547\n0.00000\n0.0000000\nNaN\nNaN\n0.00000\n\n\nS2a\n44164773\n0.5937036\n127735308298\n2892.244\n1399.821\n2285.878\n542449119\n12.28239\n12.94584\n12.55195\n136.01091\n0.0064761\n21002.14\n20737.20\n187.79161\n\n\nS3NoCD\n34500334\n0.4637853\n99842281258\n2893.951\n1506.290\n2149.867\n387517176\n11.23227\n13.68129\n12.54547\n0.00000\n0.0000000\nNaN\nNaN\n0.00000\n\n\nS3a\n34507415\n0.4638805\n106095806862\n3074.580\n1506.290\n2233.789\n387796823\n11.23807\n13.68129\n12.54793\n83.92228\n0.0024570\n34156.94\n31147.82\n38.92579\n\n\nS3b\n34499539\n0.4637746\n110363699543\n3198.991\n1506.290\n2291.322\n388038736\n11.24765\n13.68129\n12.55263\n141.45471\n0.0071575\n19763.12\n19837.73\n216.42072\n\n\nS3c\n34496334\n0.4637315\n107627143927\n3119.959\n1506.290\n2254.599\n387776879\n11.24110\n13.68129\n12.54970\n104.73230\n0.0042268\n24777.92\n25592.93\n106.60970\n\n\nS3d\n34507664\n0.4638839\n108255487965\n3137.143\n1506.290\n2262.817\n387911628\n11.24132\n13.68129\n12.54943\n112.94966\n0.0039545\n28562.49\n26887.15\n84.77408\n\n\n\n\n\n\n\nAdjusted to the total population\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nPropAgents\nProppPatientYears\nProppCopdPYs\nSABAAll\nLAMAAll\nLAMALABAAll\nICSLAMALABAAll\nMildpAgentAll\nModeratepAgentAll\nSeverepAgentAll\nVerySeverepAgentAll\nNoCOPDpPYAll\nGOLD1pPYAll\nGOLD2pPYAll\nGOLD3pPYAll\nGOLD4pPYAll\nDiagnosedpPYAll\n\n\n\n\nS1NoCDAvg\n1.0000000\n1.0000000\n1.0000000\n0.0190054\n0.1355252\n0.1513708\n0.0796683\n0.4141116\n0.0774707\n0.1304099\n0.0112712\n0.8425605\n0.0460767\n0.0491056\n0.0110053\n0.0019082\n0.1861446\n\n\nS1a\n1.0000000\n1.0000000\n1.0000000\n0.0264033\n0.1593018\n0.3099946\n0.0940914\n0.4008461\n0.0756315\n0.1277333\n0.0110873\n0.8423642\n0.0461290\n0.0491766\n0.0110496\n0.0019359\n0.4648330\n\n\nS1b\n1.0000000\n1.0000000\n1.0000000\n0.0216398\n0.1482342\n0.2406521\n0.0877947\n0.4059079\n0.0764087\n0.1290125\n0.0111606\n0.8424795\n0.0460932\n0.0491199\n0.0110444\n0.0019216\n0.3342946\n\n\nS1c\n1.0000000\n1.0000000\n1.0000000\n0.0201517\n0.1441110\n0.2141912\n0.0851720\n0.4079678\n0.0766467\n0.1293303\n0.0111361\n0.8425119\n0.0461015\n0.0491018\n0.0110241\n0.0019174\n0.2884076\n\n\nS2NoCD\n0.5935911\n0.5823656\n0.7051165\n0.0190054\n0.1355252\n0.1513708\n0.0796683\n0.4141116\n0.0774707\n0.1304099\n0.0112712\n0.8425605\n0.0460767\n0.0491056\n0.0110053\n0.0019082\n0.1861446\n\n\nS2a\n0.5937036\n0.5825616\n0.7062500\n0.0223572\n0.1465203\n0.2287835\n0.0867962\n0.4069072\n0.0764858\n0.1291620\n0.0111850\n0.8424534\n0.0460995\n0.0491621\n0.0110233\n0.0019191\n0.3102030\n\n\nS3NoCD\n0.4637853\n0.4155681\n0.5866172\n0.0190054\n0.1355252\n0.1513708\n0.0796683\n0.4141116\n0.0774707\n0.1304099\n0.0112712\n0.8425605\n0.0460767\n0.0491056\n0.0110053\n0.0019082\n0.1861446\n\n\nS3a\n0.4638805\n0.4156993\n0.5867088\n0.0201156\n0.1395918\n0.1808230\n0.0824771\n0.4108939\n0.0771596\n0.1300215\n0.0112568\n0.8425660\n0.0460728\n0.0490892\n0.0110179\n0.0019129\n0.2311946\n\n\nS3b\n0.4637746\n0.4156737\n0.5876605\n0.0227032\n0.1471017\n0.2332507\n0.0873282\n0.4068980\n0.0765657\n0.1291642\n0.0111939\n0.8424573\n0.0460719\n0.0491421\n0.0110558\n0.0019283\n0.3181560\n\n\nS3c\n0.4637315\n0.4155383\n0.5873116\n0.0213262\n0.1432680\n0.2074859\n0.0848645\n0.4090175\n0.0768424\n0.1295624\n0.0112182\n0.8424856\n0.0460621\n0.0491521\n0.0110349\n0.0019208\n0.2742435\n\n\nS3d\n0.4638839\n0.4157521\n0.5872211\n0.0207546\n0.1415673\n0.1954816\n0.0837051\n0.4098442\n0.0769796\n0.1297896\n0.0112424\n0.8425108\n0.0460587\n0.0491390\n0.0110271\n0.0019212\n0.2545329\n\n\n\n\n\n\nRun time for this notebook:\nend.time <- Sys.time()\ntime.taken <- end.time - start.time\ntime.taken\n## Time difference of 20.64576 hours"
},
{
- "objectID": "quarto_site/reproduction_readme.html#model-summary",
- "href": "quarto_site/reproduction_readme.html#model-summary",
- "title": "README for reproduction",
+ "objectID": "quarto_site/show_reproduction.html#s1-all-patients-scenario",
+ "href": "quarto_site/show_reproduction.html#s1-all-patients-scenario",
+ "title": "Main analysis",
"section": "",
- "text": "TBC"
+ "text": "All patients are eligible. The cost of case detection is:\n\n\n\nNone\nCDQ17\nFlowMeter\nFlowMeter_CDQ\n\n\n\n\n0\n11.56\n30.46\n42.01\n\n\n\n\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nAgents\nPatientYears\nCopdPYs\nNCaseDetections\nDiagnosedPYs\nOverdiagnosedPYs\nSABA\nLAMA\nLAMALABA\nICSLAMALABA\nMild\nModerate\nSevere\nVerySevere\nMildPY\nModeratePY\nSeverePY\nVerySeverePY\nNoCOPD\nGOLD1\nGOLD2\nGOLD3\nGOLD4\nCost\nCostpAgent\nQALY\nQALYpAgent\nNMB\nIncrementalCosts\nIncrementalQALY\nICER\nIncrementalNMB\n\n\n\n\nS1NoCD\n74388747\n1251903576\n142277086\n381929878\n26336153\n26747209\n0.017\n0.135\n0.151\n0.080\n30809289\n5766720\n9701609\n839538.0\n0.217\n0.041\n0.068\n0.006\n1054811684\n57697202\n61465410\n13763906\n2389153\n159789813678\n2148.037\n933216569\n12.545\n625108.5\n0.000\n0.000\nNaN\n0.000\n\n\nS1NoCD2\n74388425\n1251968458\n142302175\n249329256\n26636736\n32661650\n0.021\n0.136\n0.151\n0.080\n30801067\n5759155\n9700407\n837363.0\n0.216\n0.040\n0.068\n0.006\n1054852003\n57672835\n61488796\n13791895\n2388663\n160061317736\n2151.697\n933263557\n12.546\n625139.1\n0.000\n0.000\nNaN\n0.000\n\n\nS1NoCDAvg\n74388586\n1251936017\n142289630\n315629567\n26486444\n29704430\n0.019\n0.136\n0.151\n0.080\n30805178\n5762938\n9701008\n838450.5\n0.216\n0.041\n0.068\n0.006\n1054831844\n57685018\n61477103\n13777900\n2388908\n159925565707\n2149.867\n933240063\n12.545\n625123.8\n0.000\n0.000\nNaN\n0.000\n\n\nS1a\n74380275\n1252107231\n142568770\n372100831\n66270673\n26745802\n0.026\n0.159\n0.310\n0.094\n29815042\n5625494\n9500837\n824680.0\n0.209\n0.039\n0.067\n0.006\n1054730269\n57758498\n61574433\n13835312\n2423951\n181401075128\n2438.833\n934239602\n12.560\n625576.9\n290.796\n0.015\n19152.73\n468.354\n\n\nS1b\n74383024\n1252030978\n142410670\n376764647\n47607118\n26748888\n0.022\n0.148\n0.241\n0.088\n30192660\n5683507\n9596337\n830158.0\n0.212\n0.040\n0.067\n0.006\n1054810413\n57710159\n61499603\n13827904\n2405886\n175747140584\n2362.732\n933787724\n12.554\n625326.0\n214.694\n0.009\n24838.03\n217.495\n\n\nS1c\n74393892\n1252166867\n142379318\n378416895\n41063274\n26758496\n0.020\n0.144\n0.214\n0.085\n30350312\n5702044\n9621386\n828456.0\n0.213\n0.040\n0.068\n0.006\n1054965444\n57726777\n61483651\n13803995\n2400849\n177409072797\n2384.726\n933746980\n12.551\n625184.9\n236.689\n0.006\n37796.71\n76.419\n\n\n\nTreatment rate: SABA is expressed per all patient-years, LAMA, LAMA/LABA, ICS/LAMA/LABA are per COPD patient-years Exacerbations: Total exacerbations and rate per COPD patient-year: GOLD Stage: Cumulative patient-years Cost/QALY: Total cost and QALYs NMB: Net Monetary Benefit is calculated as QALY per patient-year * Lamba - Cost per patient-year"
},
{
- "objectID": "quarto_site/reproduction_readme.html#scope-of-the-reproduction",
- "href": "quarto_site/reproduction_readme.html#scope-of-the-reproduction",
- "title": "README for reproduction",
+ "objectID": "quarto_site/show_reproduction.html#s2-symptomatic-patients-scenario",
+ "href": "quarto_site/show_reproduction.html#s2-symptomatic-patients-scenario",
+ "title": "Main analysis",
"section": "",
- "text": "TBC"
+ "text": "Patients with symptoms at year 1 are eligible. The cost of case detection is:\n## Initializing the session\n\n## [1] 0\n\n\n\nNone\nFlowMeter\n\n\n\n\n0\n24.33\n\n\n\n\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nAgents\nPatientYears\nCopdPYs\nNCaseDetections\nDiagnosedPYs\nOverdiagnosedPYs\nSABA\nLAMA\nLAMALABA\nICSLAMALABA\nMild\nModerate\nSevere\nVerySevere\nMildPY\nModeratePY\nSeverePY\nVerySeverePY\nNoCOPD\nGOLD1\nGOLD2\nGOLD3\nGOLD4\nCost\nCostpAgent\nQALY\nQALYpAgent\nNMB\nIncrementalCosts\nIncrementalQALY\nICER\nIncrementalNMB\n\n\n\n\nS2NoCD\n44156404\n729084497\n100330760\n221928932\n19330952\n15481473\n0.017\n0.142\n0.159\n0.087\n22917076\n4253860\n7098458\n609501\n0.228\n0.042\n0.071\n0.006\n597619551\n37246010\n44990133\n11123560\n2050468\n117605933755\n2663.395\n541859031\n12.271\n610904.5\n0.000\n0.000\nNaN\n0.000\n\n\nS2a\n44164773\n729329891\n100492049\n217625447\n37010683\n15485858\n0.023\n0.157\n0.269\n0.097\n22383331\n4181008\n7006346\n603148\n0.223\n0.042\n0.070\n0.006\n597700094\n37284225\n45068617\n11147317\n2064302\n127735308298\n2892.244\n542449119\n12.282\n611227.4\n228.849\n0.011\n20737.2\n322.935\n\n\n\nTreatment rate: SABA is expressed per all patient-years, LAMA, LAMA/LABA, ICS/LAMA/LABA are per COPD patient-years Exacerbations: Total exacerbations and rate per COPD patient-year: GOLD Stage: Cumulative patient-years Cost/QALY: Total cost and QALYs NMB: Net Monetary Benefit is calculated as QALY per patient-year * Lamba - Cost per patient-year"
},
{
- "objectID": "quarto_site/reproduction_readme.html#reproducing-these-results",
- "href": "quarto_site/reproduction_readme.html#reproducing-these-results",
- "title": "README for reproduction",
+ "objectID": "quarto_site/show_reproduction.html#s3-smoking-history-scenario",
+ "href": "quarto_site/show_reproduction.html#s3-smoking-history-scenario",
+ "title": "Main analysis",
"section": "",
- "text": "TBC\n\n\n\nTBC\n\n\n\nTBC"
+ "text": "Ever smokers ≥50 years of age are eligible. The cost of case detection is:\n## Initializing the session\n\n## [1] 0\n\n\n\nNone\nCDQ195\nCDQ165\nFlowMeter\nFlowMeter_CDQ\n\n\n\n\n0\n11.56\n11.56\n24.33\n42.01\n\n\n\n\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nAgents\nPatientYears\nCopdPYs\nNCaseDetections\nDiagnosedPYs\nOverdiagnosedPYs\nSABA\nLAMA\nLAMALABA\nICSLAMALABA\nMild\nModerate\nSevere\nVerySevere\nMildPY\nModeratePY\nSeverePY\nVerySeverePY\nNoCOPD\nGOLD1\nGOLD2\nGOLD3\nGOLD4\nCost\nCostpAgent\nQALY\nQALYpAgent\nNMB\nIncrementalCosts\nIncrementalQALY\nICER\nIncrementalNMB\n\n\n\n\nS3NoCD\n34500334\n520264617\n83469550\n160191877\n15617539\n11630123\n0.018\n0.137\n0.154\n0.086\n19525292\n3631989\n6073558\n522424\n0.234\n0.044\n0.073\n0.006\n415002016\n30608834\n37422882\n9624470\n1788687\n99842281258\n2893.951\n387517176\n11.232\n558719.7\n0.000\n0.000\nNaN\n0.000\n\n\nS3a\n34507415\n520428951\n83482576\n158676520\n22030098\n11639538\n0.021\n0.144\n0.204\n0.091\n19287934\n3609226\n6045313\n521405\n0.231\n0.043\n0.072\n0.006\n415152628\n30610103\n37407716\n9641236\n1794702\n106095806862\n3074.580\n387796823\n11.238\n558829.0\n180.629\n0.006\n31147.82\n109.325\n\n\nS3b\n34499539\n520396932\n83618002\n155590263\n34428826\n11625591\n0.027\n0.156\n0.293\n0.099\n18988453\n3564625\n5980823\n516662\n0.227\n0.043\n0.072\n0.006\n414988556\n30607762\n37472866\n9688489\n1813970\n110363699543\n3198.991\n388038736\n11.248\n559183.5\n305.039\n0.015\n19837.73\n463.797\n\n\nS3c\n34496334\n520227370\n83568351\n157099275\n28171361\n11622564\n0.024\n0.150\n0.249\n0.095\n19145218\n3585036\n6010150\n518449\n0.229\n0.043\n0.072\n0.006\n414875686\n30589301\n37479876\n9661353\n1804425\n107627143927\n3119.959\n387776879\n11.241\n558935.2\n226.008\n0.009\n25592.92\n215.536\n\n\nS3d\n34507664\n520495018\n83555471\n157877292\n25364366\n11630839\n0.023\n0.147\n0.229\n0.093\n19209915\n3595849\n6028084\n520340\n0.230\n0.043\n0.072\n0.006\n415141325\n30594930\n37472224\n9653084\n1805226\n108255487965\n3137.143\n387911628\n11.241\n558928.8\n243.192\n0.009\n26887.15\n209.054\n\n\n\nTreatment rate: SABA is expressed per all patient-years, LAMA, LAMA/LABA, ICS/LAMA/LABA are per COPD patient-years Exacerbations: Total exacerbations and rate per COPD patient-year GOLD Stage: Cumulative patient-years Cost/QALY: Total cost and QALYs NMB: Net Monetary Benefit is calculated as QALY per patient-year * Lamba - Cost per patient-year"
},
{
- "objectID": "quarto_site/reproduction_readme.html#reproduction-specs-and-runtime",
- "href": "quarto_site/reproduction_readme.html#reproduction-specs-and-runtime",
- "title": "README for reproduction",
+ "objectID": "quarto_site/show_reproduction.html#all-scenarios",
+ "href": "quarto_site/show_reproduction.html#all-scenarios",
+ "title": "Main analysis",
"section": "",
- "text": "TBC"
+ "text": "Ordered by descending Net Monetary Benefit\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nAgents\nCost\nCostpAgent\nQALY\nQALYpAgent\nICER\nIncrementalNMB\n\n\n\n\nS1a\n74380275\n181401075128\n2438.833\n934239602\n12.560\n19152.73\n468.354\n\n\nS3b\n34499539\n110363699543\n3198.991\n388038736\n11.248\n19837.73\n463.797\n\n\nS2a\n44164773\n127735308298\n2892.244\n542449119\n12.282\n20737.20\n322.935\n\n\nS1b\n74383024\n175747140584\n2362.732\n933787724\n12.554\n24838.03\n217.495\n\n\nS3c\n34496334\n107627143927\n3119.959\n387776879\n11.241\n25592.92\n215.536\n\n\nS3d\n34507664\n108255487965\n3137.143\n387911628\n11.241\n26887.15\n209.054\n\n\nS3a\n34507415\n106095806862\n3074.580\n387796823\n11.238\n31147.82\n109.325\n\n\nS1c\n74393892\n177409072797\n2384.726\n933746980\n12.551\n37796.71\n76.419\n\n\nS1NoCD\n74388747\n159789813678\n2148.037\n933216569\n12.545\nNaN\n0.000\n\n\nS2NoCD\n44156404\n117605933755\n2663.395\n541859031\n12.271\nNaN\n0.000\n\n\nS3NoCD\n34500334\n99842281258\n2893.951\n387517176\n11.232\nNaN\n0.000"
},
{
- "objectID": "quarto_site/reproduction_readme.html#citation",
- "href": "quarto_site/reproduction_readme.html#citation",
- "title": "README for reproduction",
+ "objectID": "quarto_site/show_reproduction.html#cost-effectiveness-plane",
+ "href": "quarto_site/show_reproduction.html#cost-effectiveness-plane",
+ "title": "Main analysis",
"section": "",
- "text": "TBC"
+ "text": "Adjusted to the total population\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nAgents\nPropAgents\nCost\nCostpAgent\nCostpAgentExcluded\nCostpAgentAll\nQALY\nQALYpAgent\nQALYpAgentExcluded\nQALYpAgentAll\nIncrementalCosts\nIncrementalQALY\nICERAdj\nICER\nINMB\n\n\n\n\nS1NoCDAvg\n74388586\n1.0000000\n159925565707\n2149.867\n0.000\n2149.867\n933240063\n12.54547\n0.00000\n12.54547\n0.00000\n0.0000000\nNaN\nNaN\n0.00000\n\n\nS1a\n74380275\n1.0000000\n181401075128\n2438.833\n0.000\n2438.833\n934239602\n12.56031\n0.00000\n12.56031\n288.96609\n0.0148400\n19472.10\n19152.73\n453.03421\n\n\nS1b\n74383024\n1.0000000\n175747140584\n2362.732\n0.000\n2362.732\n933787724\n12.55377\n0.00000\n12.55377\n212.86487\n0.0083008\n25643.89\n24838.02\n202.17518\n\n\nS1c\n74393892\n1.0000000\n177409072797\n2384.726\n0.000\n2384.726\n933746980\n12.55139\n0.00000\n12.55139\n234.85933\n0.0059192\n39677.70\n37796.70\n61.09955\n\n\nS2NoCD\n44156404\n0.5935911\n117605933755\n2663.395\n1399.821\n2149.867\n541859031\n12.27136\n12.94584\n12.54547\n0.00000\n0.0000000\nNaN\nNaN\n0.00000\n\n\nS2a\n44164773\n0.5937036\n127735308298\n2892.244\n1399.821\n2285.878\n542449119\n12.28239\n12.94584\n12.55195\n136.01091\n0.0064761\n21002.14\n20737.20\n187.79161\n\n\nS3NoCD\n34500334\n0.4637853\n99842281258\n2893.951\n1506.290\n2149.867\n387517176\n11.23227\n13.68129\n12.54547\n0.00000\n0.0000000\nNaN\nNaN\n0.00000\n\n\nS3a\n34507415\n0.4638805\n106095806862\n3074.580\n1506.290\n2233.789\n387796823\n11.23807\n13.68129\n12.54793\n83.92228\n0.0024570\n34156.94\n31147.82\n38.92579\n\n\nS3b\n34499539\n0.4637746\n110363699543\n3198.991\n1506.290\n2291.322\n388038736\n11.24765\n13.68129\n12.55263\n141.45471\n0.0071575\n19763.12\n19837.73\n216.42072\n\n\nS3c\n34496334\n0.4637315\n107627143927\n3119.959\n1506.290\n2254.599\n387776879\n11.24110\n13.68129\n12.54970\n104.73230\n0.0042268\n24777.92\n25592.93\n106.60970\n\n\nS3d\n34507664\n0.4638839\n108255487965\n3137.143\n1506.290\n2262.817\n387911628\n11.24132\n13.68129\n12.54943\n112.94966\n0.0039545\n28562.49\n26887.15\n84.77408"
},
{
- "objectID": "quarto_site/reproduction_readme.html#license",
- "href": "quarto_site/reproduction_readme.html#license",
- "title": "README for reproduction",
+ "objectID": "quarto_site/show_reproduction.html#clinical-results-for-all-scenarios",
+ "href": "quarto_site/show_reproduction.html#clinical-results-for-all-scenarios",
+ "title": "Main analysis",
"section": "",
- "text": "TBC"
+ "text": "Adjusted to the total population\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nPropAgents\nProppPatientYears\nProppCopdPYs\nSABAAll\nLAMAAll\nLAMALABAAll\nICSLAMALABAAll\nMildpAgentAll\nModeratepAgentAll\nSeverepAgentAll\nVerySeverepAgentAll\nNoCOPDpPYAll\nGOLD1pPYAll\nGOLD2pPYAll\nGOLD3pPYAll\nGOLD4pPYAll\nDiagnosedpPYAll\n\n\n\n\nS1NoCDAvg\n1.0000000\n1.0000000\n1.0000000\n0.0190054\n0.1355252\n0.1513708\n0.0796683\n0.4141116\n0.0774707\n0.1304099\n0.0112712\n0.8425605\n0.0460767\n0.0491056\n0.0110053\n0.0019082\n0.1861446\n\n\nS1a\n1.0000000\n1.0000000\n1.0000000\n0.0264033\n0.1593018\n0.3099946\n0.0940914\n0.4008461\n0.0756315\n0.1277333\n0.0110873\n0.8423642\n0.0461290\n0.0491766\n0.0110496\n0.0019359\n0.4648330\n\n\nS1b\n1.0000000\n1.0000000\n1.0000000\n0.0216398\n0.1482342\n0.2406521\n0.0877947\n0.4059079\n0.0764087\n0.1290125\n0.0111606\n0.8424795\n0.0460932\n0.0491199\n0.0110444\n0.0019216\n0.3342946\n\n\nS1c\n1.0000000\n1.0000000\n1.0000000\n0.0201517\n0.1441110\n0.2141912\n0.0851720\n0.4079678\n0.0766467\n0.1293303\n0.0111361\n0.8425119\n0.0461015\n0.0491018\n0.0110241\n0.0019174\n0.2884076\n\n\nS2NoCD\n0.5935911\n0.5823656\n0.7051165\n0.0190054\n0.1355252\n0.1513708\n0.0796683\n0.4141116\n0.0774707\n0.1304099\n0.0112712\n0.8425605\n0.0460767\n0.0491056\n0.0110053\n0.0019082\n0.1861446\n\n\nS2a\n0.5937036\n0.5825616\n0.7062500\n0.0223572\n0.1465203\n0.2287835\n0.0867962\n0.4069072\n0.0764858\n0.1291620\n0.0111850\n0.8424534\n0.0460995\n0.0491621\n0.0110233\n0.0019191\n0.3102030\n\n\nS3NoCD\n0.4637853\n0.4155681\n0.5866172\n0.0190054\n0.1355252\n0.1513708\n0.0796683\n0.4141116\n0.0774707\n0.1304099\n0.0112712\n0.8425605\n0.0460767\n0.0491056\n0.0110053\n0.0019082\n0.1861446\n\n\nS3a\n0.4638805\n0.4156993\n0.5867088\n0.0201156\n0.1395918\n0.1808230\n0.0824771\n0.4108939\n0.0771596\n0.1300215\n0.0112568\n0.8425660\n0.0460728\n0.0490892\n0.0110179\n0.0019129\n0.2311946\n\n\nS3b\n0.4637746\n0.4156737\n0.5876605\n0.0227032\n0.1471017\n0.2332507\n0.0873282\n0.4068980\n0.0765657\n0.1291642\n0.0111939\n0.8424573\n0.0460719\n0.0491421\n0.0110558\n0.0019283\n0.3181560\n\n\nS3c\n0.4637315\n0.4155383\n0.5873116\n0.0213262\n0.1432680\n0.2074859\n0.0848645\n0.4090175\n0.0768424\n0.1295624\n0.0112182\n0.8424856\n0.0460621\n0.0491521\n0.0110349\n0.0019208\n0.2742435\n\n\nS3d\n0.4638839\n0.4157521\n0.5872211\n0.0207546\n0.1415673\n0.1954816\n0.0837051\n0.4098442\n0.0769796\n0.1297896\n0.0112424\n0.8425108\n0.0460587\n0.0491390\n0.0110271\n0.0019212\n0.2545329"
},
{
- "objectID": "logbook/posts/2024_10_18/index.html",
- "href": "logbook/posts/2024_10_18/index.html",
- "title": "Day 16",
+ "objectID": "quarto_site/show_reproduction.html#time-elapsed",
+ "href": "quarto_site/show_reproduction.html#time-elapsed",
+ "title": "Main analysis",
"section": "",
- "text": "Note\n\n\n\nX. Total time used: X"
+ "text": "Run time for this notebook:\nend.time <- Sys.time()\ntime.taken <- end.time - start.time\ntime.taken\n## Time difference of 20.64576 hours"
},
{
- "objectID": "logbook/posts/2024_10_18/index.html#review-sensitivity-analysis-with-100-million",
- "href": "logbook/posts/2024_10_18/index.html#review-sensitivity-analysis-with-100-million",
- "title": "Day 16",
- "section": "09.15-09.20, 09.27-09.33: Review sensitivity analysis with 100 million",
- "text": "09.15-09.20, 09.27-09.33: Review sensitivity analysis with 100 million\nRan Process_Sensitivity_Analysis.Rmd. These are changed to be much more similar to the original, and I feel reasonably similar enough to consider it reproduced, but will certainly need to discuss this, as there are obviously some differences too, so it feels - in this case - a very subjective decision!"
+ "objectID": "index.html",
+ "href": "index.html",
+ "title": "Reproducing Johnson et al. 2021",
+ "section": "",
+ "text": "This book captures the reproduction of:\n\nJohnson, K.M., Sadatsafavi, M., Adibi, A., Lynd, L., Harrison, M., Tavakoli, H., Sin, D., Bryan, S. Cost Effectiveness of Case Detection Strategies for the Early Detection of COPD. Applied Health Economics and Health Policy 19, p203-215 (2021). https://doi.org/10.1007/s40258-020-00616-2.\n\nUse the navigation bar above to view:\n\nOriginal study - the original study article and associated artefacts.\nReproduction - code and documentation from reproduction of the model.\nEvaluation - describes model reproduction success and compares original study against guidelines for sharing research, criteria for journal reproducibility guidelines, and article reporting guidelines.\nLogbook - chronological entries detailing reproduction work.\nSummary - summary of the computational reproducibility assessment."
},
{
- "objectID": "logbook/posts/2024_10_18/index.html#timings",
- "href": "logbook/posts/2024_10_18/index.html#timings",
- "title": "Day 16",
- "section": "Timings",
- "text": "Timings\n\nimport sys\nsys.path.append('../')\nfrom timings import calculate_times\n\n# Minutes used prior to today\nused_to_date = 1146\n\n# Times from today\ntimes = [\n ('09.15', '09.20'),\n ('09.27', '09.33')]\n\ncalculate_times(used_to_date, times)\n\nTime spent today: 11m, or 0h 11m\nTotal used to date: 1157m, or 19h 17m\nTime remaining: 1243m, or 20h 43m\nUsed 48.2% of 40 hours max"
+ "objectID": "index.html#about",
+ "href": "index.html#about",
+ "title": "Reproducing Johnson et al. 2021",
+ "section": "",
+ "text": "This book captures the reproduction of:\n\nJohnson, K.M., Sadatsafavi, M., Adibi, A., Lynd, L., Harrison, M., Tavakoli, H., Sin, D., Bryan, S. Cost Effectiveness of Case Detection Strategies for the Early Detection of COPD. Applied Health Economics and Health Policy 19, p203-215 (2021). https://doi.org/10.1007/s40258-020-00616-2.\n\nUse the navigation bar above to view:\n\nOriginal study - the original study article and associated artefacts.\nReproduction - code and documentation from reproduction of the model.\nEvaluation - describes model reproduction success and compares original study against guidelines for sharing research, criteria for journal reproducibility guidelines, and article reporting guidelines.\nLogbook - chronological entries detailing reproduction work.\nSummary - summary of the computational reproducibility assessment."
+ },
+ {
+ "objectID": "index.html#project-team",
+ "href": "index.html#project-team",
+ "title": "Reproducing Johnson et al. 2021",
+ "section": "Project team",
+ "text": "Project team\n\n\nConducting this reproduction:\n\nAmy Heather \n\nProviding support during the reproduction:\n\nThomas Monks \nAlison Harper \n\nOther members of the team on STARS:\n\nNavonil Mustafee \nAndrew Mayne"
+ },
+ {
+ "objectID": "index.html#protocol",
+ "href": "index.html#protocol",
+ "title": "Reproducing Johnson et al. 2021",
+ "section": "Protocol",
+ "text": "Protocol\nThe protocol for this work is summarised in the diagram below and archived on Zenodo:\n\nHeather, A., Monks, T., Harper, A., Mustafee, N., & Mayne, A. (2024). Protocol for assessing the computational reproducibility of discrete-event simulation models on STARS. Zenodo. https://doi.org/10.5281/zenodo.12179846.\n\n\n\n\nWorkflow for computational reproducibility assessment"
+ },
+ {
+ "objectID": "index.html#citation",
+ "href": "index.html#citation",
+ "title": "Reproducing Johnson et al. 2021",
+ "section": "Citation",
+ "text": "Citation\nAPA:\nHeather A., Monks T., Harper A. (2024). STARS: Computational reproducibility of Johnson et al. 2021 (version 0.1.0). URL: https://github.com/pythonhealthdatascience/stars-reproduce-johnson-2021\nSee CITATION.cff and citation_bibtex.bib for alternative formats."
+ },
+ {
+ "objectID": "index.html#license",
+ "href": "index.html#license",
+ "title": "Reproducing Johnson et al. 2021",
+ "section": "License",
+ "text": "License\nSee License page."
},
{
"objectID": "logbook/posts/2024_10_15/index.html",
@@ -651,172 +735,102 @@
"text": "Timings\n\nimport sys\nsys.path.append('../')\nfrom timings import calculate_times\n\n# Minutes used prior to today\nused_to_date = 1136\n\n# Times from today\ntimes = [\n ('09.28', '09.31'),\n ('09.37', '09.44')]\n\ncalculate_times(used_to_date, times)\n\nTime spent today: 10m, or 0h 10m\nTotal used to date: 1146m, or 19h 6m\nTime remaining: 1254m, or 20h 54m\nUsed 47.8% of 40 hours max"
},
{
- "objectID": "index.html",
- "href": "index.html",
- "title": "Reproducing Johnson et al. 2021",
- "section": "",
- "text": "This book captures the reproduction of:\n\nJohnson, K.M., Sadatsafavi, M., Adibi, A., Lynd, L., Harrison, M., Tavakoli, H., Sin, D., Bryan, S. Cost Effectiveness of Case Detection Strategies for the Early Detection of COPD. Applied Health Economics and Health Policy 19, p203-215 (2021). https://doi.org/10.1007/s40258-020-00616-2.\n\nUse the navigation bar above to view:\n\nOriginal study - the original study article and associated artefacts.\nReproduction - code and documentation from reproduction of the model.\nEvaluation - describes model reproduction success and compares original study against guidelines for sharing research, criteria for journal reproducibility guidelines, and article reporting guidelines.\nLogbook - chronological entries detailing reproduction work.\nSummary - summary of the computational reproducibility assessment."
- },
- {
- "objectID": "index.html#about",
- "href": "index.html#about",
- "title": "Reproducing Johnson et al. 2021",
- "section": "",
- "text": "This book captures the reproduction of:\n\nJohnson, K.M., Sadatsafavi, M., Adibi, A., Lynd, L., Harrison, M., Tavakoli, H., Sin, D., Bryan, S. Cost Effectiveness of Case Detection Strategies for the Early Detection of COPD. Applied Health Economics and Health Policy 19, p203-215 (2021). https://doi.org/10.1007/s40258-020-00616-2.\n\nUse the navigation bar above to view:\n\nOriginal study - the original study article and associated artefacts.\nReproduction - code and documentation from reproduction of the model.\nEvaluation - describes model reproduction success and compares original study against guidelines for sharing research, criteria for journal reproducibility guidelines, and article reporting guidelines.\nLogbook - chronological entries detailing reproduction work.\nSummary - summary of the computational reproducibility assessment."
- },
- {
- "objectID": "index.html#project-team",
- "href": "index.html#project-team",
- "title": "Reproducing Johnson et al. 2021",
- "section": "Project team",
- "text": "Project team\n\n\nConducting this reproduction:\n\nAmy Heather \n\nProviding support during the reproduction:\n\nThomas Monks \nAlison Harper \n\nOther members of the team on STARS:\n\nNavonil Mustafee \nAndrew Mayne"
- },
- {
- "objectID": "index.html#protocol",
- "href": "index.html#protocol",
- "title": "Reproducing Johnson et al. 2021",
- "section": "Protocol",
- "text": "Protocol\nThe protocol for this work is summarised in the diagram below and archived on Zenodo:\n\nHeather, A., Monks, T., Harper, A., Mustafee, N., & Mayne, A. (2024). Protocol for assessing the computational reproducibility of discrete-event simulation models on STARS. Zenodo. https://doi.org/10.5281/zenodo.12179846.\n\n\n\n\nWorkflow for computational reproducibility assessment"
- },
- {
- "objectID": "index.html#citation",
- "href": "index.html#citation",
- "title": "Reproducing Johnson et al. 2021",
- "section": "Citation",
- "text": "Citation\nAPA:\nHeather A., Monks T., Harper A. (2024). STARS: Computational reproducibility of Johnson et al. 2021 (version 0.1.0). URL: https://github.com/pythonhealthdatascience/stars-reproduce-johnson-2021\nSee CITATION.cff and citation_bibtex.bib for alternative formats."
- },
- {
- "objectID": "index.html#license",
- "href": "index.html#license",
- "title": "Reproducing Johnson et al. 2021",
- "section": "License",
- "text": "License\nSee License page."
- },
- {
- "objectID": "quarto_site/show_reproduction.html",
- "href": "quarto_site/show_reproduction.html",
- "title": "Main analysis",
- "section": "",
- "text": "14 October, 2024\n## \n## > errors<-c(\n## + ERR_INCORRECT_SETTING_VARIABLE=-1,\n## + ERR_INCORRECT_VECTOR_SIZE=-2,\n## + ERR_INCORRECT_INPUT_VAR=-3,\n## + ERR_EVENT_STACK_FULL=-4,\n## + .... [TRUNCATED] \n## \n## > record_mode<-c(\n## + record_mode_none=0,\n## + record_mode_agent=1,\n## + record_mode_event=2,\n## + record_mode_some_event=3\n## + )\n## \n## > agent_creation_mode<-c(\n## + agent_creation_mode_one=0,\n## + agent_creation_mode_all=1,\n## + agent_creation_mode_pre=2\n## + )\n## \n## > medication_classes<-c(\n## + MED_CLASS_SABA=1,\n## + MED_CLASS_LABA=2,\n## + MED_CLASS_LAMA=4,\n## + MED_CLASS_ICS=8,\n## + MED_CLASS_MACRO=16\n## + )\n## \n## > events<-c(\n## + event_start=0,\n## + event_fixed=1,\n## + event_birthday=2,\n## + event_smoking_change=3,\n## + event_COPD=4,\n## + event_exacerbat .... [TRUNCATED]\n# CHANGE: Add a timer (will see addition below to print time too)\n# Start timer\nstart.time <- Sys.time()\n# CHANGE: Add save paths (will see later, we also add lines to save to csv)\noutputs <- \"../outputs/\"\nfiles <- list(\n s1 = \"s1.csv\",\n s2 = \"s2.csv\",\n s3 = \"s3.csv\",\n sall = \"sall.csv\",\n ceplane = \"ceplane.csv\",\n clinical = \"clinicalresults.csv\"\n)\n\n# Apply to each element in list to create path to file in outputs\npaths <- lapply(files, function(filename) paste0(outputs, filename))\n## Initializing the session\n\n## [1] 0\nGlobal inputs:\n\nMedication adherence is 0.7\nSmoking adherence is 0.7\nCost discounting: 0.015\nQALY discounting: 0.015\nTime horizon: 20\nThe WTP threshold for NMB is 50000\n\nCase detection inputs:\n\nCase detection occurs at 3 year intervals.\nAn outpatient diagnosis costs 61.18\nThe utility gain due to symptom relief from treatment is 0.0367\n\n\n\nAll patients are eligible. The cost of case detection is:\n\n\n\nNone\nCDQ17\nFlowMeter\nFlowMeter_CDQ\n\n\n\n\n0\n11.56\n30.46\n42.01\n\n\n\n\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nAgents\nPatientYears\nCopdPYs\nNCaseDetections\nDiagnosedPYs\nOverdiagnosedPYs\nSABA\nLAMA\nLAMALABA\nICSLAMALABA\nMild\nModerate\nSevere\nVerySevere\nMildPY\nModeratePY\nSeverePY\nVerySeverePY\nNoCOPD\nGOLD1\nGOLD2\nGOLD3\nGOLD4\nCost\nCostpAgent\nQALY\nQALYpAgent\nNMB\nIncrementalCosts\nIncrementalQALY\nICER\nIncrementalNMB\n\n\n\n\nS1NoCD\n74388747\n1251903576\n142277086\n381929878\n26336153\n26747209\n0.017\n0.135\n0.151\n0.080\n30809289\n5766720\n9701609\n839538.0\n0.217\n0.041\n0.068\n0.006\n1054811684\n57697202\n61465410\n13763906\n2389153\n159789813678\n2148.037\n933216569\n12.545\n625108.5\n0.000\n0.000\nNaN\n0.000\n\n\nS1NoCD2\n74388425\n1251968458\n142302175\n249329256\n26636736\n32661650\n0.021\n0.136\n0.151\n0.080\n30801067\n5759155\n9700407\n837363.0\n0.216\n0.040\n0.068\n0.006\n1054852003\n57672835\n61488796\n13791895\n2388663\n160061317736\n2151.697\n933263557\n12.546\n625139.1\n0.000\n0.000\nNaN\n0.000\n\n\nS1NoCDAvg\n74388586\n1251936017\n142289630\n315629567\n26486444\n29704430\n0.019\n0.136\n0.151\n0.080\n30805178\n5762938\n9701008\n838450.5\n0.216\n0.041\n0.068\n0.006\n1054831844\n57685018\n61477103\n13777900\n2388908\n159925565707\n2149.867\n933240063\n12.545\n625123.8\n0.000\n0.000\nNaN\n0.000\n\n\nS1a\n74380275\n1252107231\n142568770\n372100831\n66270673\n26745802\n0.026\n0.159\n0.310\n0.094\n29815042\n5625494\n9500837\n824680.0\n0.209\n0.039\n0.067\n0.006\n1054730269\n57758498\n61574433\n13835312\n2423951\n181401075128\n2438.833\n934239602\n12.560\n625576.9\n290.796\n0.015\n19152.73\n468.354\n\n\nS1b\n74383024\n1252030978\n142410670\n376764647\n47607118\n26748888\n0.022\n0.148\n0.241\n0.088\n30192660\n5683507\n9596337\n830158.0\n0.212\n0.040\n0.067\n0.006\n1054810413\n57710159\n61499603\n13827904\n2405886\n175747140584\n2362.732\n933787724\n12.554\n625326.0\n214.694\n0.009\n24838.03\n217.495\n\n\nS1c\n74393892\n1252166867\n142379318\n378416895\n41063274\n26758496\n0.020\n0.144\n0.214\n0.085\n30350312\n5702044\n9621386\n828456.0\n0.213\n0.040\n0.068\n0.006\n1054965444\n57726777\n61483651\n13803995\n2400849\n177409072797\n2384.726\n933746980\n12.551\n625184.9\n236.689\n0.006\n37796.71\n76.419\n\n\n\nTreatment rate: SABA is expressed per all patient-years, LAMA, LAMA/LABA, ICS/LAMA/LABA are per COPD patient-years Exacerbations: Total exacerbations and rate per COPD patient-year: GOLD Stage: Cumulative patient-years Cost/QALY: Total cost and QALYs NMB: Net Monetary Benefit is calculated as QALY per patient-year * Lamba - Cost per patient-year\n\n\n\n\n\nPatients with symptoms at year 1 are eligible. The cost of case detection is:\n## Initializing the session\n\n## [1] 0\n\n\n\nNone\nFlowMeter\n\n\n\n\n0\n24.33\n\n\n\n\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nAgents\nPatientYears\nCopdPYs\nNCaseDetections\nDiagnosedPYs\nOverdiagnosedPYs\nSABA\nLAMA\nLAMALABA\nICSLAMALABA\nMild\nModerate\nSevere\nVerySevere\nMildPY\nModeratePY\nSeverePY\nVerySeverePY\nNoCOPD\nGOLD1\nGOLD2\nGOLD3\nGOLD4\nCost\nCostpAgent\nQALY\nQALYpAgent\nNMB\nIncrementalCosts\nIncrementalQALY\nICER\nIncrementalNMB\n\n\n\n\nS2NoCD\n44156404\n729084497\n100330760\n221928932\n19330952\n15481473\n0.017\n0.142\n0.159\n0.087\n22917076\n4253860\n7098458\n609501\n0.228\n0.042\n0.071\n0.006\n597619551\n37246010\n44990133\n11123560\n2050468\n117605933755\n2663.395\n541859031\n12.271\n610904.5\n0.000\n0.000\nNaN\n0.000\n\n\nS2a\n44164773\n729329891\n100492049\n217625447\n37010683\n15485858\n0.023\n0.157\n0.269\n0.097\n22383331\n4181008\n7006346\n603148\n0.223\n0.042\n0.070\n0.006\n597700094\n37284225\n45068617\n11147317\n2064302\n127735308298\n2892.244\n542449119\n12.282\n611227.4\n228.849\n0.011\n20737.2\n322.935\n\n\n\nTreatment rate: SABA is expressed per all patient-years, LAMA, LAMA/LABA, ICS/LAMA/LABA are per COPD patient-years Exacerbations: Total exacerbations and rate per COPD patient-year: GOLD Stage: Cumulative patient-years Cost/QALY: Total cost and QALYs NMB: Net Monetary Benefit is calculated as QALY per patient-year * Lamba - Cost per patient-year\n\n\n\n\n\nEver smokers ≥50 years of age are eligible. The cost of case detection is:\n## Initializing the session\n\n## [1] 0\n\n\n\nNone\nCDQ195\nCDQ165\nFlowMeter\nFlowMeter_CDQ\n\n\n\n\n0\n11.56\n11.56\n24.33\n42.01\n\n\n\n\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nAgents\nPatientYears\nCopdPYs\nNCaseDetections\nDiagnosedPYs\nOverdiagnosedPYs\nSABA\nLAMA\nLAMALABA\nICSLAMALABA\nMild\nModerate\nSevere\nVerySevere\nMildPY\nModeratePY\nSeverePY\nVerySeverePY\nNoCOPD\nGOLD1\nGOLD2\nGOLD3\nGOLD4\nCost\nCostpAgent\nQALY\nQALYpAgent\nNMB\nIncrementalCosts\nIncrementalQALY\nICER\nIncrementalNMB\n\n\n\n\nS3NoCD\n34500334\n520264617\n83469550\n160191877\n15617539\n11630123\n0.018\n0.137\n0.154\n0.086\n19525292\n3631989\n6073558\n522424\n0.234\n0.044\n0.073\n0.006\n415002016\n30608834\n37422882\n9624470\n1788687\n99842281258\n2893.951\n387517176\n11.232\n558719.7\n0.000\n0.000\nNaN\n0.000\n\n\nS3a\n34507415\n520428951\n83482576\n158676520\n22030098\n11639538\n0.021\n0.144\n0.204\n0.091\n19287934\n3609226\n6045313\n521405\n0.231\n0.043\n0.072\n0.006\n415152628\n30610103\n37407716\n9641236\n1794702\n106095806862\n3074.580\n387796823\n11.238\n558829.0\n180.629\n0.006\n31147.82\n109.325\n\n\nS3b\n34499539\n520396932\n83618002\n155590263\n34428826\n11625591\n0.027\n0.156\n0.293\n0.099\n18988453\n3564625\n5980823\n516662\n0.227\n0.043\n0.072\n0.006\n414988556\n30607762\n37472866\n9688489\n1813970\n110363699543\n3198.991\n388038736\n11.248\n559183.5\n305.039\n0.015\n19837.73\n463.797\n\n\nS3c\n34496334\n520227370\n83568351\n157099275\n28171361\n11622564\n0.024\n0.150\n0.249\n0.095\n19145218\n3585036\n6010150\n518449\n0.229\n0.043\n0.072\n0.006\n414875686\n30589301\n37479876\n9661353\n1804425\n107627143927\n3119.959\n387776879\n11.241\n558935.2\n226.008\n0.009\n25592.92\n215.536\n\n\nS3d\n34507664\n520495018\n83555471\n157877292\n25364366\n11630839\n0.023\n0.147\n0.229\n0.093\n19209915\n3595849\n6028084\n520340\n0.230\n0.043\n0.072\n0.006\n415141325\n30594930\n37472224\n9653084\n1805226\n108255487965\n3137.143\n387911628\n11.241\n558928.8\n243.192\n0.009\n26887.15\n209.054\n\n\n\nTreatment rate: SABA is expressed per all patient-years, LAMA, LAMA/LABA, ICS/LAMA/LABA are per COPD patient-years Exacerbations: Total exacerbations and rate per COPD patient-year GOLD Stage: Cumulative patient-years Cost/QALY: Total cost and QALYs NMB: Net Monetary Benefit is calculated as QALY per patient-year * Lamba - Cost per patient-year\n\n\n\n\n\nOrdered by descending Net Monetary Benefit\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nAgents\nCost\nCostpAgent\nQALY\nQALYpAgent\nICER\nIncrementalNMB\n\n\n\n\nS1a\n74380275\n181401075128\n2438.833\n934239602\n12.560\n19152.73\n468.354\n\n\nS3b\n34499539\n110363699543\n3198.991\n388038736\n11.248\n19837.73\n463.797\n\n\nS2a\n44164773\n127735308298\n2892.244\n542449119\n12.282\n20737.20\n322.935\n\n\nS1b\n74383024\n175747140584\n2362.732\n933787724\n12.554\n24838.03\n217.495\n\n\nS3c\n34496334\n107627143927\n3119.959\n387776879\n11.241\n25592.92\n215.536\n\n\nS3d\n34507664\n108255487965\n3137.143\n387911628\n11.241\n26887.15\n209.054\n\n\nS3a\n34507415\n106095806862\n3074.580\n387796823\n11.238\n31147.82\n109.325\n\n\nS1c\n74393892\n177409072797\n2384.726\n933746980\n12.551\n37796.71\n76.419\n\n\nS1NoCD\n74388747\n159789813678\n2148.037\n933216569\n12.545\nNaN\n0.000\n\n\nS2NoCD\n44156404\n117605933755\n2663.395\n541859031\n12.271\nNaN\n0.000\n\n\nS3NoCD\n34500334\n99842281258\n2893.951\n387517176\n11.232\nNaN\n0.000\n\n\n\n\n\n\n\nAdjusted to the total population\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nAgents\nPropAgents\nCost\nCostpAgent\nCostpAgentExcluded\nCostpAgentAll\nQALY\nQALYpAgent\nQALYpAgentExcluded\nQALYpAgentAll\nIncrementalCosts\nIncrementalQALY\nICERAdj\nICER\nINMB\n\n\n\n\nS1NoCDAvg\n74388586\n1.0000000\n159925565707\n2149.867\n0.000\n2149.867\n933240063\n12.54547\n0.00000\n12.54547\n0.00000\n0.0000000\nNaN\nNaN\n0.00000\n\n\nS1a\n74380275\n1.0000000\n181401075128\n2438.833\n0.000\n2438.833\n934239602\n12.56031\n0.00000\n12.56031\n288.96609\n0.0148400\n19472.10\n19152.73\n453.03421\n\n\nS1b\n74383024\n1.0000000\n175747140584\n2362.732\n0.000\n2362.732\n933787724\n12.55377\n0.00000\n12.55377\n212.86487\n0.0083008\n25643.89\n24838.02\n202.17518\n\n\nS1c\n74393892\n1.0000000\n177409072797\n2384.726\n0.000\n2384.726\n933746980\n12.55139\n0.00000\n12.55139\n234.85933\n0.0059192\n39677.70\n37796.70\n61.09955\n\n\nS2NoCD\n44156404\n0.5935911\n117605933755\n2663.395\n1399.821\n2149.867\n541859031\n12.27136\n12.94584\n12.54547\n0.00000\n0.0000000\nNaN\nNaN\n0.00000\n\n\nS2a\n44164773\n0.5937036\n127735308298\n2892.244\n1399.821\n2285.878\n542449119\n12.28239\n12.94584\n12.55195\n136.01091\n0.0064761\n21002.14\n20737.20\n187.79161\n\n\nS3NoCD\n34500334\n0.4637853\n99842281258\n2893.951\n1506.290\n2149.867\n387517176\n11.23227\n13.68129\n12.54547\n0.00000\n0.0000000\nNaN\nNaN\n0.00000\n\n\nS3a\n34507415\n0.4638805\n106095806862\n3074.580\n1506.290\n2233.789\n387796823\n11.23807\n13.68129\n12.54793\n83.92228\n0.0024570\n34156.94\n31147.82\n38.92579\n\n\nS3b\n34499539\n0.4637746\n110363699543\n3198.991\n1506.290\n2291.322\n388038736\n11.24765\n13.68129\n12.55263\n141.45471\n0.0071575\n19763.12\n19837.73\n216.42072\n\n\nS3c\n34496334\n0.4637315\n107627143927\n3119.959\n1506.290\n2254.599\n387776879\n11.24110\n13.68129\n12.54970\n104.73230\n0.0042268\n24777.92\n25592.93\n106.60970\n\n\nS3d\n34507664\n0.4638839\n108255487965\n3137.143\n1506.290\n2262.817\n387911628\n11.24132\n13.68129\n12.54943\n112.94966\n0.0039545\n28562.49\n26887.15\n84.77408\n\n\n\n\n\n\n\nAdjusted to the total population\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nPropAgents\nProppPatientYears\nProppCopdPYs\nSABAAll\nLAMAAll\nLAMALABAAll\nICSLAMALABAAll\nMildpAgentAll\nModeratepAgentAll\nSeverepAgentAll\nVerySeverepAgentAll\nNoCOPDpPYAll\nGOLD1pPYAll\nGOLD2pPYAll\nGOLD3pPYAll\nGOLD4pPYAll\nDiagnosedpPYAll\n\n\n\n\nS1NoCDAvg\n1.0000000\n1.0000000\n1.0000000\n0.0190054\n0.1355252\n0.1513708\n0.0796683\n0.4141116\n0.0774707\n0.1304099\n0.0112712\n0.8425605\n0.0460767\n0.0491056\n0.0110053\n0.0019082\n0.1861446\n\n\nS1a\n1.0000000\n1.0000000\n1.0000000\n0.0264033\n0.1593018\n0.3099946\n0.0940914\n0.4008461\n0.0756315\n0.1277333\n0.0110873\n0.8423642\n0.0461290\n0.0491766\n0.0110496\n0.0019359\n0.4648330\n\n\nS1b\n1.0000000\n1.0000000\n1.0000000\n0.0216398\n0.1482342\n0.2406521\n0.0877947\n0.4059079\n0.0764087\n0.1290125\n0.0111606\n0.8424795\n0.0460932\n0.0491199\n0.0110444\n0.0019216\n0.3342946\n\n\nS1c\n1.0000000\n1.0000000\n1.0000000\n0.0201517\n0.1441110\n0.2141912\n0.0851720\n0.4079678\n0.0766467\n0.1293303\n0.0111361\n0.8425119\n0.0461015\n0.0491018\n0.0110241\n0.0019174\n0.2884076\n\n\nS2NoCD\n0.5935911\n0.5823656\n0.7051165\n0.0190054\n0.1355252\n0.1513708\n0.0796683\n0.4141116\n0.0774707\n0.1304099\n0.0112712\n0.8425605\n0.0460767\n0.0491056\n0.0110053\n0.0019082\n0.1861446\n\n\nS2a\n0.5937036\n0.5825616\n0.7062500\n0.0223572\n0.1465203\n0.2287835\n0.0867962\n0.4069072\n0.0764858\n0.1291620\n0.0111850\n0.8424534\n0.0460995\n0.0491621\n0.0110233\n0.0019191\n0.3102030\n\n\nS3NoCD\n0.4637853\n0.4155681\n0.5866172\n0.0190054\n0.1355252\n0.1513708\n0.0796683\n0.4141116\n0.0774707\n0.1304099\n0.0112712\n0.8425605\n0.0460767\n0.0491056\n0.0110053\n0.0019082\n0.1861446\n\n\nS3a\n0.4638805\n0.4156993\n0.5867088\n0.0201156\n0.1395918\n0.1808230\n0.0824771\n0.4108939\n0.0771596\n0.1300215\n0.0112568\n0.8425660\n0.0460728\n0.0490892\n0.0110179\n0.0019129\n0.2311946\n\n\nS3b\n0.4637746\n0.4156737\n0.5876605\n0.0227032\n0.1471017\n0.2332507\n0.0873282\n0.4068980\n0.0765657\n0.1291642\n0.0111939\n0.8424573\n0.0460719\n0.0491421\n0.0110558\n0.0019283\n0.3181560\n\n\nS3c\n0.4637315\n0.4155383\n0.5873116\n0.0213262\n0.1432680\n0.2074859\n0.0848645\n0.4090175\n0.0768424\n0.1295624\n0.0112182\n0.8424856\n0.0460621\n0.0491521\n0.0110349\n0.0019208\n0.2742435\n\n\nS3d\n0.4638839\n0.4157521\n0.5872211\n0.0207546\n0.1415673\n0.1954816\n0.0837051\n0.4098442\n0.0769796\n0.1297896\n0.0112424\n0.8425108\n0.0460587\n0.0491390\n0.0110271\n0.0019212\n0.2545329\n\n\n\n\n\n\nRun time for this notebook:\nend.time <- Sys.time()\ntime.taken <- end.time - start.time\ntime.taken\n## Time difference of 20.64576 hours"
- },
- {
- "objectID": "quarto_site/show_reproduction.html#s1-all-patients-scenario",
- "href": "quarto_site/show_reproduction.html#s1-all-patients-scenario",
- "title": "Main analysis",
+ "objectID": "quarto_site/reproduction_readme.html",
+ "href": "quarto_site/reproduction_readme.html",
+ "title": "README for reproduction",
"section": "",
- "text": "All patients are eligible. The cost of case detection is:\n\n\n\nNone\nCDQ17\nFlowMeter\nFlowMeter_CDQ\n\n\n\n\n0\n11.56\n30.46\n42.01\n\n\n\n\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nAgents\nPatientYears\nCopdPYs\nNCaseDetections\nDiagnosedPYs\nOverdiagnosedPYs\nSABA\nLAMA\nLAMALABA\nICSLAMALABA\nMild\nModerate\nSevere\nVerySevere\nMildPY\nModeratePY\nSeverePY\nVerySeverePY\nNoCOPD\nGOLD1\nGOLD2\nGOLD3\nGOLD4\nCost\nCostpAgent\nQALY\nQALYpAgent\nNMB\nIncrementalCosts\nIncrementalQALY\nICER\nIncrementalNMB\n\n\n\n\nS1NoCD\n74388747\n1251903576\n142277086\n381929878\n26336153\n26747209\n0.017\n0.135\n0.151\n0.080\n30809289\n5766720\n9701609\n839538.0\n0.217\n0.041\n0.068\n0.006\n1054811684\n57697202\n61465410\n13763906\n2389153\n159789813678\n2148.037\n933216569\n12.545\n625108.5\n0.000\n0.000\nNaN\n0.000\n\n\nS1NoCD2\n74388425\n1251968458\n142302175\n249329256\n26636736\n32661650\n0.021\n0.136\n0.151\n0.080\n30801067\n5759155\n9700407\n837363.0\n0.216\n0.040\n0.068\n0.006\n1054852003\n57672835\n61488796\n13791895\n2388663\n160061317736\n2151.697\n933263557\n12.546\n625139.1\n0.000\n0.000\nNaN\n0.000\n\n\nS1NoCDAvg\n74388586\n1251936017\n142289630\n315629567\n26486444\n29704430\n0.019\n0.136\n0.151\n0.080\n30805178\n5762938\n9701008\n838450.5\n0.216\n0.041\n0.068\n0.006\n1054831844\n57685018\n61477103\n13777900\n2388908\n159925565707\n2149.867\n933240063\n12.545\n625123.8\n0.000\n0.000\nNaN\n0.000\n\n\nS1a\n74380275\n1252107231\n142568770\n372100831\n66270673\n26745802\n0.026\n0.159\n0.310\n0.094\n29815042\n5625494\n9500837\n824680.0\n0.209\n0.039\n0.067\n0.006\n1054730269\n57758498\n61574433\n13835312\n2423951\n181401075128\n2438.833\n934239602\n12.560\n625576.9\n290.796\n0.015\n19152.73\n468.354\n\n\nS1b\n74383024\n1252030978\n142410670\n376764647\n47607118\n26748888\n0.022\n0.148\n0.241\n0.088\n30192660\n5683507\n9596337\n830158.0\n0.212\n0.040\n0.067\n0.006\n1054810413\n57710159\n61499603\n13827904\n2405886\n175747140584\n2362.732\n933787724\n12.554\n625326.0\n214.694\n0.009\n24838.03\n217.495\n\n\nS1c\n74393892\n1252166867\n142379318\n378416895\n41063274\n26758496\n0.020\n0.144\n0.214\n0.085\n30350312\n5702044\n9621386\n828456.0\n0.213\n0.040\n0.068\n0.006\n1054965444\n57726777\n61483651\n13803995\n2400849\n177409072797\n2384.726\n933746980\n12.551\n625184.9\n236.689\n0.006\n37796.71\n76.419\n\n\n\nTreatment rate: SABA is expressed per all patient-years, LAMA, LAMA/LABA, ICS/LAMA/LABA are per COPD patient-years Exacerbations: Total exacerbations and rate per COPD patient-year: GOLD Stage: Cumulative patient-years Cost/QALY: Total cost and QALYs NMB: Net Monetary Benefit is calculated as QALY per patient-year * Lamba - Cost per patient-year"
+ "text": "Please note: This is a template README and has not yet been completed\n\n\n\nTBC\n\n\n\nTBC\n\n\n\n\n\nTBC\n\n\n\nTBC\n\n\n\nTBC\n\n\n\n\nTBC\n\n\n\n\nTBC\n\n\n\nTBC"
},
{
- "objectID": "quarto_site/show_reproduction.html#s2-symptomatic-patients-scenario",
- "href": "quarto_site/show_reproduction.html#s2-symptomatic-patients-scenario",
- "title": "Main analysis",
+ "objectID": "quarto_site/reproduction_readme.html#model-summary",
+ "href": "quarto_site/reproduction_readme.html#model-summary",
+ "title": "README for reproduction",
"section": "",
- "text": "Patients with symptoms at year 1 are eligible. The cost of case detection is:\n## Initializing the session\n\n## [1] 0\n\n\n\nNone\nFlowMeter\n\n\n\n\n0\n24.33\n\n\n\n\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nAgents\nPatientYears\nCopdPYs\nNCaseDetections\nDiagnosedPYs\nOverdiagnosedPYs\nSABA\nLAMA\nLAMALABA\nICSLAMALABA\nMild\nModerate\nSevere\nVerySevere\nMildPY\nModeratePY\nSeverePY\nVerySeverePY\nNoCOPD\nGOLD1\nGOLD2\nGOLD3\nGOLD4\nCost\nCostpAgent\nQALY\nQALYpAgent\nNMB\nIncrementalCosts\nIncrementalQALY\nICER\nIncrementalNMB\n\n\n\n\nS2NoCD\n44156404\n729084497\n100330760\n221928932\n19330952\n15481473\n0.017\n0.142\n0.159\n0.087\n22917076\n4253860\n7098458\n609501\n0.228\n0.042\n0.071\n0.006\n597619551\n37246010\n44990133\n11123560\n2050468\n117605933755\n2663.395\n541859031\n12.271\n610904.5\n0.000\n0.000\nNaN\n0.000\n\n\nS2a\n44164773\n729329891\n100492049\n217625447\n37010683\n15485858\n0.023\n0.157\n0.269\n0.097\n22383331\n4181008\n7006346\n603148\n0.223\n0.042\n0.070\n0.006\n597700094\n37284225\n45068617\n11147317\n2064302\n127735308298\n2892.244\n542449119\n12.282\n611227.4\n228.849\n0.011\n20737.2\n322.935\n\n\n\nTreatment rate: SABA is expressed per all patient-years, LAMA, LAMA/LABA, ICS/LAMA/LABA are per COPD patient-years Exacerbations: Total exacerbations and rate per COPD patient-year: GOLD Stage: Cumulative patient-years Cost/QALY: Total cost and QALYs NMB: Net Monetary Benefit is calculated as QALY per patient-year * Lamba - Cost per patient-year"
+ "text": "TBC"
},
{
- "objectID": "quarto_site/show_reproduction.html#s3-smoking-history-scenario",
- "href": "quarto_site/show_reproduction.html#s3-smoking-history-scenario",
- "title": "Main analysis",
+ "objectID": "quarto_site/reproduction_readme.html#scope-of-the-reproduction",
+ "href": "quarto_site/reproduction_readme.html#scope-of-the-reproduction",
+ "title": "README for reproduction",
"section": "",
- "text": "Ever smokers ≥50 years of age are eligible. The cost of case detection is:\n## Initializing the session\n\n## [1] 0\n\n\n\nNone\nCDQ195\nCDQ165\nFlowMeter\nFlowMeter_CDQ\n\n\n\n\n0\n11.56\n11.56\n24.33\n42.01\n\n\n\n\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n## Initializing the session\n\n## [1] 0\n\n## [1] 0\n\n## Terminating the session\n\n## [1] 0\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nAgents\nPatientYears\nCopdPYs\nNCaseDetections\nDiagnosedPYs\nOverdiagnosedPYs\nSABA\nLAMA\nLAMALABA\nICSLAMALABA\nMild\nModerate\nSevere\nVerySevere\nMildPY\nModeratePY\nSeverePY\nVerySeverePY\nNoCOPD\nGOLD1\nGOLD2\nGOLD3\nGOLD4\nCost\nCostpAgent\nQALY\nQALYpAgent\nNMB\nIncrementalCosts\nIncrementalQALY\nICER\nIncrementalNMB\n\n\n\n\nS3NoCD\n34500334\n520264617\n83469550\n160191877\n15617539\n11630123\n0.018\n0.137\n0.154\n0.086\n19525292\n3631989\n6073558\n522424\n0.234\n0.044\n0.073\n0.006\n415002016\n30608834\n37422882\n9624470\n1788687\n99842281258\n2893.951\n387517176\n11.232\n558719.7\n0.000\n0.000\nNaN\n0.000\n\n\nS3a\n34507415\n520428951\n83482576\n158676520\n22030098\n11639538\n0.021\n0.144\n0.204\n0.091\n19287934\n3609226\n6045313\n521405\n0.231\n0.043\n0.072\n0.006\n415152628\n30610103\n37407716\n9641236\n1794702\n106095806862\n3074.580\n387796823\n11.238\n558829.0\n180.629\n0.006\n31147.82\n109.325\n\n\nS3b\n34499539\n520396932\n83618002\n155590263\n34428826\n11625591\n0.027\n0.156\n0.293\n0.099\n18988453\n3564625\n5980823\n516662\n0.227\n0.043\n0.072\n0.006\n414988556\n30607762\n37472866\n9688489\n1813970\n110363699543\n3198.991\n388038736\n11.248\n559183.5\n305.039\n0.015\n19837.73\n463.797\n\n\nS3c\n34496334\n520227370\n83568351\n157099275\n28171361\n11622564\n0.024\n0.150\n0.249\n0.095\n19145218\n3585036\n6010150\n518449\n0.229\n0.043\n0.072\n0.006\n414875686\n30589301\n37479876\n9661353\n1804425\n107627143927\n3119.959\n387776879\n11.241\n558935.2\n226.008\n0.009\n25592.92\n215.536\n\n\nS3d\n34507664\n520495018\n83555471\n157877292\n25364366\n11630839\n0.023\n0.147\n0.229\n0.093\n19209915\n3595849\n6028084\n520340\n0.230\n0.043\n0.072\n0.006\n415141325\n30594930\n37472224\n9653084\n1805226\n108255487965\n3137.143\n387911628\n11.241\n558928.8\n243.192\n0.009\n26887.15\n209.054\n\n\n\nTreatment rate: SABA is expressed per all patient-years, LAMA, LAMA/LABA, ICS/LAMA/LABA are per COPD patient-years Exacerbations: Total exacerbations and rate per COPD patient-year GOLD Stage: Cumulative patient-years Cost/QALY: Total cost and QALYs NMB: Net Monetary Benefit is calculated as QALY per patient-year * Lamba - Cost per patient-year"
+ "text": "TBC"
},
{
- "objectID": "quarto_site/show_reproduction.html#all-scenarios",
- "href": "quarto_site/show_reproduction.html#all-scenarios",
- "title": "Main analysis",
+ "objectID": "quarto_site/reproduction_readme.html#reproducing-these-results",
+ "href": "quarto_site/reproduction_readme.html#reproducing-these-results",
+ "title": "README for reproduction",
"section": "",
- "text": "Ordered by descending Net Monetary Benefit\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nAgents\nCost\nCostpAgent\nQALY\nQALYpAgent\nICER\nIncrementalNMB\n\n\n\n\nS1a\n74380275\n181401075128\n2438.833\n934239602\n12.560\n19152.73\n468.354\n\n\nS3b\n34499539\n110363699543\n3198.991\n388038736\n11.248\n19837.73\n463.797\n\n\nS2a\n44164773\n127735308298\n2892.244\n542449119\n12.282\n20737.20\n322.935\n\n\nS1b\n74383024\n175747140584\n2362.732\n933787724\n12.554\n24838.03\n217.495\n\n\nS3c\n34496334\n107627143927\n3119.959\n387776879\n11.241\n25592.92\n215.536\n\n\nS3d\n34507664\n108255487965\n3137.143\n387911628\n11.241\n26887.15\n209.054\n\n\nS3a\n34507415\n106095806862\n3074.580\n387796823\n11.238\n31147.82\n109.325\n\n\nS1c\n74393892\n177409072797\n2384.726\n933746980\n12.551\n37796.71\n76.419\n\n\nS1NoCD\n74388747\n159789813678\n2148.037\n933216569\n12.545\nNaN\n0.000\n\n\nS2NoCD\n44156404\n117605933755\n2663.395\n541859031\n12.271\nNaN\n0.000\n\n\nS3NoCD\n34500334\n99842281258\n2893.951\n387517176\n11.232\nNaN\n0.000"
+ "text": "TBC\n\n\n\nTBC\n\n\n\nTBC"
},
{
- "objectID": "quarto_site/show_reproduction.html#cost-effectiveness-plane",
- "href": "quarto_site/show_reproduction.html#cost-effectiveness-plane",
- "title": "Main analysis",
+ "objectID": "quarto_site/reproduction_readme.html#reproduction-specs-and-runtime",
+ "href": "quarto_site/reproduction_readme.html#reproduction-specs-and-runtime",
+ "title": "README for reproduction",
"section": "",
- "text": "Adjusted to the total population\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nAgents\nPropAgents\nCost\nCostpAgent\nCostpAgentExcluded\nCostpAgentAll\nQALY\nQALYpAgent\nQALYpAgentExcluded\nQALYpAgentAll\nIncrementalCosts\nIncrementalQALY\nICERAdj\nICER\nINMB\n\n\n\n\nS1NoCDAvg\n74388586\n1.0000000\n159925565707\n2149.867\n0.000\n2149.867\n933240063\n12.54547\n0.00000\n12.54547\n0.00000\n0.0000000\nNaN\nNaN\n0.00000\n\n\nS1a\n74380275\n1.0000000\n181401075128\n2438.833\n0.000\n2438.833\n934239602\n12.56031\n0.00000\n12.56031\n288.96609\n0.0148400\n19472.10\n19152.73\n453.03421\n\n\nS1b\n74383024\n1.0000000\n175747140584\n2362.732\n0.000\n2362.732\n933787724\n12.55377\n0.00000\n12.55377\n212.86487\n0.0083008\n25643.89\n24838.02\n202.17518\n\n\nS1c\n74393892\n1.0000000\n177409072797\n2384.726\n0.000\n2384.726\n933746980\n12.55139\n0.00000\n12.55139\n234.85933\n0.0059192\n39677.70\n37796.70\n61.09955\n\n\nS2NoCD\n44156404\n0.5935911\n117605933755\n2663.395\n1399.821\n2149.867\n541859031\n12.27136\n12.94584\n12.54547\n0.00000\n0.0000000\nNaN\nNaN\n0.00000\n\n\nS2a\n44164773\n0.5937036\n127735308298\n2892.244\n1399.821\n2285.878\n542449119\n12.28239\n12.94584\n12.55195\n136.01091\n0.0064761\n21002.14\n20737.20\n187.79161\n\n\nS3NoCD\n34500334\n0.4637853\n99842281258\n2893.951\n1506.290\n2149.867\n387517176\n11.23227\n13.68129\n12.54547\n0.00000\n0.0000000\nNaN\nNaN\n0.00000\n\n\nS3a\n34507415\n0.4638805\n106095806862\n3074.580\n1506.290\n2233.789\n387796823\n11.23807\n13.68129\n12.54793\n83.92228\n0.0024570\n34156.94\n31147.82\n38.92579\n\n\nS3b\n34499539\n0.4637746\n110363699543\n3198.991\n1506.290\n2291.322\n388038736\n11.24765\n13.68129\n12.55263\n141.45471\n0.0071575\n19763.12\n19837.73\n216.42072\n\n\nS3c\n34496334\n0.4637315\n107627143927\n3119.959\n1506.290\n2254.599\n387776879\n11.24110\n13.68129\n12.54970\n104.73230\n0.0042268\n24777.92\n25592.93\n106.60970\n\n\nS3d\n34507664\n0.4638839\n108255487965\n3137.143\n1506.290\n2262.817\n387911628\n11.24132\n13.68129\n12.54943\n112.94966\n0.0039545\n28562.49\n26887.15\n84.77408"
+ "text": "TBC"
},
{
- "objectID": "quarto_site/show_reproduction.html#clinical-results-for-all-scenarios",
- "href": "quarto_site/show_reproduction.html#clinical-results-for-all-scenarios",
- "title": "Main analysis",
+ "objectID": "quarto_site/reproduction_readme.html#citation",
+ "href": "quarto_site/reproduction_readme.html#citation",
+ "title": "README for reproduction",
"section": "",
- "text": "Adjusted to the total population\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nScenario\nPropAgents\nProppPatientYears\nProppCopdPYs\nSABAAll\nLAMAAll\nLAMALABAAll\nICSLAMALABAAll\nMildpAgentAll\nModeratepAgentAll\nSeverepAgentAll\nVerySeverepAgentAll\nNoCOPDpPYAll\nGOLD1pPYAll\nGOLD2pPYAll\nGOLD3pPYAll\nGOLD4pPYAll\nDiagnosedpPYAll\n\n\n\n\nS1NoCDAvg\n1.0000000\n1.0000000\n1.0000000\n0.0190054\n0.1355252\n0.1513708\n0.0796683\n0.4141116\n0.0774707\n0.1304099\n0.0112712\n0.8425605\n0.0460767\n0.0491056\n0.0110053\n0.0019082\n0.1861446\n\n\nS1a\n1.0000000\n1.0000000\n1.0000000\n0.0264033\n0.1593018\n0.3099946\n0.0940914\n0.4008461\n0.0756315\n0.1277333\n0.0110873\n0.8423642\n0.0461290\n0.0491766\n0.0110496\n0.0019359\n0.4648330\n\n\nS1b\n1.0000000\n1.0000000\n1.0000000\n0.0216398\n0.1482342\n0.2406521\n0.0877947\n0.4059079\n0.0764087\n0.1290125\n0.0111606\n0.8424795\n0.0460932\n0.0491199\n0.0110444\n0.0019216\n0.3342946\n\n\nS1c\n1.0000000\n1.0000000\n1.0000000\n0.0201517\n0.1441110\n0.2141912\n0.0851720\n0.4079678\n0.0766467\n0.1293303\n0.0111361\n0.8425119\n0.0461015\n0.0491018\n0.0110241\n0.0019174\n0.2884076\n\n\nS2NoCD\n0.5935911\n0.5823656\n0.7051165\n0.0190054\n0.1355252\n0.1513708\n0.0796683\n0.4141116\n0.0774707\n0.1304099\n0.0112712\n0.8425605\n0.0460767\n0.0491056\n0.0110053\n0.0019082\n0.1861446\n\n\nS2a\n0.5937036\n0.5825616\n0.7062500\n0.0223572\n0.1465203\n0.2287835\n0.0867962\n0.4069072\n0.0764858\n0.1291620\n0.0111850\n0.8424534\n0.0460995\n0.0491621\n0.0110233\n0.0019191\n0.3102030\n\n\nS3NoCD\n0.4637853\n0.4155681\n0.5866172\n0.0190054\n0.1355252\n0.1513708\n0.0796683\n0.4141116\n0.0774707\n0.1304099\n0.0112712\n0.8425605\n0.0460767\n0.0491056\n0.0110053\n0.0019082\n0.1861446\n\n\nS3a\n0.4638805\n0.4156993\n0.5867088\n0.0201156\n0.1395918\n0.1808230\n0.0824771\n0.4108939\n0.0771596\n0.1300215\n0.0112568\n0.8425660\n0.0460728\n0.0490892\n0.0110179\n0.0019129\n0.2311946\n\n\nS3b\n0.4637746\n0.4156737\n0.5876605\n0.0227032\n0.1471017\n0.2332507\n0.0873282\n0.4068980\n0.0765657\n0.1291642\n0.0111939\n0.8424573\n0.0460719\n0.0491421\n0.0110558\n0.0019283\n0.3181560\n\n\nS3c\n0.4637315\n0.4155383\n0.5873116\n0.0213262\n0.1432680\n0.2074859\n0.0848645\n0.4090175\n0.0768424\n0.1295624\n0.0112182\n0.8424856\n0.0460621\n0.0491521\n0.0110349\n0.0019208\n0.2742435\n\n\nS3d\n0.4638839\n0.4157521\n0.5872211\n0.0207546\n0.1415673\n0.1954816\n0.0837051\n0.4098442\n0.0769796\n0.1297896\n0.0112424\n0.8425108\n0.0460587\n0.0491390\n0.0110271\n0.0019212\n0.2545329"
+ "text": "TBC"
},
{
- "objectID": "quarto_site/show_reproduction.html#time-elapsed",
- "href": "quarto_site/show_reproduction.html#time-elapsed",
- "title": "Main analysis",
+ "objectID": "quarto_site/reproduction_readme.html#license",
+ "href": "quarto_site/reproduction_readme.html#license",
+ "title": "README for reproduction",
"section": "",
- "text": "Run time for this notebook:\nend.time <- Sys.time()\ntime.taken <- end.time - start.time\ntime.taken\n## Time difference of 20.64576 hours"
+ "text": "TBC"
},
{
- "objectID": "logbook/posts/2024_09_06/index.html",
- "href": "logbook/posts/2024_09_06/index.html",
- "title": "Day 7",
+ "objectID": "evaluation/badges.html",
+ "href": "evaluation/badges.html",
+ "title": "Journal badges",
"section": "",
- "text": "Note\n\n\n\nWorking on Table 3, Figure 3 and Appendix 6 (none yet reproduced). Total time used: 14h 8m (35.3%)."
- },
- {
- "objectID": "logbook/posts/2024_09_06/index.html#revisiting-table-3",
- "href": "logbook/posts/2024_09_06/index.html#revisiting-table-3",
- "title": "Day 7",
- "section": "09.36-09.48: Revisiting Table 3",
- "text": "09.36-09.48: Revisiting Table 3\n\n\n\n\n\n\nReflection\n\n\n\nI’ve been finding this quite tricky/slow to get my head around, and want to acknowledge that part of the reason for that will be that this is a complex model subject and - though having studied and done a little work in health economics - I am not a health economist.\nThere are some things that might have helped though, such as having a clear indicator of which columns and outputs match up with the paper, as its still taking me a while to make sure I’m looking at the same ones, particularly as the results don’t match up.\n\n\nContinuing from yesterday, when I’d realised that incremental costs and QALYs that subjectively look really similar actually result in really different ICERs, I explored the possibility that the difference in results is actually just down to the number of agents.\nLooking in their GitHub commit history, I found results from them running with 1 million base agents (like me here):\n\nCase_Detection_Results.md\nCase_Detection_Results_5yrs.md\n\nComparing those against Table 3, they are very different from them, and from me as well. This was despite having set the seed to 333, so it appears the seed control might not be working as expected, whether that be my fault for any changes or just in general.\nHowever, their latest results (from 50 million agents) still look quite different to the original, e.g. S1b 3yr ICER $31075/QALY in their repository versus $25,894/QALY in the paper.\nHence, I’m wary that running with 50 million agents on my machine - even if I could - still might not get me the same results.\n\n\n\n\n\n\nReflection\n\n\n\nUncertain on if seed control implemented properly. Also, here, it appears that either the number of agents can’t be reduced to get same results, and that a really high number are needed for stability, or that its actually just a seed issue."
- },
- {
- "objectID": "logbook/posts/2024_09_06/index.html#working-on-figure-3",
- "href": "logbook/posts/2024_09_06/index.html#working-on-figure-3",
- "title": "Day 7",
- "section": "09.49-10.40: Working on Figure 3",
- "text": "09.49-10.40: Working on Figure 3\nTaking a break on Table 3, I looked to Figure 3. The current model code does output two visually similar figures, and it appears I would just need to combine these to get the figure from the paper. This was very easy to get started with - just copying it over, but tweaking to add a grouping for 3 years and 5 years.\nI add the efficiency frontier based on the description in the text - looking for lowest cost-effectiveness ratio, and then ruling out strategies that were less effective with a higher ICER. I’m not 100% certain I’ve implemented it correctly, but certainly, it looks different any since the points are all different.\n\n\n\nFigure 3 Attempt\n\n\n\n\n\n\n\n\nReflection\n\n\n\nWas really helpful having partial code for figures, but full code would have been fab (as e.g. I was uncertain if I’m implementing efficiency frontier correctly)."
- },
- {
- "objectID": "logbook/posts/2024_09_06/index.html#troubleshooting-difference-in-results",
- "href": "logbook/posts/2024_09_06/index.html#troubleshooting-difference-in-results",
- "title": "Day 7",
- "section": "11.00-11.11: Troubleshooting difference in results",
- "text": "11.00-11.11: Troubleshooting difference in results\nI stuck with Table 3 and Figure 3, trying to figure out the reason for the differing results.\nI checked whether the incremental costs and QALYs were based on the adjusted or unadjusted results and, from trying a calculation manually, could see it definitely uses the adjusted.\nI looked through the history of some results in their GitHub, just to help clarify to myself the discrepancies I’m seeing v.s. what they see.\nICERs:\n\n\n\nScenario (3yr)\nTable 3\nMe (1m)\nThem (50m)\nThem (1m)\n\n\n\n\nS1a\n19632\n14171\n19664\n26317\n\n\nS2a\n18908\n7905\n21183\n16496\n\n\nS3a\n30366\n5438\n32247\n-36092\n\n\n\nThis reaffirms to me - again - that the problem is likely agent number, and so the best course of action would be to run it with 50 million (like their repository) and potentially even 100 million (like their paper), although I’m not sure if our computer will be capable of it."
+ "text": "Please note: This is a template page and has not yet been completed, so all criteria are currently set as unmet (❌)\nThis page evaluates the extent to which the author-published research artefacts meet the criteria of badges related to reproducibility from various organisations and journals.\nCaveat: Please note that these criteria are based on available information about each badge online, and that we have likely differences in our procedure (e.g. allowed troubleshooting for execution and reproduction, not under tight time pressure to complete). Moreover, we focus only on reproduction of the discrete-event simulation, and not on other aspects of the article. We cannot guarantee that the badges below would have been awarded in practice by these journals."
},
{
- "objectID": "logbook/posts/2024_09_06/index.html#running-one-scenario-with-50-million",
- "href": "logbook/posts/2024_09_06/index.html#running-one-scenario-with-50-million",
- "title": "Day 7",
- "section": "11.12-11.18: Running one scenario with 50 million",
- "text": "11.12-11.18: Running one scenario with 50 million\nI remade a mini analysis file with a single scenario, so we can test how long the run takes without trying to run all at once."
+ "objectID": "evaluation/badges.html#criteria",
+ "href": "evaluation/badges.html#criteria",
+ "title": "Journal badges",
+ "section": "Criteria",
+ "text": "Criteria\n\n\nCode\nfrom IPython.display import display, Markdown\nimport numpy as np\nimport pandas as pd\n\n# Criteria and their definitions\ncriteria = {\n 'archive': 'Stored in a permanent archive that is publicly and openly accessible',\n 'id': 'Has a persistent identifier',\n 'license': 'Includes an open license',\n 'relevant': '''Artefacts are relevant to and contribute to the article's results''',\n 'complete': 'Complete set of materials shared (as would be needed to fully reproduce article)',\n 'structure': 'Artefacts are well structured/organised (e.g. to the extent that reuse and repurposing is facilitated, adhering to norms and standards of research community)',\n 'documentation_sufficient': 'Artefacts are sufficiently documented (i.e. to understand how it works, to enable it to be run, including package versions)',\n 'documentation_careful': 'Artefacts are carefully documented (more than sufficient - i.e. to the extent that reuse and repurposing is facilitated - e.g. changing parameters, reusing for own purpose)',\n # This criteria is kept seperate to documentation_careful, as it specifically requires a README file\n 'documentation_readme': 'Artefacts are clearly documented and accompanied by a README file with step-by-step instructions on how to reproduce results in the manuscript',\n 'execute': 'Scripts can be successfully executed',\n 'regenerated': 'Independent party regenerated results using the authors research artefacts',\n 'hour': 'Reproduced within approximately one hour (excluding compute time)',\n}\n\n# Evaluation for this study\n# TODO: Complete evaluate for each criteria\neval = pd.Series({\n 'archive': 0,\n 'id': 0,\n 'license': 0,\n 'relevant': 0,\n 'complete': 0,\n 'structure': 0,\n 'documentation_sufficient': 0,\n 'documentation_careful': 0,\n 'documentation_readme': 0,\n 'execute': 0,\n 'regenerated': 0,\n 'hour': 0,\n})\n\n# Get list of criteria met (True/False) overall\neval_list = list(eval)\n\n# Define function for creating the markdown formatted list of criteria met\ndef create_criteria_list(criteria_dict):\n '''\n Creates a string which contains a Markdown formatted list with icons to\n indicate whether each criteria was met\n\n Parameters:\n -----------\n criteria_dict : dict\n Dictionary where keys are the criteria (variable name) and values are\n Boolean (True/False of whether this study met the criteria)\n\n Returns:\n --------\n formatted_list : string\n Markdown formatted list\n '''\n callout_icon = {True: '✅',\n False: '❌'}\n # Create list with...\n formatted_list = ''.join([\n '* ' +\n callout_icon[eval[key]] + # Icon based on whether it met criteria\n ' ' +\n value + # Full text description of criteria\n '\\n' for key, value in criteria_dict.items()])\n return(formatted_list)\n\n# Define groups of criteria\ncriteria_share_how = ['archive', 'id', 'license']\ncriteria_share_what = ['relevant', 'complete']\ncriteria_doc_struc = ['structure', 'documentation_sufficient', 'documentation_careful', 'documentation_readme']\ncriteria_run = ['execute', 'regenerated', 'hour']\n\n# Create text section\ndisplay(Markdown(f'''\nTo assess whether the author's materials met the requirements of each badge, a list of criteria was produced. Between each badge (and between categories of badge), there is often alot of overlap in criteria.\n\nThis study met **{sum(eval_list)} of the {len(eval_list)}** unique criteria items. These were as follows:\n\nCriteria related to how artefacts are shared -\n\n{create_criteria_list({k: criteria[k] for k in criteria_share_how})}\n\nCriteria related to what artefacts are shared -\n\n{create_criteria_list({k: criteria[k] for k in criteria_share_what})}\n\nCriteria related to the structure and documentation of the artefacts -\n\n{create_criteria_list({k: criteria[k] for k in criteria_doc_struc})}\n\nCriteria related to running and reproducing results -\n\n{create_criteria_list({k: criteria[k] for k in criteria_run})}\n'''))\n\n\nTo assess whether the author’s materials met the requirements of each badge, a list of criteria was produced. Between each badge (and between categories of badge), there is often alot of overlap in criteria.\nThis study met 0 of the 12 unique criteria items. These were as follows:\nCriteria related to how artefacts are shared -\n\n❌ Stored in a permanent archive that is publicly and openly accessible\n❌ Has a persistent identifier\n❌ Includes an open license\n\nCriteria related to what artefacts are shared -\n\n❌ Artefacts are relevant to and contribute to the article’s results\n❌ Complete set of materials shared (as would be needed to fully reproduce article)\n\nCriteria related to the structure and documentation of the artefacts -\n\n❌ Artefacts are well structured/organised (e.g. to the extent that reuse and repurposing is facilitated, adhering to norms and standards of research community)\n❌ Artefacts are sufficiently documented (i.e. to understand how it works, to enable it to be run, including package versions)\n❌ Artefacts are carefully documented (more than sufficient - i.e. to the extent that reuse and repurposing is facilitated - e.g. changing parameters, reusing for own purpose)\n❌ Artefacts are clearly documented and accompanied by a README file with step-by-step instructions on how to reproduce results in the manuscript\n\nCriteria related to running and reproducing results -\n\n❌ Scripts can be successfully executed\n❌ Independent party regenerated results using the authors research artefacts\n❌ Reproduced within approximately one hour (excluding compute time)"
},
{
- "objectID": "logbook/posts/2024_09_06/index.html#appendix-6",
- "href": "logbook/posts/2024_09_06/index.html#appendix-6",
- "title": "Day 7",
- "section": "11.24-12.15: Appendix 6",
- "text": "11.24-12.15: Appendix 6\nAppendix 6 presents the clinical results from S1a 3 yrs. I couldn’t spot code for these figures in the closedcohort .Rmd files. I could see the package itself had plotting functions in figures.R and some of these appeared potentially relevant. This appeared to need to be called whilst running the model (rather than using the model outputs we had saved to csv).\nI made a new .Rmd file to run S1A, reduce from 1e6 to 50000 just while testing. Looking through the package, it wasn’t immediately clear to me how/when to call the export_figures() function. I tried adding it after all the S1A code. This had error:\nError in `cost_by_GOLD[, 2:5] <- (op_ex$cumul_cost_gold_ctime[, 2:5])`:\n! number of items to replace is not a multiple of replacement length\nBacktrace:\n 1. epicR::export_figures()\nI tried instead putting it before s1a’s terminate_session(), but the error persisted.\nI tried a different approach, to instead manually copy of the relevant code from export_figures() so I can altered it more closely and figure out why its breaking. Looking at the function, it was running the model itself and using:\ndata <- as.data.frame(Cget_all_events_matrix())\nop <- Cget_output()\nop_ex <- Cget_output_ex()\nI then realised that actually, the results we need might just be in the output S1.csv, and that these figures are more complicated than we require, so I instead looked to make them manually.\nThere are columns with Mild, Moderate, etc. counts but this needs to be transformed to per 1000. I wasn’t sure whether to:\n\nDivide by 1000 (as 1 million / 1000 = 1000)\nUse the “PY” results and multiply by 1000 (as that appears to be a bit like what they do in export_figures() although not super sure)\n\nEach of the two methods came out with different results. I searched in the repository but couldn’t find anything about MildPY or the others, so decided to go with the non-PY method, since I wasn’t actually sure what PY stood for, and whether it might actually be to do with “pack years” or “patient years”.\nDoing it manually, I made the first of the four figures, and found it actually looked very visually similar to the original, even if the number of exacerbations was lower. I suspected this was an issue with my adjustments. Looking at s1.csv, the Agents are actually lower than the base agents we set (apx 700k v.s. 1m). I tried using that to adjust - and that worked out much closer."
+ "objectID": "evaluation/badges.html#badges",
+ "href": "evaluation/badges.html#badges",
+ "title": "Journal badges",
+ "section": "Badges",
+ "text": "Badges\n\n\nCode\n# Full badge names\nbadge_names = {\n # Open objects\n 'open_niso': 'NISO \"Open Research Objects (ORO)\"',\n 'open_niso_all': 'NISO \"Open Research Objects - All (ORO-A)\"',\n 'open_acm': 'ACM \"Artifacts Available\"',\n 'open_cos': 'COS \"Open Code\"',\n 'open_ieee': 'IEEE \"Code Available\"',\n # Object review\n 'review_acm_functional': 'ACM \"Artifacts Evaluated - Functional\"',\n 'review_acm_reusable': 'ACM \"Artifacts Evaluated - Reusable\"',\n 'review_ieee': 'IEEE \"Code Reviewed\"',\n # Results reproduced\n 'reproduce_niso': 'NISO \"Results Reproduced (ROR-R)\"',\n 'reproduce_acm': 'ACM \"Results Reproduced\"',\n 'reproduce_ieee': 'IEEE \"Code Reproducible\"',\n 'reproduce_psy': 'Psychological Science \"Computational Reproducibility\"'\n}\n\n# Criteria required by each badge\nbadges = {\n # Open objects\n 'open_niso': ['archive', 'id', 'license'],\n 'open_niso_all': ['archive', 'id', 'license', 'complete'],\n 'open_acm': ['archive', 'id'],\n 'open_cos': ['archive', 'id', 'license', 'complete', 'documentation_sufficient'],\n 'open_ieee': ['complete'],\n # Object review\n 'review_acm_functional': ['documentation_sufficient', 'relevant', 'complete', 'execute'],\n 'review_acm_reusable': ['documentation_sufficient', 'documentation_careful', 'relevant', 'complete', 'execute', 'structure'],\n 'review_ieee': ['complete', 'execute'],\n # Results reproduced\n 'reproduce_niso': ['regenerated'],\n 'reproduce_acm': ['regenerated'],\n 'reproduce_ieee': ['regenerated'],\n 'reproduce_psy': ['regenerated', 'hour', 'structure', 'documentation_readme'],\n}\n\n# Identify which badges would be awarded based on criteria\n# Get list of badges met (True/False) overall\naward = {}\nfor badge in badges:\n award[badge] = all([eval[key] == 1 for key in badges[badge]])\naward_list = list(award.values())\n\n# Write introduction\n# Get list of badges met (True/False) by category\naward_open = [v for k,v in award.items() if k.startswith('open_')]\naward_review = [v for k,v in award.items() if k.startswith('review_')]\naward_reproduce = [v for k,v in award.items() if k.startswith('reproduce_')]\n\n# Create and display text for introduction\ndisplay(Markdown(f'''\nIn total, the original study met the criteria for **{sum(award_list)} of the {len(award_list)} badges**. This included:\n\n* **{sum(award_open)} of the {len(award_open)}** “open objects” badges\n* **{sum(award_review)} of the {len(award_review)}** “object review” badges\n* **{sum(award_reproduce)} of the {len(award_reproduce)}** “reproduced” badges\n'''))\n\n# Make function that creates collapsible callouts for each badge\ndef create_badge_callout(award_dict):\n '''\n Displays Markdown callouts created for each badge in the dictionary, showing\n whether the criteria for that badge was met.\n\n Parameters:\n -----------\n award_dict : dict\n Dictionary where key is badge (as variable name), and value is Boolean\n (whether badge is awarded)\n '''\n callout_appearance = {True: 'tip',\n False: 'warning'}\n callout_icon = {True: '✅',\n False: '❌'}\n callout_text = {True: 'Meets all criteria:',\n False: 'Does not meet all criteria:'}\n\n for key, value in award_dict.items():\n # Create Markdown list with...\n criteria_list = ''.join([\n '* ' +\n callout_icon[eval[k]] + # Icon based on whether it met criteria\n ' ' +\n criteria[k] + # Full text description of criteria\n '\\n' for k in badges[key]])\n # Create the callout and display it\n display(Markdown(f'''\n::: {{.callout-{callout_appearance[value]} appearance=\"minimal\" collapse=true}}\n\n## {callout_icon[value]} {badge_names[key]}\n\n{callout_text[value]}\n\n{criteria_list}\n:::\n'''))\n\n# Create badge functions with introductions and callouts\ndisplay(Markdown('''\n### \"Open objects\" badges\n\nThese badges relate to research artefacts being made openly available.\n'''))\ncreate_badge_callout({k: v for (k, v) in award.items() if k.startswith('open_')})\n\ndisplay(Markdown('''\n### \"Object review\" badges\n\nThese badges relate to the research artefacts being reviewed against criteria of the badge issuer.\n'''))\ncreate_badge_callout({k: v for (k, v) in award.items() if k.startswith('review_')})\n\ndisplay(Markdown('''\n### \"Reproduced\" badges\n\nThese badges relate to an independent party regenerating the reuslts of the article using the author objects.\n'''))\ncreate_badge_callout({k: v for (k, v) in award.items() if k.startswith('reproduce_')})\n\n\nIn total, the original study met the criteria for 0 of the 12 badges. This included:\n\n0 of the 5 “open objects” badges\n0 of the 3 “object review” badges\n0 of the 4 “reproduced” badges\n\n\n\n“Open objects” badges\nThese badges relate to research artefacts being made openly available.\n\n\n\n\n\n\n\n\n❌ NISO “Open Research Objects (ORO)”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Stored in a permanent archive that is publicly and openly accessible\n❌ Has a persistent identifier\n❌ Includes an open license\n\n\n\n\n\n\n\n\n\n\n\n\n❌ NISO “Open Research Objects - All (ORO-A)”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Stored in a permanent archive that is publicly and openly accessible\n❌ Has a persistent identifier\n❌ Includes an open license\n❌ Complete set of materials shared (as would be needed to fully reproduce article)\n\n\n\n\n\n\n\n\n\n\n\n\n❌ ACM “Artifacts Available”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Stored in a permanent archive that is publicly and openly accessible\n❌ Has a persistent identifier\n\n\n\n\n\n\n\n\n\n\n\n\n❌ COS “Open Code”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Stored in a permanent archive that is publicly and openly accessible\n❌ Has a persistent identifier\n❌ Includes an open license\n❌ Complete set of materials shared (as would be needed to fully reproduce article)\n❌ Artefacts are sufficiently documented (i.e. to understand how it works, to enable it to be run, including package versions)\n\n\n\n\n\n\n\n\n\n\n\n\n❌ IEEE “Code Available”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Complete set of materials shared (as would be needed to fully reproduce article)\n\n\n\n\n\n\n“Object review” badges\nThese badges relate to the research artefacts being reviewed against criteria of the badge issuer.\n\n\n\n\n\n\n\n\n❌ ACM “Artifacts Evaluated - Functional”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Artefacts are sufficiently documented (i.e. to understand how it works, to enable it to be run, including package versions)\n❌ Artefacts are relevant to and contribute to the article’s results\n❌ Complete set of materials shared (as would be needed to fully reproduce article)\n❌ Scripts can be successfully executed\n\n\n\n\n\n\n\n\n\n\n\n\n❌ ACM “Artifacts Evaluated - Reusable”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Artefacts are sufficiently documented (i.e. to understand how it works, to enable it to be run, including package versions)\n❌ Artefacts are carefully documented (more than sufficient - i.e. to the extent that reuse and repurposing is facilitated - e.g. changing parameters, reusing for own purpose)\n❌ Artefacts are relevant to and contribute to the article’s results\n❌ Complete set of materials shared (as would be needed to fully reproduce article)\n❌ Scripts can be successfully executed\n❌ Artefacts are well structured/organised (e.g. to the extent that reuse and repurposing is facilitated, adhering to norms and standards of research community)\n\n\n\n\n\n\n\n\n\n\n\n\n❌ IEEE “Code Reviewed”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Complete set of materials shared (as would be needed to fully reproduce article)\n❌ Scripts can be successfully executed\n\n\n\n\n\n\n“Reproduced” badges\nThese badges relate to an independent party regenerating the reuslts of the article using the author objects.\n\n\n\n\n\n\n\n\n❌ NISO “Results Reproduced (ROR-R)”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Independent party regenerated results using the authors research artefacts\n\n\n\n\n\n\n\n\n\n\n\n\n❌ ACM “Results Reproduced”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Independent party regenerated results using the authors research artefacts\n\n\n\n\n\n\n\n\n\n\n\n\n❌ IEEE “Code Reproducible”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Independent party regenerated results using the authors research artefacts\n\n\n\n\n\n\n\n\n\n\n\n\n❌ Psychological Science “Computational Reproducibility”\n\n\n\n\n\nDoes not meet all criteria:\n\n❌ Independent party regenerated results using the authors research artefacts\n❌ Reproduced within approximately one hour (excluding compute time)\n❌ Artefacts are well structured/organised (e.g. to the extent that reuse and repurposing is facilitated, adhering to norms and standards of research community)\n❌ Artefacts are clearly documented and accompanied by a README file with step-by-step instructions on how to reproduce results in the manuscript"
},
{
- "objectID": "logbook/posts/2024_09_06/index.html#result-from-50-million",
- "href": "logbook/posts/2024_09_06/index.html#result-from-50-million",
- "title": "Day 7",
- "section": "13.15-13.20, 13.25-13.28: Result from 50 million",
- "text": "13.15-13.20, 13.25-13.28: Result from 50 million\nTo run S1NoCD with 50 million base agents, it took 1.644506 hours (1 hour 38 minutes)\nThere’s 24 scenarios, so running those with 50 million would take apx. 1 day 15 hours 28 minutes. With 100 million (as in the paper), it would be at least double that (so at least 3 days 7 hours).\nThat’s repeated a total of 8 times, for the reference case and then for the scenario analyses (so, 26 days 8 hours).\nIt appears we may therefore need to explore access to a HPC.\nThe result was:\n\nimport pandas as pd\n\npd.read_csv('s1NoCD_50mil.csv')\n\n\n\n\n\n\n\n\nScenario\nAgents\nPatientYears\nCopdPYs\nNCaseDetections\nDiagnosedPYs\nOverdiagnosedPYs\nSABA\nLAMA\nLAMALABA\n...\nNoCOPD\nGOLD1\nGOLD2\nGOLD3\nGOLD4\nCost\nCostpAgent\nQALY\nQALYpAgent\nNMB\n\n\n\n\n0\nS1NoCD\n37196486\n6.259536e+08\n7.111802e+07\n190966216\n1.316646e+07\n13371458\n0.017152\n0.135457\n0.151286\n...\n527427474\n28845970\n30727603\n6872520\n1191666\n7.986489e+10\n2147.108537\n4.666148e+08\n12.544592\n625082.515224\n\n\n\n\n1 rows × 29 columns"
+ "objectID": "evaluation/badges.html#sources",
+ "href": "evaluation/badges.html#sources",
+ "title": "Journal badges",
+ "section": "Sources",
+ "text": "Sources\nNational Information Standards Organisation (NISO) (NISO Reproducibility Badging and Definitions Working Group (2021))\n\n“Open Research Objects (ORO)”\n“Open Research Objects - All (ORO-A)”\n“Results Reproduced (ROR-R)”\n\nAssociation for Computing Machinery (ACM) (Association for Computing Machinery (ACM) (2020))\n\n“Artifacts Available”\n“Artifacts Evaluated - Functional”\n“Artifacts Evaluated - Resuable”\n“Results Reproduced”\n\nCenter for Open Science (COS) (Blohowiak et al. (2023))\n\n“Open Code”\n\nInstitute of Electrical and Electronics Engineers (IEEE) (Institute of Electrical and Electronics Engineers (IEEE) (n.d.))\n\n“Code Available”\n“Code Reviewed”\n“Code Reproducible”\n\nPsychological Science (Hardwicke and Vazire (2023) and Association for Psychological Science (APS) (2023))\n\n“Computational Reproducibility”"
},
{
- "objectID": "logbook/posts/2024_09_06/index.html#returning-to-appendix-6",
- "href": "logbook/posts/2024_09_06/index.html#returning-to-appendix-6",
- "title": "Day 7",
- "section": "13.30-14.42: Returning to appendix 6",
- "text": "13.30-14.42: Returning to appendix 6\nI created the four figures. For the fourth, the transformation I’d done to make it reflect 1000 didn’t seem to make sense for that one, but I wasn’t quite sure what the data was or how to transform it.\nI tried multiplifying by 1000, and that got results about half of the original. I tried multiplifying by 2000 - although this is just a total guess at this point.\nLooking across the results, I would say that I am satisified this matches the original for all except one bar: SABA. The SABA bars for patient years on treatment are much lower than in the original figure.\n\n\n\nAppendix 6 attempt\n\n\nSABA is one of the treatments: short-acting beta-agonists.\nI went to check my results, then realised there was an adjusted table of clinical results, and that I should try using that instead of S1, which is the more “raw” model results. Notably, that table just seems to have the columns used in the appendix plot - and they are adjusted to be per agent, so I can confidently just multiply by 1000 to get results for 1000 people.\n\n\n\n\n\n\nReflection\n\n\n\nIt’s great that this was provided, and makes it much easier. This is partly my fault for not noticing that I should be using that table. However, as for the other parts of this reproduction, this might have been supported by more guidance on which tables produced which figures, or inclusion of code to produce that figure in the paper. And, I wasn’t sure what to multiply for these either in the end (see below).\n\n\nI realised that multiplying by 1000 works with pAgent columns, but not for the pPYAll. It seems I’d need to multiply by about 16,000.\nI experimented with the others, but these were just guesses of what to multiply by, and I still found SABA results to be too low.\n\n\n\nAppendix 6 attempt 2"
+ "objectID": "logbook/posts/2024_10_14/index.html",
+ "href": "logbook/posts/2024_10_14/index.html",
+ "title": "Day 13",
+ "section": "",
+ "text": "Note\n\n\n\nReviewed results from 50 million (as in their repository) and decided to run 100 million (as in paper). Total time used: 18h 31m (46.3%)"
},
{
- "objectID": "logbook/posts/2024_09_06/index.html#troubleshooting-appendix-6",
- "href": "logbook/posts/2024_09_06/index.html#troubleshooting-appendix-6",
- "title": "Day 7",
- "section": "15.10-15.40, 15.47-16.06: Troubleshooting appendix 6",
- "text": "15.10-15.40, 15.47-16.06: Troubleshooting appendix 6\nLooking at their results from 50 million, they also have a low SABA\nComparing to other SABA results (with LAMA for context):\n\n\n\nScenario\nMy SABA (LAMA)\nTheir 50 million SABA (LAMA)\n\n\n\n\nS1NoCDAvg\n0.019 (0.136)\n0.019 (0.135)\n\n\nS1a\n0.026 (0.159)\n0.026 (0.159)\n\n\n\nGiven how similar these results are, I’m suspecting the issue might not be with the result, but with my transformation, as I just guessed at multiplying it all by 2000 to get similar-ish results for some bars. Looking at Case_Detecetion_Results.Rmd, SABAAll was created as follows:\nProppPatientYears = ifelse(Scenario==\"S2a\" | Scenario==\"S2NoCD\" | Scenario==\"S3NoCD\" | Scenario==\"S3a\" | Scenario==\"S3b\" |\n Scenario==\"S3c\" | Scenario==\"S3d\", PatientYears/sall[sall$Scenario==\"S1NoCDAvg\",\"PatientYears\"], 1),\n # SABA\nSABAExcluded = ifelse(Scenario==\"S1NoCDAvg\" | Scenario==\"S1a\" | Scenario==\"S1b\" | Scenario==\"S1c\", 0,\n ifelse(Scenario==\"S2a\" | Scenario==\"S2NoCD\", \n (sall[sall$Scenario==\"S1NoCDAvg\",\"SABATotal\"]- sall[sall$Scenario==\"S2NoCD\",\"SABATotal\"])/\n (sall[sall$Scenario==\"S1NoCDAvg\",\"PatientYears\"]- sall[sall$Scenario==\"S2NoCD\",\"PatientYears\"]),\n ifelse(Scenario==\"S3NoCD\" | Scenario==\"S3a\" | Scenario==\"S3b\" | Scenario==\"S3c\" | Scenario==\"S3d\", \n (sall[sall$Scenario==\"S1NoCDAvg\",\"SABATotal\"]- sall[sall$Scenario==\"S3NoCD\",\"SABATotal\"])/ (sall[sall$Scenario==\"S1NoCDAvg\",\"PatientYears\"]- sall[sall$Scenario==\"S3NoCD\",\"PatientYears\"]),999))),\n SABAAll = ProppPatientYears*SABA + (1-ProppPatientYears)*SABAExcluded,\nProppPatientYears adjusts the years for scenarios 2 and 3 by dividing them by the patient years of S1NoCDAvg. For scenario 1, it just sets this value to 1. This appears to be some adjustment of scenarios 2 and 3 so they compare against a single reference population. However, it remains unclear to me quite what the units are. SABAAll works out the same for scenarios 1 as SABA.\n\n\n\n\n\n\nReflection\n\n\n\nFound these transformations tricky to work out. I think part of the difficulty is that I’m really not quite sure what these columns are, and there isn’t documentation/comments to explain it, so I then don’t know how to transform them to match up with the paper.\n\n\nI then tried looking at the COPD DiagnosedpPY results, since those are the same thing just not broken down by treatment, and since clinical gives us adjusted versions of these but we also have “raw” versions from s1.\nclin_diag <- clinical %>%\n filter(Scenario %in% c(\"S1NoCDAvg\", \"S1a\")) %>%\n select(Scenario, DiagnosedpPYAll)\n\ns1_diag <- s1 %>%\n filter(Scenario %in% c(\"S1NoCDAvg\", \"S1a\")) %>%\n select(Scenario, Agents, DiagnosedPYs) %>%\n mutate(AdjDiagnosedPYs = DiagnosedPYs / (Agents/1000))\n\ndiag_investigate <- merge(clin_diag, s1_diag) %>%\n mutate(AdjDivideByPYAll = AdjDiagnosedPYs / DiagnosedpPYAll)\n\nwrite.csv(diag_investigate, \"../../logbook/posts/2024_09_06/diag_investigate.csv\")\n\npd.read_csv('diag_investigate.csv')\n\n\n\n\n\n\n\n\nUnnamed: 0\nScenario\nDiagnosedpPYAll\nAgents\nDiagnosedPYs\nAdjDiagnosedPYs\nAdjDivideByPYAll\n\n\n\n\n0\n1\nS1a\n0.462874\n743762\n659857.061426\n887.188457\n1916.696154\n\n\n1\n2\nS1NoCDAvg\n0.186304\n743240\n264401.460862\n355.741700\n1909.470132\n\n\n\n\n\n\n\nYou can convert between the two by multiplying by about 1910, but that doesn’t work for SABA.\nInstead, I tried looking back at export_figures() and the paper each again, but struggled to spot any answers there. I then tried looking at the first EpicR paper, “Development and Validation of the Evaluation Platform in COPD (EPIC): A Population-Based Outcomes Model of COPD for Canada”, but couldn’t find anything to explain it there either.\nI then looked to the main epicR repo, and found it has some documentation in the docs/ folder. I opened and read the html files in docs/articles/, but struggled to find answers here too."
+ "objectID": "logbook/posts/2024_10_14/index.html#review-results-from-50-million-agents-and-set-to-run-with-100-million",
+ "href": "logbook/posts/2024_10_14/index.html#review-results-from-50-million-agents-and-set-to-run-with-100-million",
+ "title": "Day 13",
+ "section": "09.16-09.19, 09.36-10.22: Review results from 50 million agents and set to run with 100 million",
+ "text": "09.16-09.19, 09.36-10.22: Review results from 50 million agents and set to run with 100 million\nI looked over them, comparing them to the article.\nI had to remind myself of what appendix 6 v1 vs v2 were, recalling that I’d remained uncertain on whether the implementation of this figure was correct. Version 2 was the “more correct”, so I removed version 1. What had remained uncertain for version 2 was how to adjust the results. I had multiplied by 2000, and found that resulted in all bars matching except SABA, and this still remained the case.\nFor table 3, I add back in the code for comparing against the original.\nThe results from 50 million are closer to the original.\nRun times:\n\nCase_Detection_Results.md - 14.13 hours - 14 hours 8 minutes\nCase_Detection_Results_5yrs.md - 15.38 hours - 15 hours 23 minutes\n\nAlthough note, these were run in parallel to each other, so total time required (with our machine) was 15 hours 23 minutes.\nTable 3:\n\nimport pandas as pd\npd.set_option('display.max_columns', None)\n\npd.read_csv('50m_tab3.csv')\n\n\n\n\n\n\n\n\nScenario\nTesting interval\nCosts per patient\nQALYs per patient\nICER\nINMB (ranking)\n\n\n\n\n0\nS0: No case detection\nNaN\n$2149\n12.545\nNaN\nNaN\n\n\n1\n(S1a) CDQ ≥ 17 points\n3 years\n$2440\n12.562\n17669.0\n533(1)\n\n\n2\n(S1a) CDQ ≥ 17 points\n5 years\n$2355\n12.556\n19073.0\n335(2)\n\n\n3\n(S1b) Screening spirometry\n3 years\n$2362\n12.553\n26497.0\n189(5)\n\n\n4\n(S1b) Screening spirometry\n5 years\n$2298\n12.552\n22920.0\n177(8)\n\n\n5\n(S1c) CDQ + screening spirometry\n3 years\n$2387\n12.550\n46445.0\n18(16)\n\n\n6\n(S1c) CDQ + screening spirometry\n5 years\n$2314\n12.551\n27818.0\n132(9)\n\n\n7\n(S2a) Screening spirometry\n3 years\n$2286\n12.553\n18082.0\n242(4)\n\n\n8\n(S2a) Screening spirometry\n5 years\n$2241\n12.549\n27650.0\n74(11)\n\n\n9\n(S3a) CDQ ≥ 19.5 points\n3 years\n$2234\n12.548\n28714.0\n63(14)\n\n\n10\n(S3a) CDQ ≥ 19.5 points\n5 years\n$2203\n12.548\n22107.0\n68(12)\n\n\n11\n(S3b) CDQ ≥ 16.5 points\n3 years\n$2289\n12.552\n21354.0\n188(6)\n\n\n12\n(S3b) CDQ ≥ 16.5 points\n5 years\n$2245\n12.552\n13503.0\n260(3)\n\n\n13\n(S3c) Screening spirometry\n3 years\n$2253\n12.549\n28161.0\n81(10)\n\n\n14\n(S3c) Screening spirometry\n5 years\n$2220\n12.550\n14224.0\n178(7)\n\n\n15\n(S3d) CDQ + screening spirometry\n3 years\n$2262\n12.549\n33377.0\n56(15)\n\n\n16\n(S3d) CDQ + screening spirometry\n5 years\n$2223\n12.548\n26612.0\n65(13)\n\n\n\n\n\n\n\n\npd.read_csv('50m_tab3_compare_to_original.csv')\n\n\n\n\n\n\n\n\nScenario\nInterval\nCostpAgent_paper\nCostpAgentAll\nCostpAgentAll_repo\nCostMyDiff\nCostMyPerc\nCostRepoDiff\nCostRepoPerc\nQALYpAgent_paper\nQALYpAgentAll\nQALYpAgentAll_repo\nQALYMyDiff\nQALYMyPerc\nQALYRepoDiff\nQALYRepoPerc\nICER_paper\nICERAdj\nICERAdj_repo\nICERMyDiff\nICERMyPerc\nICERRepoDiff\nICERRepoPerc\nIncrementalNMB_paper\nINMB\nINMB_repo\nINMBMyDiff\nINMBMyPerc\nINMBRepoDiff\nINMBRepoPerc\n\n\n\n\n0\nS0\nNaN\n2151\n2149\n2151\n2\n0.1%\n0\n0.0%\n12.546\n12.545\n12.545\n0.001\n0%\n0.001\n0%\nNaN\nNaN\nNaN\nNaN\nNaN\nNaN\nNaN\nNaN\nNaN\n0\nNaN\nNaN\nNaN\nNaN\n\n\n1\nS1a\n3 years\n2438\n2440\n2439\n-2\n-0.1%\n-1\n0.0%\n12.560\n12.562\n12.560\n-0.002\n0%\n0.000\n0%\n19632.0\n17669.0\n19665.0\n1963.0\n11.1%\n-33.0\n-0.2%\n444.0\n533.0\n446\n-89.0\n-16.7%\n-2.0\n-0.40%\n\n\n2\nS1a\n5 years\n2356\n2355\n2355\n1\n0.0%\n1\n0.0%\n12.556\n12.556\n12.557\n0.000\n0%\n-0.001\n0%\n19847.0\n19073.0\n16600.0\n774.0\n4.1%\n3247.0\n19.6%\n312.0\n335.0\n411\n-23.0\n-6.9%\n-99.0\n-24.10%\n\n\n3\nS1b\n3 years\n2363\n2362\n2365\n1\n0.0%\n-2\n-0.1%\n12.554\n12.553\n12.552\n0.001\n0%\n0.002\n0%\n25894.0\n26497.0\n31076.0\n-603.0\n-2.3%\n-5182.0\n-16.7%\n198.0\n189.0\n131\n9.0\n4.8%\n67.0\n51.10%\n\n\n4\nS1b\n5 years\n2296\n2298\n2298\n-2\n-0.1%\n-2\n-0.1%\n12.552\n12.552\n12.551\n0.000\n0%\n0.001\n0%\n23187.0\n22920.0\n25501.0\n267.0\n1.2%\n-2314.0\n-9.1%\n168.0\n177.0\n142\n-9.0\n-5.1%\n26.0\n18.30%\n\n\n5\nS1c\n3 years\n2386\n2387\n2386\n-1\n0.0%\n0\n0.0%\n12.551\n12.550\n12.551\n0.001\n0%\n0.000\n0%\n46956.0\n46445.0\n42697.0\n511.0\n1.1%\n4259.0\n10.0%\n15.0\n18.0\n40\n-3.0\n-16.7%\n-25.0\n-62.50%\n\n\n6\nS1c\n5 years\n2313\n2314\n2311\n-1\n0.0%\n2\n0.1%\n12.550\n12.551\n12.551\n-0.001\n0%\n-0.001\n0%\n38673.0\n27818.0\n29600.0\n10855.0\n39.0%\n9073.0\n30.7%\n47.0\n132.0\n110\n-85.0\n-64.4%\n-63.0\n-57.30%\n\n\n7\nS2a\n3 years\n2286\n2286\n2287\n0\n0.0%\n-1\n0.0%\n12.553\n12.553\n12.552\n0.000\n0%\n0.001\n0%\n18908.0\n18082.0\n21184.0\n826.0\n4.6%\n-2276.0\n-10.7%\n223.0\n242.0\n186\n-19.0\n-7.9%\n37.0\n19.90%\n\n\n8\nS2a\n5 years\n2246\n2241\n2245\n5\n0.2%\n1\n0.0%\n12.551\n12.549\n12.550\n0.002\n0%\n0.001\n0%\n17514.0\n27650.0\n20394.0\n-10136.0\n-36.7%\n-2880.0\n-14.1%\n176.0\n74.0\n137\n102.0\n137.8%\n39.0\n28.50%\n\n\n9\nS3a\n3 years\n2234\n2234\n2232\n0\n0.0%\n2\n0.1%\n12.548\n12.548\n12.548\n0.000\n0%\n0.000\n0%\n30366.0\n28714.0\n32247.0\n1652.0\n5.8%\n-1881.0\n-5.8%\n54.0\n63.0\n45\n-9.0\n-14.3%\n9.0\n20.00%\n\n\n10\nS3a\n5 years\n2207\n2203\n2207\n4\n0.2%\n0\n0.0%\n12.548\n12.548\n12.548\n0.000\n0%\n0.000\n0%\n22636.0\n22107.0\n22694.0\n529.0\n2.4%\n-58.0\n-0.3%\n68.0\n68.0\n67\n0.0\n0.0%\n1.0\n1.50%\n\n\n11\nS3b\n3 years\n2292\n2289\n2289\n3\n0.1%\n3\n0.1%\n12.553\n12.552\n12.553\n0.001\n0%\n0.000\n0%\n18438.0\n21354.0\n18157.0\n-2916.0\n-13.7%\n281.0\n1.5%\n241.0\n188.0\n243\n53.0\n28.2%\n-2.0\n-0.80%\n\n\n12\nS3b\n5 years\n2250\n2245\n2248\n5\n0.2%\n2\n0.1%\n12.552\n12.552\n12.551\n0.000\n0%\n0.001\n0%\n16251.0\n13503.0\n17325.0\n2748.0\n20.4%\n-1074.0\n-6.2%\n206.0\n260.0\n183\n-54.0\n-20.8%\n23.0\n12.60%\n\n\n13\nS3c\n3 years\n2256\n2253\n2255\n3\n0.1%\n1\n0.0%\n12.550\n12.549\n12.550\n0.001\n0%\n0.000\n0%\n23972.0\n28161.0\n20700.0\n-4189.0\n-14.9%\n3272.0\n15.8%\n114.0\n81.0\n148\n33.0\n40.7%\n-34.0\n-23.00%\n\n\n14\nS3c\n5 years\n2224\n2220\n2221\n4\n0.2%\n3\n0.1%\n12.549\n12.550\n12.549\n-0.001\n0%\n0.000\n0%\n20278.0\n14224.0\n19250.0\n6054.0\n42.6%\n1028.0\n5.3%\n107.0\n178.0\n113\n-71.0\n-39.9%\n-6.0\n-5.30%\n\n\n15\nS3d\n3 years\n2263\n2262\n2261\n1\n0.0%\n2\n0.1%\n12.549\n12.549\n12.550\n0.000\n0%\n-0.001\n0%\n28245.0\n33377.0\n23462.0\n-5132.0\n-15.4%\n4783.0\n20.4%\n86.0\n56.0\n124\n30.0\n53.6%\n-38.0\n-30.60%\n\n\n16\nS3d\n5 years\n2227\n2223\n2226\n4\n0.2%\n1\n0.0%\n12.548\n12.548\n12.548\n0.000\n0%\n0.000\n0%\n27591.0\n26612.0\n26517.0\n979.0\n3.7%\n1074.0\n4.1%\n62.0\n65.0\n67\n-3.0\n-4.6%\n-5.0\n-7.50%\n\n\n\n\n\n\n\nLooking at each of the columns…\n\nCosts per patient - max difference from paper is 5 (0.2%) (e.g. $2250 vs $2245 for S3b 5 years). With lower number of agents, max difference was 23 (1%)\nQALYs per patient - max difference from paper is 0.002 (0%) (e.g. 12.562 vs 12.560). With lower agent numbers, max difference was 0.01 (0.1%)\nICER - max difference is 10,136 (36.7%) (mine 27,650 v.s. paper 17,514) for S2A 5 years. Biggest percentage difference was 42.6% (14,224 vs 20,278). With lower agent numbers, my biggest absolute difference was 93,297, and biggest percentage difference was 458%.\nINMB - max difference was 102 (137.8%) (mine 74 vs paper 176 for S2A 5 years).\n\nDue to the large difference in ICERs and INMB, I would unfortunately say this is not reproduced.\nFigure 3:\n\nThis looks fairly similar, at a glance (unlike with higher agent numbers which was wildly different). I think I would argue this to be reproduced, as the points are pretty close to where they were in the original, although given the tiny scale is this working to, this would definitely require some consensus and second opinion.\nAppendix 6:\n\nThis hasn’t changed much from running with 50 million agents v.s. lower numbers. The same issue remains - that everything is spot on except SABA - and so, unfortunately, still not reproduced. My result is very similar to the result from their repository when they ran with 50 million.\nHaving looked over these, I think it would be worth running this again with 100 million (as in the paper), just to be sure, in case that does happen to resolve the remaining differences in Table 3 (big difference) and Figure 3 (smaller difference) (although unfortunately I suspect the difference might remain for Appendix 6 as that didn’t change with agent numbers)."
},
{
- "objectID": "logbook/posts/2024_09_06/index.html#timings",
- "href": "logbook/posts/2024_09_06/index.html#timings",
- "title": "Day 7",
+ "objectID": "logbook/posts/2024_10_14/index.html#timings",
+ "href": "logbook/posts/2024_10_14/index.html#timings",
+ "title": "Day 13",
"section": "Timings",
- "text": "Timings\n\nimport sys\nsys.path.append('../')\nfrom timings import calculate_times\n\n# Minutes used prior to today\nused_to_date = 588\n\n# Times from today\ntimes = [\n ('09.36', '09.48'),\n ('09.49', '10.40'),\n ('11.00', '11.11'),\n ('11.12', '11.18'),\n ('11.24', '12.15'),\n ('13.15', '13.20'),\n ('13.25', '13.28'),\n ('13.30', '14.42'),\n ('15.10', '15.40'),\n ('15.47', '16.06')]\n\ncalculate_times(used_to_date, times)\n\nTime spent today: 260m, or 4h 20m\nTotal used to date: 848m, or 14h 8m\nTime remaining: 1552m, or 25h 52m\nUsed 35.3% of 40 hours max"
+ "text": "Timings\n\nimport sys\nsys.path.append('../')\nfrom timings import calculate_times\n\n# Minutes used prior to today\nused_to_date = 1062\n\n# Times from today\ntimes = [\n ('09.16', '09.19'),\n ('09.36', '10.22')]\n\ncalculate_times(used_to_date, times)\n\nTime spent today: 49m, or 0h 49m\nTotal used to date: 1111m, or 18h 31m\nTime remaining: 1289m, or 21h 29m\nUsed 46.3% of 40 hours max"
},
{
"objectID": "quarto_site/license.html",
diff --git a/site_libs/quarto-contrib/glightbox/glightbox.min.css b/site_libs/quarto-contrib/glightbox/glightbox.min.css
new file mode 100644
index 0000000..3c9ff87
--- /dev/null
+++ b/site_libs/quarto-contrib/glightbox/glightbox.min.css
@@ -0,0 +1 @@
+.glightbox-container{width:100%;height:100%;position:fixed;top:0;left:0;z-index:999999!important;overflow:hidden;-ms-touch-action:none;touch-action:none;-webkit-text-size-adjust:100%;-moz-text-size-adjust:100%;-ms-text-size-adjust:100%;text-size-adjust:100%;-webkit-backface-visibility:hidden;backface-visibility:hidden;outline:0}.glightbox-container.inactive{display:none}.glightbox-container .gcontainer{position:relative;width:100%;height:100%;z-index:9999;overflow:hidden}.glightbox-container .gslider{-webkit-transition:-webkit-transform .4s ease;transition:-webkit-transform .4s ease;transition:transform .4s ease;transition:transform .4s ease,-webkit-transform .4s ease;height:100%;left:0;top:0;width:100%;position:relative;overflow:hidden;display:-webkit-box!important;display:-ms-flexbox!important;display:flex!important;-webkit-box-pack:center;-ms-flex-pack:center;justify-content:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-transform:translate3d(0,0,0);transform:translate3d(0,0,0)}.glightbox-container .gslide{width:100%;position:absolute;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none;display:-webkit-box;display:-ms-flexbox;display:flex;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-pack:center;-ms-flex-pack:center;justify-content:center;opacity:0}.glightbox-container .gslide.current{opacity:1;z-index:99999;position:relative}.glightbox-container .gslide.prev{opacity:1;z-index:9999}.glightbox-container .gslide-inner-content{width:100%}.glightbox-container .ginner-container{position:relative;width:100%;display:-webkit-box;display:-ms-flexbox;display:flex;-webkit-box-pack:center;-ms-flex-pack:center;justify-content:center;-webkit-box-orient:vertical;-webkit-box-direction:normal;-ms-flex-direction:column;flex-direction:column;max-width:100%;margin:auto;height:100vh}.glightbox-container .ginner-container.gvideo-container{width:100%}.glightbox-container .ginner-container.desc-bottom,.glightbox-container .ginner-container.desc-top{-webkit-box-orient:vertical;-webkit-box-direction:normal;-ms-flex-direction:column;flex-direction:column}.glightbox-container .ginner-container.desc-left,.glightbox-container .ginner-container.desc-right{max-width:100%!important}.gslide iframe,.gslide video{outline:0!important;border:none;min-height:165px;-webkit-overflow-scrolling:touch;-ms-touch-action:auto;touch-action:auto}.gslide:not(.current){pointer-events:none}.gslide-image{-webkit-box-align:center;-ms-flex-align:center;align-items:center}.gslide-image img{max-height:100vh;display:block;padding:0;float:none;outline:0;border:none;-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none;max-width:100vw;width:auto;height:auto;-o-object-fit:cover;object-fit:cover;-ms-touch-action:none;touch-action:none;margin:auto;min-width:200px}.desc-bottom .gslide-image img,.desc-top .gslide-image img{width:auto}.desc-left .gslide-image img,.desc-right .gslide-image img{width:auto;max-width:100%}.gslide-image img.zoomable{position:relative}.gslide-image img.dragging{cursor:-webkit-grabbing!important;cursor:grabbing!important;-webkit-transition:none;transition:none}.gslide-video{position:relative;max-width:100vh;width:100%!important}.gslide-video .plyr__poster-enabled.plyr--loading .plyr__poster{display:none}.gslide-video .gvideo-wrapper{width:100%;margin:auto}.gslide-video::before{content:'';position:absolute;width:100%;height:100%;background:rgba(255,0,0,.34);display:none}.gslide-video.playing::before{display:none}.gslide-video.fullscreen{max-width:100%!important;min-width:100%;height:75vh}.gslide-video.fullscreen video{max-width:100%!important;width:100%!important}.gslide-inline{background:#fff;text-align:left;max-height:calc(100vh - 40px);overflow:auto;max-width:100%;margin:auto}.gslide-inline .ginlined-content{padding:20px;width:100%}.gslide-inline .dragging{cursor:-webkit-grabbing!important;cursor:grabbing!important;-webkit-transition:none;transition:none}.ginlined-content{overflow:auto;display:block!important;opacity:1}.gslide-external{display:-webkit-box;display:-ms-flexbox;display:flex;width:100%;min-width:100%;background:#fff;padding:0;overflow:auto;max-height:75vh;height:100%}.gslide-media{display:-webkit-box;display:-ms-flexbox;display:flex;width:auto}.zoomed .gslide-media{-webkit-box-shadow:none!important;box-shadow:none!important}.desc-bottom .gslide-media,.desc-top .gslide-media{margin:0 auto;-webkit-box-orient:vertical;-webkit-box-direction:normal;-ms-flex-direction:column;flex-direction:column}.gslide-description{position:relative;-webkit-box-flex:1;-ms-flex:1 0 100%;flex:1 0 100%}.gslide-description.description-left,.gslide-description.description-right{max-width:100%}.gslide-description.description-bottom,.gslide-description.description-top{margin:0 auto;width:100%}.gslide-description p{margin-bottom:12px}.gslide-description p:last-child{margin-bottom:0}.zoomed .gslide-description{display:none}.glightbox-button-hidden{display:none}.glightbox-mobile .glightbox-container .gslide-description{height:auto!important;width:100%;position:absolute;bottom:0;padding:19px 11px;max-width:100vw!important;-webkit-box-ordinal-group:3!important;-ms-flex-order:2!important;order:2!important;max-height:78vh;overflow:auto!important;background:-webkit-gradient(linear,left top,left bottom,from(rgba(0,0,0,0)),to(rgba(0,0,0,.75)));background:linear-gradient(to bottom,rgba(0,0,0,0) 0,rgba(0,0,0,.75) 100%);-webkit-transition:opacity .3s linear;transition:opacity .3s linear;padding-bottom:50px}.glightbox-mobile .glightbox-container .gslide-title{color:#fff;font-size:1em}.glightbox-mobile .glightbox-container .gslide-desc{color:#a1a1a1}.glightbox-mobile .glightbox-container .gslide-desc a{color:#fff;font-weight:700}.glightbox-mobile .glightbox-container .gslide-desc *{color:inherit}.glightbox-mobile .glightbox-container .gslide-desc .desc-more{color:#fff;opacity:.4}.gdesc-open .gslide-media{-webkit-transition:opacity .5s ease;transition:opacity .5s ease;opacity:.4}.gdesc-open .gdesc-inner{padding-bottom:30px}.gdesc-closed .gslide-media{-webkit-transition:opacity .5s ease;transition:opacity .5s ease;opacity:1}.greset{-webkit-transition:all .3s ease;transition:all .3s ease}.gabsolute{position:absolute}.grelative{position:relative}.glightbox-desc{display:none!important}.glightbox-open{overflow:hidden}.gloader{height:25px;width:25px;-webkit-animation:lightboxLoader .8s infinite linear;animation:lightboxLoader .8s infinite linear;border:2px solid #fff;border-right-color:transparent;border-radius:50%;position:absolute;display:block;z-index:9999;left:0;right:0;margin:0 auto;top:47%}.goverlay{width:100%;height:calc(100vh + 1px);position:fixed;top:-1px;left:0;background:#000;will-change:opacity}.glightbox-mobile .goverlay{background:#000}.gclose,.gnext,.gprev{z-index:99999;cursor:pointer;width:26px;height:44px;border:none;display:-webkit-box;display:-ms-flexbox;display:flex;-webkit-box-pack:center;-ms-flex-pack:center;justify-content:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;-webkit-box-orient:vertical;-webkit-box-direction:normal;-ms-flex-direction:column;flex-direction:column}.gclose svg,.gnext svg,.gprev svg{display:block;width:25px;height:auto;margin:0;padding:0}.gclose.disabled,.gnext.disabled,.gprev.disabled{opacity:.1}.gclose .garrow,.gnext .garrow,.gprev .garrow{stroke:#fff}.gbtn.focused{outline:2px solid #0f3d81}iframe.wait-autoplay{opacity:0}.glightbox-closing .gclose,.glightbox-closing .gnext,.glightbox-closing .gprev{opacity:0!important}.glightbox-clean .gslide-description{background:#fff}.glightbox-clean .gdesc-inner{padding:22px 20px}.glightbox-clean .gslide-title{font-size:1em;font-weight:400;font-family:arial;color:#000;margin-bottom:19px;line-height:1.4em}.glightbox-clean .gslide-desc{font-size:.86em;margin-bottom:0;font-family:arial;line-height:1.4em}.glightbox-clean .gslide-video{background:#000}.glightbox-clean .gclose,.glightbox-clean .gnext,.glightbox-clean .gprev{background-color:rgba(0,0,0,.75);border-radius:4px}.glightbox-clean .gclose path,.glightbox-clean .gnext path,.glightbox-clean .gprev path{fill:#fff}.glightbox-clean .gprev{position:absolute;top:-100%;left:30px;width:40px;height:50px}.glightbox-clean .gnext{position:absolute;top:-100%;right:30px;width:40px;height:50px}.glightbox-clean .gclose{width:35px;height:35px;top:15px;right:10px;position:absolute}.glightbox-clean .gclose svg{width:18px;height:auto}.glightbox-clean .gclose:hover{opacity:1}.gfadeIn{-webkit-animation:gfadeIn .5s ease;animation:gfadeIn .5s ease}.gfadeOut{-webkit-animation:gfadeOut .5s ease;animation:gfadeOut .5s ease}.gslideOutLeft{-webkit-animation:gslideOutLeft .3s ease;animation:gslideOutLeft .3s ease}.gslideInLeft{-webkit-animation:gslideInLeft .3s ease;animation:gslideInLeft .3s ease}.gslideOutRight{-webkit-animation:gslideOutRight .3s ease;animation:gslideOutRight .3s ease}.gslideInRight{-webkit-animation:gslideInRight .3s ease;animation:gslideInRight .3s ease}.gzoomIn{-webkit-animation:gzoomIn .5s ease;animation:gzoomIn .5s ease}.gzoomOut{-webkit-animation:gzoomOut .5s ease;animation:gzoomOut .5s ease}@-webkit-keyframes lightboxLoader{0%{-webkit-transform:rotate(0);transform:rotate(0)}100%{-webkit-transform:rotate(360deg);transform:rotate(360deg)}}@keyframes lightboxLoader{0%{-webkit-transform:rotate(0);transform:rotate(0)}100%{-webkit-transform:rotate(360deg);transform:rotate(360deg)}}@-webkit-keyframes gfadeIn{from{opacity:0}to{opacity:1}}@keyframes gfadeIn{from{opacity:0}to{opacity:1}}@-webkit-keyframes gfadeOut{from{opacity:1}to{opacity:0}}@keyframes gfadeOut{from{opacity:1}to{opacity:0}}@-webkit-keyframes gslideInLeft{from{opacity:0;-webkit-transform:translate3d(-60%,0,0);transform:translate3d(-60%,0,0)}to{visibility:visible;-webkit-transform:translate3d(0,0,0);transform:translate3d(0,0,0);opacity:1}}@keyframes gslideInLeft{from{opacity:0;-webkit-transform:translate3d(-60%,0,0);transform:translate3d(-60%,0,0)}to{visibility:visible;-webkit-transform:translate3d(0,0,0);transform:translate3d(0,0,0);opacity:1}}@-webkit-keyframes gslideOutLeft{from{opacity:1;visibility:visible;-webkit-transform:translate3d(0,0,0);transform:translate3d(0,0,0)}to{-webkit-transform:translate3d(-60%,0,0);transform:translate3d(-60%,0,0);opacity:0;visibility:hidden}}@keyframes gslideOutLeft{from{opacity:1;visibility:visible;-webkit-transform:translate3d(0,0,0);transform:translate3d(0,0,0)}to{-webkit-transform:translate3d(-60%,0,0);transform:translate3d(-60%,0,0);opacity:0;visibility:hidden}}@-webkit-keyframes gslideInRight{from{opacity:0;visibility:visible;-webkit-transform:translate3d(60%,0,0);transform:translate3d(60%,0,0)}to{-webkit-transform:translate3d(0,0,0);transform:translate3d(0,0,0);opacity:1}}@keyframes gslideInRight{from{opacity:0;visibility:visible;-webkit-transform:translate3d(60%,0,0);transform:translate3d(60%,0,0)}to{-webkit-transform:translate3d(0,0,0);transform:translate3d(0,0,0);opacity:1}}@-webkit-keyframes gslideOutRight{from{opacity:1;visibility:visible;-webkit-transform:translate3d(0,0,0);transform:translate3d(0,0,0)}to{-webkit-transform:translate3d(60%,0,0);transform:translate3d(60%,0,0);opacity:0}}@keyframes gslideOutRight{from{opacity:1;visibility:visible;-webkit-transform:translate3d(0,0,0);transform:translate3d(0,0,0)}to{-webkit-transform:translate3d(60%,0,0);transform:translate3d(60%,0,0);opacity:0}}@-webkit-keyframes gzoomIn{from{opacity:0;-webkit-transform:scale3d(.3,.3,.3);transform:scale3d(.3,.3,.3)}to{opacity:1}}@keyframes gzoomIn{from{opacity:0;-webkit-transform:scale3d(.3,.3,.3);transform:scale3d(.3,.3,.3)}to{opacity:1}}@-webkit-keyframes gzoomOut{from{opacity:1}50%{opacity:0;-webkit-transform:scale3d(.3,.3,.3);transform:scale3d(.3,.3,.3)}to{opacity:0}}@keyframes gzoomOut{from{opacity:1}50%{opacity:0;-webkit-transform:scale3d(.3,.3,.3);transform:scale3d(.3,.3,.3)}to{opacity:0}}@media (min-width:769px){.glightbox-container .ginner-container{width:auto;height:auto;-webkit-box-orient:horizontal;-webkit-box-direction:normal;-ms-flex-direction:row;flex-direction:row}.glightbox-container .ginner-container.desc-top .gslide-description{-webkit-box-ordinal-group:1;-ms-flex-order:0;order:0}.glightbox-container .ginner-container.desc-top .gslide-image,.glightbox-container .ginner-container.desc-top .gslide-image img{-webkit-box-ordinal-group:2;-ms-flex-order:1;order:1}.glightbox-container .ginner-container.desc-left .gslide-description{-webkit-box-ordinal-group:1;-ms-flex-order:0;order:0}.glightbox-container .ginner-container.desc-left .gslide-image{-webkit-box-ordinal-group:2;-ms-flex-order:1;order:1}.gslide-image img{max-height:97vh;max-width:100%}.gslide-image img.zoomable{cursor:-webkit-zoom-in;cursor:zoom-in}.zoomed .gslide-image img.zoomable{cursor:-webkit-grab;cursor:grab}.gslide-inline{max-height:95vh}.gslide-external{max-height:100vh}.gslide-description.description-left,.gslide-description.description-right{max-width:275px}.glightbox-open{height:auto}.goverlay{background:rgba(0,0,0,.92)}.glightbox-clean .gslide-media{-webkit-box-shadow:1px 2px 9px 0 rgba(0,0,0,.65);box-shadow:1px 2px 9px 0 rgba(0,0,0,.65)}.glightbox-clean .description-left .gdesc-inner,.glightbox-clean .description-right .gdesc-inner{position:absolute;height:100%;overflow-y:auto}.glightbox-clean .gclose,.glightbox-clean .gnext,.glightbox-clean .gprev{background-color:rgba(0,0,0,.32)}.glightbox-clean .gclose:hover,.glightbox-clean .gnext:hover,.glightbox-clean .gprev:hover{background-color:rgba(0,0,0,.7)}.glightbox-clean .gprev{top:45%}.glightbox-clean .gnext{top:45%}}@media (min-width:992px){.glightbox-clean .gclose{opacity:.7;right:20px}}@media screen and (max-height:420px){.goverlay{background:#000}}
\ No newline at end of file
diff --git a/site_libs/quarto-contrib/glightbox/glightbox.min.js b/site_libs/quarto-contrib/glightbox/glightbox.min.js
new file mode 100644
index 0000000..997908b
--- /dev/null
+++ b/site_libs/quarto-contrib/glightbox/glightbox.min.js
@@ -0,0 +1 @@
+!function(e,t){"object"==typeof exports&&"undefined"!=typeof module?module.exports=t():"function"==typeof define&&define.amd?define(t):(e=e||self).GLightbox=t()}(this,(function(){"use strict";function e(t){return(e="function"==typeof Symbol&&"symbol"==typeof Symbol.iterator?function(e){return typeof e}:function(e){return e&&"function"==typeof Symbol&&e.constructor===Symbol&&e!==Symbol.prototype?"symbol":typeof e})(t)}function t(e,t){if(!(e instanceof t))throw new TypeError("Cannot call a class as a function")}function i(e,t){for(var i=0;i1&&void 0!==arguments[1]?arguments[1]:null,i=arguments.length>2&&void 0!==arguments[2]?arguments[2]:null,n=e[s]=e[s]||[],l={all:n,evt:null,found:null};return t&&i&&P(n)>0&&o(n,(function(e,n){if(e.eventName==t&&e.fn.toString()==i.toString())return l.found=!0,l.evt=n,!1})),l}function a(e){var t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:{},i=t.onElement,n=t.withCallback,s=t.avoidDuplicate,l=void 0===s||s,a=t.once,h=void 0!==a&&a,d=t.useCapture,c=void 0!==d&&d,u=arguments.length>2?arguments[2]:void 0,g=i||[];function v(e){T(n)&&n.call(u,e,this),h&&v.destroy()}return C(g)&&(g=document.querySelectorAll(g)),v.destroy=function(){o(g,(function(t){var i=r(t,e,v);i.found&&i.all.splice(i.evt,1),t.removeEventListener&&t.removeEventListener(e,v,c)}))},o(g,(function(t){var i=r(t,e,v);(t.addEventListener&&l&&!i.found||!l)&&(t.addEventListener(e,v,c),i.all.push({eventName:e,fn:v}))})),v}function h(e,t){o(t.split(" "),(function(t){return e.classList.add(t)}))}function d(e,t){o(t.split(" "),(function(t){return e.classList.remove(t)}))}function c(e,t){return e.classList.contains(t)}function u(e,t){for(;e!==document.body;){if(!(e=e.parentElement))return!1;if("function"==typeof e.matches?e.matches(t):e.msMatchesSelector(t))return e}}function g(e){var t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:"",i=arguments.length>2&&void 0!==arguments[2]&&arguments[2];if(!e||""===t)return!1;if("none"===t)return T(i)&&i(),!1;var n=x(),s=t.split(" ");o(s,(function(t){h(e,"g"+t)})),a(n,{onElement:e,avoidDuplicate:!1,once:!0,withCallback:function(e,t){o(s,(function(e){d(t,"g"+e)})),T(i)&&i()}})}function v(e){var t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:"";if(""===t)return e.style.webkitTransform="",e.style.MozTransform="",e.style.msTransform="",e.style.OTransform="",e.style.transform="",!1;e.style.webkitTransform=t,e.style.MozTransform=t,e.style.msTransform=t,e.style.OTransform=t,e.style.transform=t}function f(e){e.style.display="block"}function p(e){e.style.display="none"}function m(e){var t=document.createDocumentFragment(),i=document.createElement("div");for(i.innerHTML=e;i.firstChild;)t.appendChild(i.firstChild);return t}function y(){return{width:window.innerWidth||document.documentElement.clientWidth||document.body.clientWidth,height:window.innerHeight||document.documentElement.clientHeight||document.body.clientHeight}}function x(){var e,t=document.createElement("fakeelement"),i={animation:"animationend",OAnimation:"oAnimationEnd",MozAnimation:"animationend",WebkitAnimation:"webkitAnimationEnd"};for(e in i)if(void 0!==t.style[e])return i[e]}function b(e,t,i,n){if(e())t();else{var s;i||(i=100);var l=setInterval((function(){e()&&(clearInterval(l),s&&clearTimeout(s),t())}),i);n&&(s=setTimeout((function(){clearInterval(l)}),n))}}function S(e,t,i){if(I(e))console.error("Inject assets error");else if(T(t)&&(i=t,t=!1),C(t)&&t in window)T(i)&&i();else{var n;if(-1!==e.indexOf(".css")){if((n=document.querySelectorAll('link[href="'+e+'"]'))&&n.length>0)return void(T(i)&&i());var s=document.getElementsByTagName("head")[0],l=s.querySelectorAll('link[rel="stylesheet"]'),o=document.createElement("link");return o.rel="stylesheet",o.type="text/css",o.href=e,o.media="all",l?s.insertBefore(o,l[0]):s.appendChild(o),void(T(i)&&i())}if((n=document.querySelectorAll('script[src="'+e+'"]'))&&n.length>0){if(T(i)){if(C(t))return b((function(){return void 0!==window[t]}),(function(){i()})),!1;i()}}else{var r=document.createElement("script");r.type="text/javascript",r.src=e,r.onload=function(){if(T(i)){if(C(t))return b((function(){return void 0!==window[t]}),(function(){i()})),!1;i()}},document.body.appendChild(r)}}}function w(){return"navigator"in window&&window.navigator.userAgent.match(/(iPad)|(iPhone)|(iPod)|(Android)|(PlayBook)|(BB10)|(BlackBerry)|(Opera Mini)|(IEMobile)|(webOS)|(MeeGo)/i)}function T(e){return"function"==typeof e}function C(e){return"string"==typeof e}function k(e){return!(!e||!e.nodeType||1!=e.nodeType)}function E(e){return Array.isArray(e)}function A(e){return e&&e.length&&isFinite(e.length)}function L(t){return"object"===e(t)&&null!=t&&!T(t)&&!E(t)}function I(e){return null==e}function O(e,t){return null!==e&&hasOwnProperty.call(e,t)}function P(e){if(L(e)){if(e.keys)return e.keys().length;var t=0;for(var i in e)O(e,i)&&t++;return t}return e.length}function M(e){return!isNaN(parseFloat(e))&&isFinite(e)}function z(){var e=arguments.length>0&&void 0!==arguments[0]?arguments[0]:-1,t=document.querySelectorAll(".gbtn[data-taborder]:not(.disabled)");if(!t.length)return!1;if(1==t.length)return t[0];"string"==typeof e&&(e=parseInt(e));var i=[];o(t,(function(e){i.push(e.getAttribute("data-taborder"))}));var n=Math.max.apply(Math,i.map((function(e){return parseInt(e)}))),s=e<0?1:e+1;s>n&&(s="1");var l=i.filter((function(e){return e>=parseInt(s)})),r=l.sort()[0];return document.querySelector('.gbtn[data-taborder="'.concat(r,'"]'))}function X(e){if(e.events.hasOwnProperty("keyboard"))return!1;e.events.keyboard=a("keydown",{onElement:window,withCallback:function(t,i){var n=(t=t||window.event).keyCode;if(9==n){var s=document.querySelector(".gbtn.focused");if(!s){var l=!(!document.activeElement||!document.activeElement.nodeName)&&document.activeElement.nodeName.toLocaleLowerCase();if("input"==l||"textarea"==l||"button"==l)return}t.preventDefault();var o=document.querySelectorAll(".gbtn[data-taborder]");if(!o||o.length<=0)return;if(!s){var r=z();return void(r&&(r.focus(),h(r,"focused")))}var a=z(s.getAttribute("data-taborder"));d(s,"focused"),a&&(a.focus(),h(a,"focused"))}39==n&&e.nextSlide(),37==n&&e.prevSlide(),27==n&&e.close()}})}function Y(e){return Math.sqrt(e.x*e.x+e.y*e.y)}function q(e,t){var i=function(e,t){var i=Y(e)*Y(t);if(0===i)return 0;var n=function(e,t){return e.x*t.x+e.y*t.y}(e,t)/i;return n>1&&(n=1),Math.acos(n)}(e,t);return function(e,t){return e.x*t.y-t.x*e.y}(e,t)>0&&(i*=-1),180*i/Math.PI}var N=function(){function e(i){t(this,e),this.handlers=[],this.el=i}return n(e,[{key:"add",value:function(e){this.handlers.push(e)}},{key:"del",value:function(e){e||(this.handlers=[]);for(var t=this.handlers.length;t>=0;t--)this.handlers[t]===e&&this.handlers.splice(t,1)}},{key:"dispatch",value:function(){for(var e=0,t=this.handlers.length;e=0)console.log("ignore drag for this touched element",e.target.nodeName.toLowerCase());else{this.now=Date.now(),this.x1=e.touches[0].pageX,this.y1=e.touches[0].pageY,this.delta=this.now-(this.last||this.now),this.touchStart.dispatch(e,this.element),null!==this.preTapPosition.x&&(this.isDoubleTap=this.delta>0&&this.delta<=250&&Math.abs(this.preTapPosition.x-this.x1)<30&&Math.abs(this.preTapPosition.y-this.y1)<30,this.isDoubleTap&&clearTimeout(this.singleTapTimeout)),this.preTapPosition.x=this.x1,this.preTapPosition.y=this.y1,this.last=this.now;var t=this.preV;if(e.touches.length>1){this._cancelLongTap(),this._cancelSingleTap();var i={x:e.touches[1].pageX-this.x1,y:e.touches[1].pageY-this.y1};t.x=i.x,t.y=i.y,this.pinchStartLen=Y(t),this.multipointStart.dispatch(e,this.element)}this._preventTap=!1,this.longTapTimeout=setTimeout(function(){this.longTap.dispatch(e,this.element),this._preventTap=!0}.bind(this),750)}}}},{key:"move",value:function(e){if(e.touches){var t=this.preV,i=e.touches.length,n=e.touches[0].pageX,s=e.touches[0].pageY;if(this.isDoubleTap=!1,i>1){var l=e.touches[1].pageX,o=e.touches[1].pageY,r={x:e.touches[1].pageX-n,y:e.touches[1].pageY-s};null!==t.x&&(this.pinchStartLen>0&&(e.zoom=Y(r)/this.pinchStartLen,this.pinch.dispatch(e,this.element)),e.angle=q(r,t),this.rotate.dispatch(e,this.element)),t.x=r.x,t.y=r.y,null!==this.x2&&null!==this.sx2?(e.deltaX=(n-this.x2+l-this.sx2)/2,e.deltaY=(s-this.y2+o-this.sy2)/2):(e.deltaX=0,e.deltaY=0),this.twoFingerPressMove.dispatch(e,this.element),this.sx2=l,this.sy2=o}else{if(null!==this.x2){e.deltaX=n-this.x2,e.deltaY=s-this.y2;var a=Math.abs(this.x1-this.x2),h=Math.abs(this.y1-this.y2);(a>10||h>10)&&(this._preventTap=!0)}else e.deltaX=0,e.deltaY=0;this.pressMove.dispatch(e,this.element)}this.touchMove.dispatch(e,this.element),this._cancelLongTap(),this.x2=n,this.y2=s,i>1&&e.preventDefault()}}},{key:"end",value:function(e){if(e.changedTouches){this._cancelLongTap();var t=this;e.touches.length<2&&(this.multipointEnd.dispatch(e,this.element),this.sx2=this.sy2=null),this.x2&&Math.abs(this.x1-this.x2)>30||this.y2&&Math.abs(this.y1-this.y2)>30?(e.direction=this._swipeDirection(this.x1,this.x2,this.y1,this.y2),this.swipeTimeout=setTimeout((function(){t.swipe.dispatch(e,t.element)}),0)):(this.tapTimeout=setTimeout((function(){t._preventTap||t.tap.dispatch(e,t.element),t.isDoubleTap&&(t.doubleTap.dispatch(e,t.element),t.isDoubleTap=!1)}),0),t.isDoubleTap||(t.singleTapTimeout=setTimeout((function(){t.singleTap.dispatch(e,t.element)}),250))),this.touchEnd.dispatch(e,this.element),this.preV.x=0,this.preV.y=0,this.zoom=1,this.pinchStartLen=null,this.x1=this.x2=this.y1=this.y2=null}}},{key:"cancelAll",value:function(){this._preventTap=!0,clearTimeout(this.singleTapTimeout),clearTimeout(this.tapTimeout),clearTimeout(this.longTapTimeout),clearTimeout(this.swipeTimeout)}},{key:"cancel",value:function(e){this.cancelAll(),this.touchCancel.dispatch(e,this.element)}},{key:"_cancelLongTap",value:function(){clearTimeout(this.longTapTimeout)}},{key:"_cancelSingleTap",value:function(){clearTimeout(this.singleTapTimeout)}},{key:"_swipeDirection",value:function(e,t,i,n){return Math.abs(e-t)>=Math.abs(i-n)?e-t>0?"Left":"Right":i-n>0?"Up":"Down"}},{key:"on",value:function(e,t){this[e]&&this[e].add(t)}},{key:"off",value:function(e,t){this[e]&&this[e].del(t)}},{key:"destroy",value:function(){return this.singleTapTimeout&&clearTimeout(this.singleTapTimeout),this.tapTimeout&&clearTimeout(this.tapTimeout),this.longTapTimeout&&clearTimeout(this.longTapTimeout),this.swipeTimeout&&clearTimeout(this.swipeTimeout),this.element.removeEventListener("touchstart",this.start),this.element.removeEventListener("touchmove",this.move),this.element.removeEventListener("touchend",this.end),this.element.removeEventListener("touchcancel",this.cancel),this.rotate.del(),this.touchStart.del(),this.multipointStart.del(),this.multipointEnd.del(),this.pinch.del(),this.swipe.del(),this.tap.del(),this.doubleTap.del(),this.longTap.del(),this.singleTap.del(),this.pressMove.del(),this.twoFingerPressMove.del(),this.touchMove.del(),this.touchEnd.del(),this.touchCancel.del(),this.preV=this.pinchStartLen=this.zoom=this.isDoubleTap=this.delta=this.last=this.now=this.tapTimeout=this.singleTapTimeout=this.longTapTimeout=this.swipeTimeout=this.x1=this.x2=this.y1=this.y2=this.preTapPosition=this.rotate=this.touchStart=this.multipointStart=this.multipointEnd=this.pinch=this.swipe=this.tap=this.doubleTap=this.longTap=this.singleTap=this.pressMove=this.touchMove=this.touchEnd=this.touchCancel=this.twoFingerPressMove=null,window.removeEventListener("scroll",this._cancelAllHandler),null}}]),e}();function W(e){var t=function(){var e,t=document.createElement("fakeelement"),i={transition:"transitionend",OTransition:"oTransitionEnd",MozTransition:"transitionend",WebkitTransition:"webkitTransitionEnd"};for(e in i)if(void 0!==t.style[e])return i[e]}(),i=window.innerWidth||document.documentElement.clientWidth||document.body.clientWidth,n=c(e,"gslide-media")?e:e.querySelector(".gslide-media"),s=u(n,".ginner-container"),l=e.querySelector(".gslide-description");i>769&&(n=s),h(n,"greset"),v(n,"translate3d(0, 0, 0)"),a(t,{onElement:n,once:!0,withCallback:function(e,t){d(n,"greset")}}),n.style.opacity="",l&&(l.style.opacity="")}function B(e){if(e.events.hasOwnProperty("touch"))return!1;var t,i,n,s=y(),l=s.width,o=s.height,r=!1,a=null,g=null,f=null,p=!1,m=1,x=1,b=!1,S=!1,w=null,T=null,C=null,k=null,E=0,A=0,L=!1,I=!1,O={},P={},M=0,z=0,X=document.getElementById("glightbox-slider"),Y=document.querySelector(".goverlay"),q=new _(X,{touchStart:function(t){if(r=!0,(c(t.targetTouches[0].target,"ginner-container")||u(t.targetTouches[0].target,".gslide-desc")||"a"==t.targetTouches[0].target.nodeName.toLowerCase())&&(r=!1),u(t.targetTouches[0].target,".gslide-inline")&&!c(t.targetTouches[0].target.parentNode,"gslide-inline")&&(r=!1),r){if(P=t.targetTouches[0],O.pageX=t.targetTouches[0].pageX,O.pageY=t.targetTouches[0].pageY,M=t.targetTouches[0].clientX,z=t.targetTouches[0].clientY,a=e.activeSlide,g=a.querySelector(".gslide-media"),n=a.querySelector(".gslide-inline"),f=null,c(g,"gslide-image")&&(f=g.querySelector("img")),(window.innerWidth||document.documentElement.clientWidth||document.body.clientWidth)>769&&(g=a.querySelector(".ginner-container")),d(Y,"greset"),t.pageX>20&&t.pageXo){var a=O.pageX-P.pageX;if(Math.abs(a)<=13)return!1}p=!0;var h,d=s.targetTouches[0].clientX,c=s.targetTouches[0].clientY,u=M-d,m=z-c;if(Math.abs(u)>Math.abs(m)?(L=!1,I=!0):(I=!1,L=!0),t=P.pageX-O.pageX,E=100*t/l,i=P.pageY-O.pageY,A=100*i/o,L&&f&&(h=1-Math.abs(i)/o,Y.style.opacity=h,e.settings.touchFollowAxis&&(E=0)),I&&(h=1-Math.abs(t)/l,g.style.opacity=h,e.settings.touchFollowAxis&&(A=0)),!f)return v(g,"translate3d(".concat(E,"%, 0, 0)"));v(g,"translate3d(".concat(E,"%, ").concat(A,"%, 0)"))}},touchEnd:function(){if(r){if(p=!1,S||b)return C=w,void(k=T);var t=Math.abs(parseInt(A)),i=Math.abs(parseInt(E));if(!(t>29&&f))return t<29&&i<25?(h(Y,"greset"),Y.style.opacity=1,W(g)):void 0;e.close()}},multipointEnd:function(){setTimeout((function(){b=!1}),50)},multipointStart:function(){b=!0,m=x||1},pinch:function(e){if(!f||p)return!1;b=!0,f.scaleX=f.scaleY=m*e.zoom;var t=m*e.zoom;if(S=!0,t<=1)return S=!1,t=1,k=null,C=null,w=null,T=null,void f.setAttribute("style","");t>4.5&&(t=4.5),f.style.transform="scale3d(".concat(t,", ").concat(t,", 1)"),x=t},pressMove:function(e){if(S&&!b){var t=P.pageX-O.pageX,i=P.pageY-O.pageY;C&&(t+=C),k&&(i+=k),w=t,T=i;var n="translate3d(".concat(t,"px, ").concat(i,"px, 0)");x&&(n+=" scale3d(".concat(x,", ").concat(x,", 1)")),v(f,n)}},swipe:function(t){if(!S)if(b)b=!1;else{if("Left"==t.direction){if(e.index==e.elements.length-1)return W(g);e.nextSlide()}if("Right"==t.direction){if(0==e.index)return W(g);e.prevSlide()}}}});e.events.touch=q}var H=function(){function e(i,n){var s=this,l=arguments.length>2&&void 0!==arguments[2]?arguments[2]:null;if(t(this,e),this.img=i,this.slide=n,this.onclose=l,this.img.setZoomEvents)return!1;this.active=!1,this.zoomedIn=!1,this.dragging=!1,this.currentX=null,this.currentY=null,this.initialX=null,this.initialY=null,this.xOffset=0,this.yOffset=0,this.img.addEventListener("mousedown",(function(e){return s.dragStart(e)}),!1),this.img.addEventListener("mouseup",(function(e){return s.dragEnd(e)}),!1),this.img.addEventListener("mousemove",(function(e){return s.drag(e)}),!1),this.img.addEventListener("click",(function(e){return s.slide.classList.contains("dragging-nav")?(s.zoomOut(),!1):s.zoomedIn?void(s.zoomedIn&&!s.dragging&&s.zoomOut()):s.zoomIn()}),!1),this.img.setZoomEvents=!0}return n(e,[{key:"zoomIn",value:function(){var e=this.widowWidth();if(!(this.zoomedIn||e<=768)){var t=this.img;if(t.setAttribute("data-style",t.getAttribute("style")),t.style.maxWidth=t.naturalWidth+"px",t.style.maxHeight=t.naturalHeight+"px",t.naturalWidth>e){var i=e/2-t.naturalWidth/2;this.setTranslate(this.img.parentNode,i,0)}this.slide.classList.add("zoomed"),this.zoomedIn=!0}}},{key:"zoomOut",value:function(){this.img.parentNode.setAttribute("style",""),this.img.setAttribute("style",this.img.getAttribute("data-style")),this.slide.classList.remove("zoomed"),this.zoomedIn=!1,this.currentX=null,this.currentY=null,this.initialX=null,this.initialY=null,this.xOffset=0,this.yOffset=0,this.onclose&&"function"==typeof this.onclose&&this.onclose()}},{key:"dragStart",value:function(e){e.preventDefault(),this.zoomedIn?("touchstart"===e.type?(this.initialX=e.touches[0].clientX-this.xOffset,this.initialY=e.touches[0].clientY-this.yOffset):(this.initialX=e.clientX-this.xOffset,this.initialY=e.clientY-this.yOffset),e.target===this.img&&(this.active=!0,this.img.classList.add("dragging"))):this.active=!1}},{key:"dragEnd",value:function(e){var t=this;e.preventDefault(),this.initialX=this.currentX,this.initialY=this.currentY,this.active=!1,setTimeout((function(){t.dragging=!1,t.img.isDragging=!1,t.img.classList.remove("dragging")}),100)}},{key:"drag",value:function(e){this.active&&(e.preventDefault(),"touchmove"===e.type?(this.currentX=e.touches[0].clientX-this.initialX,this.currentY=e.touches[0].clientY-this.initialY):(this.currentX=e.clientX-this.initialX,this.currentY=e.clientY-this.initialY),this.xOffset=this.currentX,this.yOffset=this.currentY,this.img.isDragging=!0,this.dragging=!0,this.setTranslate(this.img,this.currentX,this.currentY))}},{key:"onMove",value:function(e){if(this.zoomedIn){var t=e.clientX-this.img.naturalWidth/2,i=e.clientY-this.img.naturalHeight/2;this.setTranslate(this.img,t,i)}}},{key:"setTranslate",value:function(e,t,i){e.style.transform="translate3d("+t+"px, "+i+"px, 0)"}},{key:"widowWidth",value:function(){return window.innerWidth||document.documentElement.clientWidth||document.body.clientWidth}}]),e}(),V=function(){function e(){var i=this,n=arguments.length>0&&void 0!==arguments[0]?arguments[0]:{};t(this,e);var s=n.dragEl,l=n.toleranceX,o=void 0===l?40:l,r=n.toleranceY,a=void 0===r?65:r,h=n.slide,d=void 0===h?null:h,c=n.instance,u=void 0===c?null:c;this.el=s,this.active=!1,this.dragging=!1,this.currentX=null,this.currentY=null,this.initialX=null,this.initialY=null,this.xOffset=0,this.yOffset=0,this.direction=null,this.lastDirection=null,this.toleranceX=o,this.toleranceY=a,this.toleranceReached=!1,this.dragContainer=this.el,this.slide=d,this.instance=u,this.el.addEventListener("mousedown",(function(e){return i.dragStart(e)}),!1),this.el.addEventListener("mouseup",(function(e){return i.dragEnd(e)}),!1),this.el.addEventListener("mousemove",(function(e){return i.drag(e)}),!1)}return n(e,[{key:"dragStart",value:function(e){if(this.slide.classList.contains("zoomed"))this.active=!1;else{"touchstart"===e.type?(this.initialX=e.touches[0].clientX-this.xOffset,this.initialY=e.touches[0].clientY-this.yOffset):(this.initialX=e.clientX-this.xOffset,this.initialY=e.clientY-this.yOffset);var t=e.target.nodeName.toLowerCase();e.target.classList.contains("nodrag")||u(e.target,".nodrag")||-1!==["input","select","textarea","button","a"].indexOf(t)?this.active=!1:(e.preventDefault(),(e.target===this.el||"img"!==t&&u(e.target,".gslide-inline"))&&(this.active=!0,this.el.classList.add("dragging"),this.dragContainer=u(e.target,".ginner-container")))}}},{key:"dragEnd",value:function(e){var t=this;e&&e.preventDefault(),this.initialX=0,this.initialY=0,this.currentX=null,this.currentY=null,this.initialX=null,this.initialY=null,this.xOffset=0,this.yOffset=0,this.active=!1,this.doSlideChange&&(this.instance.preventOutsideClick=!0,"right"==this.doSlideChange&&this.instance.prevSlide(),"left"==this.doSlideChange&&this.instance.nextSlide()),this.doSlideClose&&this.instance.close(),this.toleranceReached||this.setTranslate(this.dragContainer,0,0,!0),setTimeout((function(){t.instance.preventOutsideClick=!1,t.toleranceReached=!1,t.lastDirection=null,t.dragging=!1,t.el.isDragging=!1,t.el.classList.remove("dragging"),t.slide.classList.remove("dragging-nav"),t.dragContainer.style.transform="",t.dragContainer.style.transition=""}),100)}},{key:"drag",value:function(e){if(this.active){e.preventDefault(),this.slide.classList.add("dragging-nav"),"touchmove"===e.type?(this.currentX=e.touches[0].clientX-this.initialX,this.currentY=e.touches[0].clientY-this.initialY):(this.currentX=e.clientX-this.initialX,this.currentY=e.clientY-this.initialY),this.xOffset=this.currentX,this.yOffset=this.currentY,this.el.isDragging=!0,this.dragging=!0,this.doSlideChange=!1,this.doSlideClose=!1;var t=Math.abs(this.currentX),i=Math.abs(this.currentY);if(t>0&&t>=Math.abs(this.currentY)&&(!this.lastDirection||"x"==this.lastDirection)){this.yOffset=0,this.lastDirection="x",this.setTranslate(this.dragContainer,this.currentX,0);var n=this.shouldChange();if(!this.instance.settings.dragAutoSnap&&n&&(this.doSlideChange=n),this.instance.settings.dragAutoSnap&&n)return this.instance.preventOutsideClick=!0,this.toleranceReached=!0,this.active=!1,this.instance.preventOutsideClick=!0,this.dragEnd(null),"right"==n&&this.instance.prevSlide(),void("left"==n&&this.instance.nextSlide())}if(this.toleranceY>0&&i>0&&i>=t&&(!this.lastDirection||"y"==this.lastDirection)){this.xOffset=0,this.lastDirection="y",this.setTranslate(this.dragContainer,0,this.currentY);var s=this.shouldClose();return!this.instance.settings.dragAutoSnap&&s&&(this.doSlideClose=!0),void(this.instance.settings.dragAutoSnap&&s&&this.instance.close())}}}},{key:"shouldChange",value:function(){var e=!1;if(Math.abs(this.currentX)>=this.toleranceX){var t=this.currentX>0?"right":"left";("left"==t&&this.slide!==this.slide.parentNode.lastChild||"right"==t&&this.slide!==this.slide.parentNode.firstChild)&&(e=t)}return e}},{key:"shouldClose",value:function(){var e=!1;return Math.abs(this.currentY)>=this.toleranceY&&(e=!0),e}},{key:"setTranslate",value:function(e,t,i){var n=arguments.length>3&&void 0!==arguments[3]&&arguments[3];e.style.transition=n?"all .2s ease":"",e.style.transform="translate3d(".concat(t,"px, ").concat(i,"px, 0)")}}]),e}();function j(e,t,i,n){var s=e.querySelector(".gslide-media"),l=new Image,o="gSlideTitle_"+i,r="gSlideDesc_"+i;l.addEventListener("load",(function(){T(n)&&n()}),!1),l.src=t.href,""!=t.sizes&&""!=t.srcset&&(l.sizes=t.sizes,l.srcset=t.srcset),l.alt="",I(t.alt)||""===t.alt||(l.alt=t.alt),""!==t.title&&l.setAttribute("aria-labelledby",o),""!==t.description&&l.setAttribute("aria-describedby",r),t.hasOwnProperty("_hasCustomWidth")&&t._hasCustomWidth&&(l.style.width=t.width),t.hasOwnProperty("_hasCustomHeight")&&t._hasCustomHeight&&(l.style.height=t.height),s.insertBefore(l,s.firstChild)}function F(e,t,i,n){var s=this,l=e.querySelector(".ginner-container"),o="gvideo"+i,r=e.querySelector(".gslide-media"),a=this.getAllPlayers();h(l,"gvideo-container"),r.insertBefore(m(''),r.firstChild);var d=e.querySelector(".gvideo-wrapper");S(this.settings.plyr.css,"Plyr");var c=t.href,u=null==t?void 0:t.videoProvider,g=!1;r.style.maxWidth=t.width,S(this.settings.plyr.js,"Plyr",(function(){if(!u&&c.match(/vimeo\.com\/([0-9]*)/)&&(u="vimeo"),!u&&(c.match(/(youtube\.com|youtube-nocookie\.com)\/watch\?v=([a-zA-Z0-9\-_]+)/)||c.match(/youtu\.be\/([a-zA-Z0-9\-_]+)/)||c.match(/(youtube\.com|youtube-nocookie\.com)\/embed\/([a-zA-Z0-9\-_]+)/))&&(u="youtube"),"local"===u||!u){u="local";var l='")}var r=g||m(''));h(d,"".concat(u,"-video gvideo")),d.appendChild(r),d.setAttribute("data-id",o),d.setAttribute("data-index",i);var v=O(s.settings.plyr,"config")?s.settings.plyr.config:{},f=new Plyr("#"+o,v);f.on("ready",(function(e){a[o]=e.detail.plyr,T(n)&&n()})),b((function(){return e.querySelector("iframe")&&"true"==e.querySelector("iframe").dataset.ready}),(function(){s.resize(e)})),f.on("enterfullscreen",R),f.on("exitfullscreen",R)}))}function R(e){var t=u(e.target,".gslide-media");"enterfullscreen"===e.type&&h(t,"fullscreen"),"exitfullscreen"===e.type&&d(t,"fullscreen")}function G(e,t,i,n){var s,l=this,o=e.querySelector(".gslide-media"),r=!(!O(t,"href")||!t.href)&&t.href.split("#").pop().trim(),d=!(!O(t,"content")||!t.content)&&t.content;if(d&&(C(d)&&(s=m(''.concat(d,""))),k(d))){"none"==d.style.display&&(d.style.display="block");var c=document.createElement("div");c.className="ginlined-content",c.appendChild(d),s=c}if(r){var u=document.getElementById(r);if(!u)return!1;var g=u.cloneNode(!0);g.style.height=t.height,g.style.maxWidth=t.width,h(g,"ginlined-content"),s=g}if(!s)return console.error("Unable to append inline slide content",t),!1;o.style.height=t.height,o.style.width=t.width,o.appendChild(s),this.events["inlineclose"+r]=a("click",{onElement:o.querySelectorAll(".gtrigger-close"),withCallback:function(e){e.preventDefault(),l.close()}}),T(n)&&n()}function Z(e,t,i,n){var s=e.querySelector(".gslide-media"),l=function(e){var t=e.url,i=e.allow,n=e.callback,s=e.appendTo,l=document.createElement("iframe");return l.className="vimeo-video gvideo",l.src=t,l.style.width="100%",l.style.height="100%",i&&l.setAttribute("allow",i),l.onload=function(){l.onload=null,h(l,"node-ready"),T(n)&&n()},s&&s.appendChild(l),l}({url:t.href,callback:n});s.parentNode.style.maxWidth=t.width,s.parentNode.style.height=t.height,s.appendChild(l)}var U=function(){function e(){var i=arguments.length>0&&void 0!==arguments[0]?arguments[0]:{};t(this,e),this.defaults={href:"",sizes:"",srcset:"",title:"",type:"",videoProvider:"",description:"",alt:"",descPosition:"bottom",effect:"",width:"",height:"",content:!1,zoomable:!0,draggable:!0},L(i)&&(this.defaults=l(this.defaults,i))}return n(e,[{key:"sourceType",value:function(e){var t=e;if(null!==(e=e.toLowerCase()).match(/\.(jpeg|jpg|jpe|gif|png|apn|webp|avif|svg)/))return"image";if(e.match(/(youtube\.com|youtube-nocookie\.com)\/watch\?v=([a-zA-Z0-9\-_]+)/)||e.match(/youtu\.be\/([a-zA-Z0-9\-_]+)/)||e.match(/(youtube\.com|youtube-nocookie\.com)\/embed\/([a-zA-Z0-9\-_]+)/))return"video";if(e.match(/vimeo\.com\/([0-9]*)/))return"video";if(null!==e.match(/\.(mp4|ogg|webm|mov)/))return"video";if(null!==e.match(/\.(mp3|wav|wma|aac|ogg)/))return"audio";if(e.indexOf("#")>-1&&""!==t.split("#").pop().trim())return"inline";return e.indexOf("goajax=true")>-1?"ajax":"external"}},{key:"parseConfig",value:function(e,t){var i=this,n=l({descPosition:t.descPosition},this.defaults);if(L(e)&&!k(e)){O(e,"type")||(O(e,"content")&&e.content?e.type="inline":O(e,"href")&&(e.type=this.sourceType(e.href)));var s=l(n,e);return this.setSize(s,t),s}var r="",a=e.getAttribute("data-glightbox"),h=e.nodeName.toLowerCase();if("a"===h&&(r=e.href),"img"===h&&(r=e.src,n.alt=e.alt),n.href=r,o(n,(function(s,l){O(t,l)&&"width"!==l&&(n[l]=t[l]);var o=e.dataset[l];I(o)||(n[l]=i.sanitizeValue(o))})),n.content&&(n.type="inline"),!n.type&&r&&(n.type=this.sourceType(r)),I(a)){if(!n.title&&"a"==h){var d=e.title;I(d)||""===d||(n.title=d)}if(!n.title&&"img"==h){var c=e.alt;I(c)||""===c||(n.title=c)}}else{var u=[];o(n,(function(e,t){u.push(";\\s?"+t)})),u=u.join("\\s?:|"),""!==a.trim()&&o(n,(function(e,t){var s=a,l=new RegExp("s?"+t+"s?:s?(.*?)("+u+"s?:|$)"),o=s.match(l);if(o&&o.length&&o[1]){var r=o[1].trim().replace(/;\s*$/,"");n[t]=i.sanitizeValue(r)}}))}if(n.description&&"."===n.description.substring(0,1)){var g;try{g=document.querySelector(n.description).innerHTML}catch(e){if(!(e instanceof DOMException))throw e}g&&(n.description=g)}if(!n.description){var v=e.querySelector(".glightbox-desc");v&&(n.description=v.innerHTML)}return this.setSize(n,t,e),this.slideConfig=n,n}},{key:"setSize",value:function(e,t){var i=arguments.length>2&&void 0!==arguments[2]?arguments[2]:null,n="video"==e.type?this.checkSize(t.videosWidth):this.checkSize(t.width),s=this.checkSize(t.height);return e.width=O(e,"width")&&""!==e.width?this.checkSize(e.width):n,e.height=O(e,"height")&&""!==e.height?this.checkSize(e.height):s,i&&"image"==e.type&&(e._hasCustomWidth=!!i.dataset.width,e._hasCustomHeight=!!i.dataset.height),e}},{key:"checkSize",value:function(e){return M(e)?"".concat(e,"px"):e}},{key:"sanitizeValue",value:function(e){return"true"!==e&&"false"!==e?e:"true"===e}}]),e}(),$=function(){function e(i,n,s){t(this,e),this.element=i,this.instance=n,this.index=s}return n(e,[{key:"setContent",value:function(){var e=this,t=arguments.length>0&&void 0!==arguments[0]?arguments[0]:null,i=arguments.length>1&&void 0!==arguments[1]&&arguments[1];if(c(t,"loaded"))return!1;var n=this.instance.settings,s=this.slideConfig,l=w();T(n.beforeSlideLoad)&&n.beforeSlideLoad({index:this.index,slide:t,player:!1});var o=s.type,r=s.descPosition,a=t.querySelector(".gslide-media"),d=t.querySelector(".gslide-title"),u=t.querySelector(".gslide-desc"),g=t.querySelector(".gdesc-inner"),v=i,f="gSlideTitle_"+this.index,p="gSlideDesc_"+this.index;if(T(n.afterSlideLoad)&&(v=function(){T(i)&&i(),n.afterSlideLoad({index:e.index,slide:t,player:e.instance.getSlidePlayerInstance(e.index)})}),""==s.title&&""==s.description?g&&g.parentNode.parentNode.removeChild(g.parentNode):(d&&""!==s.title?(d.id=f,d.innerHTML=s.title):d.parentNode.removeChild(d),u&&""!==s.description?(u.id=p,l&&n.moreLength>0?(s.smallDescription=this.slideShortDesc(s.description,n.moreLength,n.moreText),u.innerHTML=s.smallDescription,this.descriptionEvents(u,s)):u.innerHTML=s.description):u.parentNode.removeChild(u),h(a.parentNode,"desc-".concat(r)),h(g.parentNode,"description-".concat(r))),h(a,"gslide-".concat(o)),h(t,"loaded"),"video"!==o){if("external"!==o)return"inline"===o?(G.apply(this.instance,[t,s,this.index,v]),void(s.draggable&&new V({dragEl:t.querySelector(".gslide-inline"),toleranceX:n.dragToleranceX,toleranceY:n.dragToleranceY,slide:t,instance:this.instance}))):void("image"!==o?T(v)&&v():j(t,s,this.index,(function(){var i=t.querySelector("img");s.draggable&&new V({dragEl:i,toleranceX:n.dragToleranceX,toleranceY:n.dragToleranceY,slide:t,instance:e.instance}),s.zoomable&&i.naturalWidth>i.offsetWidth&&(h(i,"zoomable"),new H(i,t,(function(){e.instance.resize()}))),T(v)&&v()})));Z.apply(this,[t,s,this.index,v])}else F.apply(this.instance,[t,s,this.index,v])}},{key:"slideShortDesc",value:function(e){var t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:50,i=arguments.length>2&&void 0!==arguments[2]&&arguments[2],n=document.createElement("div");n.innerHTML=e;var s=n.innerText,l=i;if((e=s.trim()).length<=t)return e;var o=e.substr(0,t-1);return l?(n=null,o+'... '+i+""):o}},{key:"descriptionEvents",value:function(e,t){var i=this,n=e.querySelector(".desc-more");if(!n)return!1;a("click",{onElement:n,withCallback:function(e,n){e.preventDefault();var s=document.body,l=u(n,".gslide-desc");if(!l)return!1;l.innerHTML=t.description,h(s,"gdesc-open");var o=a("click",{onElement:[s,u(l,".gslide-description")],withCallback:function(e,n){"a"!==e.target.nodeName.toLowerCase()&&(d(s,"gdesc-open"),h(s,"gdesc-closed"),l.innerHTML=t.smallDescription,i.descriptionEvents(l,t),setTimeout((function(){d(s,"gdesc-closed")}),400),o.destroy())}})}})}},{key:"create",value:function(){return m(this.instance.settings.slideHTML)}},{key:"getConfig",value:function(){k(this.element)||this.element.hasOwnProperty("draggable")||(this.element.draggable=this.instance.settings.draggable);var e=new U(this.instance.settings.slideExtraAttributes);return this.slideConfig=e.parseConfig(this.element,this.instance.settings),this.slideConfig}}]),e}(),J=w(),K=null!==w()||void 0!==document.createTouch||"ontouchstart"in window||"onmsgesturechange"in window||navigator.msMaxTouchPoints,Q=document.getElementsByTagName("html")[0],ee={selector:".glightbox",elements:null,skin:"clean",theme:"clean",closeButton:!0,startAt:null,autoplayVideos:!0,autofocusVideos:!0,descPosition:"bottom",width:"900px",height:"506px",videosWidth:"960px",beforeSlideChange:null,afterSlideChange:null,beforeSlideLoad:null,afterSlideLoad:null,slideInserted:null,slideRemoved:null,slideExtraAttributes:null,onOpen:null,onClose:null,loop:!1,zoomable:!0,draggable:!0,dragAutoSnap:!1,dragToleranceX:40,dragToleranceY:65,preload:!0,oneSlidePerOpen:!1,touchNavigation:!0,touchFollowAxis:!0,keyboardNavigation:!0,closeOnOutsideClick:!0,plugins:!1,plyr:{css:"https://cdn.plyr.io/3.6.12/plyr.css",js:"https://cdn.plyr.io/3.6.12/plyr.js",config:{ratio:"16:9",fullscreen:{enabled:!0,iosNative:!0},youtube:{noCookie:!0,rel:0,showinfo:0,iv_load_policy:3},vimeo:{byline:!1,portrait:!1,title:!1,transparent:!1}}},openEffect:"zoom",closeEffect:"zoom",slideEffect:"slide",moreText:"See more",moreLength:60,cssEfects:{fade:{in:"fadeIn",out:"fadeOut"},zoom:{in:"zoomIn",out:"zoomOut"},slide:{in:"slideInRight",out:"slideOutLeft"},slideBack:{in:"slideInLeft",out:"slideOutRight"},none:{in:"none",out:"none"}},svg:{close:'',next:'',prev:''},slideHTML:'',lightboxHTML:'\n \n \n \n \n \n \n \n\n'},te=function(){function e(){var i=arguments.length>0&&void 0!==arguments[0]?arguments[0]:{};t(this,e),this.customOptions=i,this.settings=l(ee,i),this.effectsClasses=this.getAnimationClasses(),this.videoPlayers={},this.apiEvents=[],this.fullElementsList=!1}return n(e,[{key:"init",value:function(){var e=this,t=this.getSelector();t&&(this.baseEvents=a("click",{onElement:t,withCallback:function(t,i){t.preventDefault(),e.open(i)}})),this.elements=this.getElements()}},{key:"open",value:function(){var e=arguments.length>0&&void 0!==arguments[0]?arguments[0]:null,t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:null;if(0===this.elements.length)return!1;this.activeSlide=null,this.prevActiveSlideIndex=null,this.prevActiveSlide=null;var i=M(t)?t:this.settings.startAt;if(k(e)){var n=e.getAttribute("data-gallery");n&&(this.fullElementsList=this.elements,this.elements=this.getGalleryElements(this.elements,n)),I(i)&&(i=this.getElementIndex(e))<0&&(i=0)}M(i)||(i=0),this.build(),g(this.overlay,"none"===this.settings.openEffect?"none":this.settings.cssEfects.fade.in);var s=document.body,l=window.innerWidth-document.documentElement.clientWidth;if(l>0){var o=document.createElement("style");o.type="text/css",o.className="gcss-styles",o.innerText=".gscrollbar-fixer {margin-right: ".concat(l,"px}"),document.head.appendChild(o),h(s,"gscrollbar-fixer")}h(s,"glightbox-open"),h(Q,"glightbox-open"),J&&(h(document.body,"glightbox-mobile"),this.settings.slideEffect="slide"),this.showSlide(i,!0),1===this.elements.length?(h(this.prevButton,"glightbox-button-hidden"),h(this.nextButton,"glightbox-button-hidden")):(d(this.prevButton,"glightbox-button-hidden"),d(this.nextButton,"glightbox-button-hidden")),this.lightboxOpen=!0,this.trigger("open"),T(this.settings.onOpen)&&this.settings.onOpen(),K&&this.settings.touchNavigation&&B(this),this.settings.keyboardNavigation&&X(this)}},{key:"openAt",value:function(){var e=arguments.length>0&&void 0!==arguments[0]?arguments[0]:0;this.open(null,e)}},{key:"showSlide",value:function(){var e=this,t=arguments.length>0&&void 0!==arguments[0]?arguments[0]:0,i=arguments.length>1&&void 0!==arguments[1]&&arguments[1];f(this.loader),this.index=parseInt(t);var n=this.slidesContainer.querySelector(".current");n&&d(n,"current"),this.slideAnimateOut();var s=this.slidesContainer.querySelectorAll(".gslide")[t];if(c(s,"loaded"))this.slideAnimateIn(s,i),p(this.loader);else{f(this.loader);var l=this.elements[t],o={index:this.index,slide:s,slideNode:s,slideConfig:l.slideConfig,slideIndex:this.index,trigger:l.node,player:null};this.trigger("slide_before_load",o),l.instance.setContent(s,(function(){p(e.loader),e.resize(),e.slideAnimateIn(s,i),e.trigger("slide_after_load",o)}))}this.slideDescription=s.querySelector(".gslide-description"),this.slideDescriptionContained=this.slideDescription&&c(this.slideDescription.parentNode,"gslide-media"),this.settings.preload&&(this.preloadSlide(t+1),this.preloadSlide(t-1)),this.updateNavigationClasses(),this.activeSlide=s}},{key:"preloadSlide",value:function(e){var t=this;if(e<0||e>this.elements.length-1)return!1;if(I(this.elements[e]))return!1;var i=this.slidesContainer.querySelectorAll(".gslide")[e];if(c(i,"loaded"))return!1;var n=this.elements[e],s=n.type,l={index:e,slide:i,slideNode:i,slideConfig:n.slideConfig,slideIndex:e,trigger:n.node,player:null};this.trigger("slide_before_load",l),"video"===s||"external"===s?setTimeout((function(){n.instance.setContent(i,(function(){t.trigger("slide_after_load",l)}))}),200):n.instance.setContent(i,(function(){t.trigger("slide_after_load",l)}))}},{key:"prevSlide",value:function(){this.goToSlide(this.index-1)}},{key:"nextSlide",value:function(){this.goToSlide(this.index+1)}},{key:"goToSlide",value:function(){var e=arguments.length>0&&void 0!==arguments[0]&&arguments[0];if(this.prevActiveSlide=this.activeSlide,this.prevActiveSlideIndex=this.index,!this.loop()&&(e<0||e>this.elements.length-1))return!1;e<0?e=this.elements.length-1:e>=this.elements.length&&(e=0),this.showSlide(e)}},{key:"insertSlide",value:function(){var e=arguments.length>0&&void 0!==arguments[0]?arguments[0]:{},t=arguments.length>1&&void 0!==arguments[1]?arguments[1]:-1;t<0&&(t=this.elements.length);var i=new $(e,this,t),n=i.getConfig(),s=l({},n),o=i.create(),r=this.elements.length-1;s.index=t,s.node=!1,s.instance=i,s.slideConfig=n,this.elements.splice(t,0,s);var a=null,h=null;if(this.slidesContainer){if(t>r)this.slidesContainer.appendChild(o);else{var d=this.slidesContainer.querySelectorAll(".gslide")[t];this.slidesContainer.insertBefore(o,d)}(this.settings.preload&&0==this.index&&0==t||this.index-1==t||this.index+1==t)&&this.preloadSlide(t),0===this.index&&0===t&&(this.index=1),this.updateNavigationClasses(),a=this.slidesContainer.querySelectorAll(".gslide")[t],h=this.getSlidePlayerInstance(t),s.slideNode=a}this.trigger("slide_inserted",{index:t,slide:a,slideNode:a,slideConfig:n,slideIndex:t,trigger:null,player:h}),T(this.settings.slideInserted)&&this.settings.slideInserted({index:t,slide:a,player:h})}},{key:"removeSlide",value:function(){var e=arguments.length>0&&void 0!==arguments[0]?arguments[0]:-1;if(e<0||e>this.elements.length-1)return!1;var t=this.slidesContainer&&this.slidesContainer.querySelectorAll(".gslide")[e];t&&(this.getActiveSlideIndex()==e&&(e==this.elements.length-1?this.prevSlide():this.nextSlide()),t.parentNode.removeChild(t)),this.elements.splice(e,1),this.trigger("slide_removed",e),T(this.settings.slideRemoved)&&this.settings.slideRemoved(e)}},{key:"slideAnimateIn",value:function(e,t){var i=this,n=e.querySelector(".gslide-media"),s=e.querySelector(".gslide-description"),l={index:this.prevActiveSlideIndex,slide:this.prevActiveSlide,slideNode:this.prevActiveSlide,slideIndex:this.prevActiveSlide,slideConfig:I(this.prevActiveSlideIndex)?null:this.elements[this.prevActiveSlideIndex].slideConfig,trigger:I(this.prevActiveSlideIndex)?null:this.elements[this.prevActiveSlideIndex].node,player:this.getSlidePlayerInstance(this.prevActiveSlideIndex)},o={index:this.index,slide:this.activeSlide,slideNode:this.activeSlide,slideConfig:this.elements[this.index].slideConfig,slideIndex:this.index,trigger:this.elements[this.index].node,player:this.getSlidePlayerInstance(this.index)};if(n.offsetWidth>0&&s&&(p(s),s.style.display=""),d(e,this.effectsClasses),t)g(e,this.settings.cssEfects[this.settings.openEffect].in,(function(){i.settings.autoplayVideos&&i.slidePlayerPlay(e),i.trigger("slide_changed",{prev:l,current:o}),T(i.settings.afterSlideChange)&&i.settings.afterSlideChange.apply(i,[l,o])}));else{var r=this.settings.slideEffect,a="none"!==r?this.settings.cssEfects[r].in:r;this.prevActiveSlideIndex>this.index&&"slide"==this.settings.slideEffect&&(a=this.settings.cssEfects.slideBack.in),g(e,a,(function(){i.settings.autoplayVideos&&i.slidePlayerPlay(e),i.trigger("slide_changed",{prev:l,current:o}),T(i.settings.afterSlideChange)&&i.settings.afterSlideChange.apply(i,[l,o])}))}setTimeout((function(){i.resize(e)}),100),h(e,"current")}},{key:"slideAnimateOut",value:function(){if(!this.prevActiveSlide)return!1;var e=this.prevActiveSlide;d(e,this.effectsClasses),h(e,"prev");var t=this.settings.slideEffect,i="none"!==t?this.settings.cssEfects[t].out:t;this.slidePlayerPause(e),this.trigger("slide_before_change",{prev:{index:this.prevActiveSlideIndex,slide:this.prevActiveSlide,slideNode:this.prevActiveSlide,slideIndex:this.prevActiveSlideIndex,slideConfig:I(this.prevActiveSlideIndex)?null:this.elements[this.prevActiveSlideIndex].slideConfig,trigger:I(this.prevActiveSlideIndex)?null:this.elements[this.prevActiveSlideIndex].node,player:this.getSlidePlayerInstance(this.prevActiveSlideIndex)},current:{index:this.index,slide:this.activeSlide,slideNode:this.activeSlide,slideIndex:this.index,slideConfig:this.elements[this.index].slideConfig,trigger:this.elements[this.index].node,player:this.getSlidePlayerInstance(this.index)}}),T(this.settings.beforeSlideChange)&&this.settings.beforeSlideChange.apply(this,[{index:this.prevActiveSlideIndex,slide:this.prevActiveSlide,player:this.getSlidePlayerInstance(this.prevActiveSlideIndex)},{index:this.index,slide:this.activeSlide,player:this.getSlidePlayerInstance(this.index)}]),this.prevActiveSlideIndex>this.index&&"slide"==this.settings.slideEffect&&(i=this.settings.cssEfects.slideBack.out),g(e,i,(function(){var t=e.querySelector(".ginner-container"),i=e.querySelector(".gslide-media"),n=e.querySelector(".gslide-description");t.style.transform="",i.style.transform="",d(i,"greset"),i.style.opacity="",n&&(n.style.opacity=""),d(e,"prev")}))}},{key:"getAllPlayers",value:function(){return this.videoPlayers}},{key:"getSlidePlayerInstance",value:function(e){var t="gvideo"+e,i=this.getAllPlayers();return!(!O(i,t)||!i[t])&&i[t]}},{key:"stopSlideVideo",value:function(e){if(k(e)){var t=e.querySelector(".gvideo-wrapper");t&&(e=t.getAttribute("data-index"))}console.log("stopSlideVideo is deprecated, use slidePlayerPause");var i=this.getSlidePlayerInstance(e);i&&i.playing&&i.pause()}},{key:"slidePlayerPause",value:function(e){if(k(e)){var t=e.querySelector(".gvideo-wrapper");t&&(e=t.getAttribute("data-index"))}var i=this.getSlidePlayerInstance(e);i&&i.playing&&i.pause()}},{key:"playSlideVideo",value:function(e){if(k(e)){var t=e.querySelector(".gvideo-wrapper");t&&(e=t.getAttribute("data-index"))}console.log("playSlideVideo is deprecated, use slidePlayerPlay");var i=this.getSlidePlayerInstance(e);i&&!i.playing&&i.play()}},{key:"slidePlayerPlay",value:function(e){var t;if(!J||null!==(t=this.settings.plyr.config)&&void 0!==t&&t.muted){if(k(e)){var i=e.querySelector(".gvideo-wrapper");i&&(e=i.getAttribute("data-index"))}var n=this.getSlidePlayerInstance(e);n&&!n.playing&&(n.play(),this.settings.autofocusVideos&&n.elements.container.focus())}}},{key:"setElements",value:function(e){var t=this;this.settings.elements=!1;var i=[];e&&e.length&&o(e,(function(e,n){var s=new $(e,t,n),o=s.getConfig(),r=l({},o);r.slideConfig=o,r.instance=s,r.index=n,i.push(r)})),this.elements=i,this.lightboxOpen&&(this.slidesContainer.innerHTML="",this.elements.length&&(o(this.elements,(function(){var e=m(t.settings.slideHTML);t.slidesContainer.appendChild(e)})),this.showSlide(0,!0)))}},{key:"getElementIndex",value:function(e){var t=!1;return o(this.elements,(function(i,n){if(O(i,"node")&&i.node==e)return t=n,!0})),t}},{key:"getElements",value:function(){var e=this,t=[];this.elements=this.elements?this.elements:[],!I(this.settings.elements)&&E(this.settings.elements)&&this.settings.elements.length&&o(this.settings.elements,(function(i,n){var s=new $(i,e,n),o=s.getConfig(),r=l({},o);r.node=!1,r.index=n,r.instance=s,r.slideConfig=o,t.push(r)}));var i=!1;return this.getSelector()&&(i=document.querySelectorAll(this.getSelector())),i?(o(i,(function(i,n){var s=new $(i,e,n),o=s.getConfig(),r=l({},o);r.node=i,r.index=n,r.instance=s,r.slideConfig=o,r.gallery=i.getAttribute("data-gallery"),t.push(r)})),t):t}},{key:"getGalleryElements",value:function(e,t){return e.filter((function(e){return e.gallery==t}))}},{key:"getSelector",value:function(){return!this.settings.elements&&(this.settings.selector&&"data-"==this.settings.selector.substring(0,5)?"*[".concat(this.settings.selector,"]"):this.settings.selector)}},{key:"getActiveSlide",value:function(){return this.slidesContainer.querySelectorAll(".gslide")[this.index]}},{key:"getActiveSlideIndex",value:function(){return this.index}},{key:"getAnimationClasses",value:function(){var e=[];for(var t in this.settings.cssEfects)if(this.settings.cssEfects.hasOwnProperty(t)){var i=this.settings.cssEfects[t];e.push("g".concat(i.in)),e.push("g".concat(i.out))}return e.join(" ")}},{key:"build",value:function(){var e=this;if(this.built)return!1;var t=document.body.childNodes,i=[];o(t,(function(e){e.parentNode==document.body&&"#"!==e.nodeName.charAt(0)&&e.hasAttribute&&!e.hasAttribute("aria-hidden")&&(i.push(e),e.setAttribute("aria-hidden","true"))}));var n=O(this.settings.svg,"next")?this.settings.svg.next:"",s=O(this.settings.svg,"prev")?this.settings.svg.prev:"",l=O(this.settings.svg,"close")?this.settings.svg.close:"",r=this.settings.lightboxHTML;r=m(r=(r=(r=r.replace(/{nextSVG}/g,n)).replace(/{prevSVG}/g,s)).replace(/{closeSVG}/g,l)),document.body.appendChild(r);var d=document.getElementById("glightbox-body");this.modal=d;var g=d.querySelector(".gclose");this.prevButton=d.querySelector(".gprev"),this.nextButton=d.querySelector(".gnext"),this.overlay=d.querySelector(".goverlay"),this.loader=d.querySelector(".gloader"),this.slidesContainer=document.getElementById("glightbox-slider"),this.bodyHiddenChildElms=i,this.events={},h(this.modal,"glightbox-"+this.settings.skin),this.settings.closeButton&&g&&(this.events.close=a("click",{onElement:g,withCallback:function(t,i){t.preventDefault(),e.close()}})),g&&!this.settings.closeButton&&g.parentNode.removeChild(g),this.nextButton&&(this.events.next=a("click",{onElement:this.nextButton,withCallback:function(t,i){t.preventDefault(),e.nextSlide()}})),this.prevButton&&(this.events.prev=a("click",{onElement:this.prevButton,withCallback:function(t,i){t.preventDefault(),e.prevSlide()}})),this.settings.closeOnOutsideClick&&(this.events.outClose=a("click",{onElement:d,withCallback:function(t,i){e.preventOutsideClick||c(document.body,"glightbox-mobile")||u(t.target,".ginner-container")||u(t.target,".gbtn")||c(t.target,"gnext")||c(t.target,"gprev")||e.close()}})),o(this.elements,(function(t,i){e.slidesContainer.appendChild(t.instance.create()),t.slideNode=e.slidesContainer.querySelectorAll(".gslide")[i]})),K&&h(document.body,"glightbox-touch"),this.events.resize=a("resize",{onElement:window,withCallback:function(){e.resize()}}),this.built=!0}},{key:"resize",value:function(){var e=arguments.length>0&&void 0!==arguments[0]?arguments[0]:null;if((e=e||this.activeSlide)&&!c(e,"zoomed")){var t=y(),i=e.querySelector(".gvideo-wrapper"),n=e.querySelector(".gslide-image"),s=this.slideDescription,l=t.width,o=t.height;if(l<=768?h(document.body,"glightbox-mobile"):d(document.body,"glightbox-mobile"),i||n){var r=!1;if(s&&(c(s,"description-bottom")||c(s,"description-top"))&&!c(s,"gabsolute")&&(r=!0),n)if(l<=768)n.querySelector("img");else if(r){var a=s.offsetHeight,u=n.querySelector("img");u.setAttribute("style","max-height: calc(100vh - ".concat(a,"px)")),s.setAttribute("style","max-width: ".concat(u.offsetWidth,"px;"))}if(i){var g=O(this.settings.plyr.config,"ratio")?this.settings.plyr.config.ratio:"";if(!g){var v=i.clientWidth,f=i.clientHeight,p=v/f;g="".concat(v/p,":").concat(f/p)}var m=g.split(":"),x=this.settings.videosWidth,b=this.settings.videosWidth,S=(b=M(x)||-1!==x.indexOf("px")?parseInt(x):-1!==x.indexOf("vw")?l*parseInt(x)/100:-1!==x.indexOf("vh")?o*parseInt(x)/100:-1!==x.indexOf("%")?l*parseInt(x)/100:parseInt(i.clientWidth))/(parseInt(m[0])/parseInt(m[1]));if(S=Math.floor(S),r&&(o-=s.offsetHeight),b>l||S>o||ob){var w=i.offsetWidth,T=i.offsetHeight,C=o/T,k={width:w*C,height:T*C};i.parentNode.setAttribute("style","max-width: ".concat(k.width,"px")),r&&s.setAttribute("style","max-width: ".concat(k.width,"px;"))}else i.parentNode.style.maxWidth="".concat(x),r&&s.setAttribute("style","max-width: ".concat(x,";"))}}}}},{key:"reload",value:function(){this.init()}},{key:"updateNavigationClasses",value:function(){var e=this.loop();d(this.nextButton,"disabled"),d(this.prevButton,"disabled"),0==this.index&&this.elements.length-1==0?(h(this.prevButton,"disabled"),h(this.nextButton,"disabled")):0!==this.index||e?this.index!==this.elements.length-1||e||h(this.nextButton,"disabled"):h(this.prevButton,"disabled")}},{key:"loop",value:function(){var e=O(this.settings,"loopAtEnd")?this.settings.loopAtEnd:null;return e=O(this.settings,"loop")?this.settings.loop:e,e}},{key:"close",value:function(){var e=this;if(!this.lightboxOpen){if(this.events){for(var t in this.events)this.events.hasOwnProperty(t)&&this.events[t].destroy();this.events=null}return!1}if(this.closing)return!1;this.closing=!0,this.slidePlayerPause(this.activeSlide),this.fullElementsList&&(this.elements=this.fullElementsList),this.bodyHiddenChildElms.length&&o(this.bodyHiddenChildElms,(function(e){e.removeAttribute("aria-hidden")})),h(this.modal,"glightbox-closing"),g(this.overlay,"none"==this.settings.openEffect?"none":this.settings.cssEfects.fade.out),g(this.activeSlide,this.settings.cssEfects[this.settings.closeEffect].out,(function(){if(e.activeSlide=null,e.prevActiveSlideIndex=null,e.prevActiveSlide=null,e.built=!1,e.events){for(var t in e.events)e.events.hasOwnProperty(t)&&e.events[t].destroy();e.events=null}var i=document.body;d(Q,"glightbox-open"),d(i,"glightbox-open touching gdesc-open glightbox-touch glightbox-mobile gscrollbar-fixer"),e.modal.parentNode.removeChild(e.modal),e.trigger("close"),T(e.settings.onClose)&&e.settings.onClose();var n=document.querySelector(".gcss-styles");n&&n.parentNode.removeChild(n),e.lightboxOpen=!1,e.closing=null}))}},{key:"destroy",value:function(){this.close(),this.clearAllEvents(),this.baseEvents&&this.baseEvents.destroy()}},{key:"on",value:function(e,t){var i=arguments.length>2&&void 0!==arguments[2]&&arguments[2];if(!e||!T(t))throw new TypeError("Event name and callback must be defined");this.apiEvents.push({evt:e,once:i,callback:t})}},{key:"once",value:function(e,t){this.on(e,t,!0)}},{key:"trigger",value:function(e){var t=this,i=arguments.length>1&&void 0!==arguments[1]?arguments[1]:null,n=[];o(this.apiEvents,(function(t,s){var l=t.evt,o=t.once,r=t.callback;l==e&&(r(i),o&&n.push(s))})),n.length&&o(n,(function(e){return t.apiEvents.splice(e,1)}))}},{key:"clearAllEvents",value:function(){this.apiEvents.splice(0,this.apiEvents.length)}},{key:"version",value:function(){return"3.1.0"}}]),e}();return function(){var e=arguments.length>0&&void 0!==arguments[0]?arguments[0]:{},t=new te(e);return t.init(),t}}));
\ No newline at end of file
diff --git a/site_libs/quarto-contrib/glightbox/lightbox.css b/site_libs/quarto-contrib/glightbox/lightbox.css
new file mode 100644
index 0000000..46432d9
--- /dev/null
+++ b/site_libs/quarto-contrib/glightbox/lightbox.css
@@ -0,0 +1,26 @@
+body:not(.glightbox-mobile) div.gslide div.gslide-description,
+body:not(.glightbox-mobile) div.gslide-description .gslide-title,
+body:not(.glightbox-mobile) div.gslide-description .gslide-desc {
+ color: var(--quarto-body-color);
+ background-color: var(--quarto-body-bg);
+}
+
+body:not(.glightbox-mobile) div.gslide-media {
+ background-color: var(--quarto-body-bg);
+}
+
+.goverlay {
+ background: rgba(0, 0, 0, 0.7);
+}
+
+div.gslide-description .gslide-title {
+ margin-top: 0.25em;
+ margin-bottom: 0.25em;
+ font-weight: 500;
+ font-family: inherit;
+}
+
+div.gslide-description .gslide-desc {
+ padding-bottom: 0.5em;
+ font-family: inherit;
+}
diff --git a/sitemap.xml b/sitemap.xml
index 3b3773b..e3f3c94 100644
--- a/sitemap.xml
+++ b/sitemap.xml
@@ -2,118 +2,118 @@
https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/logbook/posts/2024_09_05/index.html
- 2024-10-18T10:07:55.911Z
+ 2024-10-18T10:20:13.489Z
https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/evaluation/reproduction_success.html
- 2024-10-18T10:07:55.907Z
+ 2024-10-18T10:20:13.489Z
- https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/logbook/posts/2024_10_14/index.html
- 2024-10-18T10:07:55.915Z
+ https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/logbook/posts/2024_10_18/index.html
+ 2024-10-18T10:20:13.497Z
- https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/evaluation/badges.html
- 2024-10-18T10:07:55.907Z
+ https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/logbook/posts/2024_09_06/index.html
+ 2024-10-18T10:20:13.493Z
- https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/quarto_site/reproduction_readme.html
- 2024-10-18T10:07:55.923Z
+ https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/quarto_site/show_reproduction.html
+ 2024-10-18T10:20:13.505Z
- https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/logbook/posts/2024_10_18/index.html
- 2024-10-18T10:07:55.915Z
+ https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/index.html
+ 2024-10-18T10:20:13.489Z
https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/logbook/posts/2024_10_15/index.html
- 2024-10-18T10:07:55.915Z
+ 2024-10-18T10:20:13.493Z
https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/logbook/posts/2024_09_24/index.html
- 2024-10-18T10:07:55.911Z
+ 2024-10-18T10:20:13.493Z
https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/logbook/posts/2024_09_10/index.html
- 2024-10-18T10:07:55.911Z
+ 2024-10-18T10:20:13.493Z
https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/logbook/posts/2024_08_14/index.html
- 2024-10-18T10:07:55.911Z
+ 2024-10-18T10:20:13.489Z
https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/logbook/posts/2024_08_01/index.html
- 2024-10-18T10:07:55.911Z
+ 2024-10-18T10:20:13.489Z
https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/evaluation/reproduction_report.html
- 2024-10-18T10:07:55.907Z
+ 2024-10-18T10:20:13.489Z
https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/logbook/logbook.html
- 2024-10-18T10:07:55.907Z
+ 2024-10-18T10:20:13.489Z
https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/evaluation/reporting.html
- 2024-10-18T10:07:55.907Z
+ 2024-10-18T10:20:13.489Z
https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/evaluation/artefacts.html
- 2024-10-18T10:07:55.907Z
+ 2024-10-18T10:20:13.489Z
https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/evaluation/reflections.html
- 2024-10-18T10:07:55.907Z
+ 2024-10-18T10:20:13.489Z
https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/evaluation/scope.html
- 2024-10-18T10:07:55.907Z
+ 2024-10-18T10:20:13.489Z
https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/quarto_site/study_publication.html
- 2024-10-18T10:07:55.931Z
+ 2024-10-18T10:20:13.509Z
https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/logbook/posts/2024_07_31/index.html
- 2024-10-18T10:07:55.911Z
+ 2024-10-18T10:20:13.489Z
https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/logbook/posts/2024_08_13/index.html
- 2024-10-18T10:07:55.911Z
+ 2024-10-18T10:20:13.489Z
https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/logbook/posts/2024_09_09/index.html
- 2024-10-18T10:07:55.911Z
+ 2024-10-18T10:20:13.493Z
https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/logbook/posts/2024_09_16/index.html
- 2024-10-18T10:07:55.911Z
+ 2024-10-18T10:20:13.493Z
https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/logbook/posts/2024_10_11/index.html
- 2024-10-18T10:07:55.911Z
+ 2024-10-18T10:20:13.493Z
https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/logbook/posts/2024_10_16/index.html
- 2024-10-18T10:07:55.915Z
+ 2024-10-18T10:20:13.497Z
- https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/index.html
- 2024-10-18T10:07:55.907Z
+ https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/quarto_site/reproduction_readme.html
+ 2024-10-18T10:20:13.505Z
- https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/quarto_site/show_reproduction.html
- 2024-10-18T10:07:55.923Z
+ https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/evaluation/badges.html
+ 2024-10-18T10:20:13.489Z
- https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/logbook/posts/2024_09_06/index.html
- 2024-10-18T10:07:55.911Z
+ https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/logbook/posts/2024_10_14/index.html
+ 2024-10-18T10:20:13.493Z
https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/quarto_site/license.html
- 2024-10-18T10:07:55.923Z
+ 2024-10-18T10:20:13.505Z
https://pythonhealthdatascience.github.io/stars-reproduce-johnson-2021/logbook/posts/2024_09_04/index.html
- 2024-10-18T10:07:55.911Z
+ 2024-10-18T10:20:13.489Z
