-
Notifications
You must be signed in to change notification settings - Fork 2
/
README.Rmd
143 lines (105 loc) · 8.9 KB
/
README.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
---
output: github_document
---
<!-- README.md is generated from README.Rmd. Please edit that file -->
```{r setupfigures, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "man/figures/README-",
out.width = "100%"
)
```
# regions <a href='https://regions.dataobservatory.eu/'><img src='man/figures/logo.png' align="right"/></a>
<!-- badges: start -->
[](https://codecov.io/gh/antaldaniel/regions?branch=master)
[](https://www.repostatus.org/#wip)
[](https://www.gnu.org/licenses/gpl-3.0.en.html)
[](https://cran.r-project.org/package=regions)
[](https://cran.r-project.org/package=regions)"
[](https://cran.r-project.org/package=regions)
[](https://doi.org/10.5281/zenodo.5006056)
[](https://twitter.com/intent/follow?screen_name=ropengov)
[](https://twitter.com/intent/follow?screen_name=antaldaniel)
[](https://github.com/rOpenGov/regions/actions)
<!-- badges: end -->
## Installation
You can install the development version from [GitHub](https://github.com/) with:
```{r install, eval=FALSE}
devtools::install_github("rOpenGov/regions")
```
or the released version from CRAN:
```{r cran, eval=FALSE}
install.packages("regions")
```
You can review the complete package documentation on [regions.dataobservaotry.eu](https://regions.dataobservatory.eu/). If you find any problems with the code, please raise an issue on [Github](https://github.com/rOpenGov/regions). Pull requests are welcome if you agree with the [Contributor Code of Conduct](https://contributor-covenant.org/version/2/0/CODE_OF_CONDUCT.html)
If you use `regions` in your work, please [cite the package](https://doi.org/10.5281/zenodo.5006056).
## Working with Sub-national Statistics
In international comparison, using nationally aggregated indicators often have many disadvantages, which result from the very different levels of homogeneity, but also from the often very limited observation numbers in a cross-sectional analysis. When comparing European countries, a few missing cases can limit the cross-section of countries to around 20 cases which disallows the use of many analytical methods. Working with sub-national statistics has many advantages: the similarity of the aggregation level and high number of observations can allow more precise control of model parameters and errors, and the number of observations grows from 20 to 200-300.
```{r original-map, echo=FALSE, out.width='80%', fig.align='center', fig.cap='The change from national to sub-national level comes with a huge data processing price: internal administrative boundaries, their names, codes codes change very frequently.'}
knitr::include_graphics(
here::here("vignettes", "indicator_with_map.png")
)
```
Yet the change from national to sub-national level comes with a huge data processing price. While national boundaries are relatively stable, with only a handful of changes in each recent decade. The change of national boundaries requires a more-or-less global consensus. But states are free to change their internal administrative boundaries, and they do it with large frequency. This means that the names, identification codes and boundary definitions of sub-national regions change very frequently. Joining data from different sources and different years can be very difficult.
```{r recoded-map, echo=FALSE, out.width='80%', fig.align='center', fig.cap='Our regions R package helps the data processing, validation and imputation of sub-national, regional datasets and their coding.'}
knitr::include_graphics(
here::here("vignettes", "recoded_indicator_with_map.png")
)
```
There are numerous advantages of switching from a national level of the analysis to a sub-national level comes with a huge price in data processing, validation and imputation. The [regions](https://regions.dataobservatory.eu/) package aims to help this process.
This package is an offspring of the [eurostat](https://ropengov.github.io/eurostat/) package on [rOpenGov](https://ropengov.github.io/). It started as a tool to validate and re-code regional Eurostat statistics, but it aims to be a general solution for all sub-national statistics. It will be developed parallel with other rOpenGov packages.
## Sub-national Statistics Have Many Challenges
**Frequent boundary changes**: as opposed to national boundaries, the territorial units, typologies are often change, and this makes the validation and recoding of observation necessary across time. For example, in the European Union, sub-national typologies change about every three years and you have to make sure that you compare the right French region in time, or, if you can make the time-wise comparison at all.
```{r recode, results='asis'}
library(regions)
library(dplyr)
example_df <- data.frame (
geo = c("FR", "DEE32", "UKI3" ,
"HU12", "DED",
"FRK"),
values = runif(6, 0, 100 ),
stringsAsFactors = FALSE )
recode_nuts(dat = example_df,
nuts_year = 2013) %>%
select ( .data$geo, .data$values, .data$code_2013) %>%
knitr::kable()
```
**Hierarchical aggregation and special imputation**: missingness is very frequent in sub-national statistics, because they are created with a serious time-lag compared to national ones, and because they are often not back-casted after boundary changes. You cannot use standard imputation algorithms because the observations are not similarly aggregated or averaged. Often, the information is seemingly missing, and it is present with an obsolete typology code. This is a basic example which shows you how to impute data from a larger territorial unit, such as a national statistic, to lower territorial units:
```{r example}
library(regions)
upstream <- data.frame (
country_code = rep("AU", 2),
year = c(2019:2020),
my_var = c(10,12)
)
downstream <- australia_states
imputed <- impute_down (
upstream_data = upstream,
downstream_data = downstream,
country_var = "country_code",
regional_code = "geo_code",
values_var = "my_var",
time_var = "year" )
knitr::kable(imputed)
```
## Package functionality
* Generic vocabulary translation and joining functions for geographically coded data
* Keeping track of the boundary changes within the European Union between 1999-2024
* Vocabulary translation and joining functions for standardized European Union statistics
* Vocabulary translation for the `ISO-3166-2` based Google data and the European Union
* Imputation functions from higher aggregation hierarchy levels to lower ones, for example from `NUTS1` to `NUTS2` or from `ISO-3166-1` to `ISO-3166-2` (impute down)
* Imputation functions from lower hierarchy levels to higher ones (impute up)
* Aggregation function from lower hierarchy levels to higher ones, for example from NUTS3 to `NUTS1` or from `ISO-3166-2` to `ISO-3166-1` (aggregate; under development)
* Disaggregation functions from higher hierarchy levels to lower ones, again, for example from `NUTS1` to `NUTS2` or from `ISO-3166-1` to `ISO-3166-2` (disaggregate; under development)
We started building an experimental APIs data is running regions regularly and improving known statistical data sources. See: [Digital Music Observatory](https://music.dataobservatory.eu/), [Green Deal Data Observatory](https://greendeal.dataobservatory.eu/), [Economy Data Observatory](https://economy.dataobservatory.eu/).
## Vignettes / Articles
- [Mapping Regional Data, Mapping Metadata Problem](https://regions.dataobservatory.eu/articles/mapping.html)
- [Working With Regional, Sub-National Statistical Products](https://regions.dataobservatory.eu/articles/Regional_stats.html)
- [Validating Your Typology](https://regions.dataobservatory.eu/articles/validation.html)
- [Recoding And Relabelling](https://regions.dataobservatory.eu/articles/recode.html)
## Contributors
Thanks for [\@KKulma](https://github.com/KKulma) for the improved continous integration on Github.
## Code of Conduct
Please note that the regions project is released with a [Contributor Code of Conduct](https://contributor-covenant.org/version/2/0/CODE_OF_CONDUCT.html). By contributing to this project, you agree to abide by its terms.