Skip to content

Commit

Permalink
Update linter config and format code base
Browse files Browse the repository at this point in the history
  • Loading branch information
J535D165 committed Sep 28, 2024
1 parent 1620e80 commit a552978
Show file tree
Hide file tree
Showing 7 changed files with 35 additions and 32 deletions.
23 changes: 10 additions & 13 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,16 +1,13 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-added-large-files
- repo: https://github.com/charliermarsh/ruff-pre-commit
rev: v0.0.272
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.6.0
hooks:
- id: ruff
- repo: https://github.com/psf/black
rev: 23.3.0
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-added-large-files
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.6.8
hooks:
- id: black
- id: ruff
- id: ruff-format
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -200,4 +200,4 @@ Copyright 2020 The ASReview Authors. All rights reserved.
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
limitations under the License.
20 changes: 10 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,9 +41,9 @@ extension can plot or compute the values for such metrics from ASReview
project files. [O'Mara-Eves et al.
(2015)](https://doi.org/10.1186/2046-4053-4-5) provides a comprehensive
overview of different metrics used in the field of actrive learning. Below we
describe the metrics available in the software.
describe the metrics available in the software.

### Recall
### Recall

The recall is the proportion of relevant records that have been found at a
certain point during the screening phase. It is sometimes also called the
Expand All @@ -58,12 +58,12 @@ The confusion matrix consist of the True Positives (TP), False Positives (FP),
True Negatives (TN), and False Negatives (FN). Definitions are provided in the
following table retrieved at a certain recall (r%).

| | Definition | Calculation |
| | Definition | Calculation |
|----------------------|----------------------------------------------------------------------------------------|---------------------------------|
| True Positives (TP) | The number of relevant records found at recall level | Relevant Records * r% |
| True Positives (TP) | The number of relevant records found at recall level | Relevant Records * r% |
| False Positives (FP) | The number of irrelevant records reviewed at recall level | Records Reviewed – TP |
| True Negatives (TN) | The number of irrelevant records correctly not reviewed at recall level | Irrelevant Records – FP |
| False Negatives (FN) | The number of relevant records not reviewed at recall level (missing relevant records) | Relevant Records – TP |
| True Negatives (TN) | The number of irrelevant records correctly not reviewed at recall level | Irrelevant Records – FP |
| False Negatives (FN) | The number of relevant records not reviewed at recall level (missing relevant records) | Relevant Records – TP |

### Work saved over sampling

Expand All @@ -81,7 +81,7 @@ normalize the WSS for class imbalance (denoted as the nWSS). Moreover, Kusa et
al. showed that nWSS is equal to the True Negative Rate (TNR). The TNR is the
proportion of irrelevant records that were correctly not reviewed at level of
recall. The nWSS is useful to compare performance in terms of work saved
across datasets and models while controlling for dataset class imbalance.
across datasets and models while controlling for dataset class imbalance.

The following table provides a hypothetical dataset example:

Expand Down Expand Up @@ -262,11 +262,11 @@ related to the steep recall curve.
Optional arguments for the command line are `--priors` to include prior
knowledge, `--x_absolute` and `--y_absolute` to use absolute axes.

See `asreview plot -h` for all command line arguments.
See `asreview plot -h` for all command line arguments.

### Plotting multiple files
It is possible to show the curves of multiple files in one plot. Use this
syntax (replace `YOUR_ASREVIEW_FILE_1` and `YOUR_ASREVIEW_FILE_2` by the
It is possible to show the curves of multiple files in one plot. Use this
syntax (replace `YOUR_ASREVIEW_FILE_1` and `YOUR_ASREVIEW_FILE_2` by the
asreview_files that you want to include in the plot):

```bash
Expand Down
15 changes: 11 additions & 4 deletions asreviewcontrib/insights/plot.py
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,7 @@ def _plot_recall(
ax = _add_recall_info(ax, labels, x_absolute, y_absolute)

if show_random:
ax = _add_random_curve(ax, labels, x_absolute, y_absolute)
ax = _add_random_curve(ax, labels, x_absolute, y_absolute)

if show_perfect:
ax = _add_perfect_curve(ax, labels, x_absolute, y_absolute)
Expand Down Expand Up @@ -423,16 +423,23 @@ def _add_perfect_curve(ax, labels, x_absolute, y_absolute):
n_docs = len(labels)

# Create x and y arrays for step plot
x = np.arange(0, n_pos_docs + 1) if x_absolute else np.arange(0, n_pos_docs + 1) / n_docs # noqa: E501
y = np.arange(0, n_pos_docs + 1) if y_absolute else np.arange(0, n_pos_docs + 1) / n_pos_docs # noqa: E501
x = (
np.arange(0, n_pos_docs + 1)
if x_absolute
else np.arange(0, n_pos_docs + 1) / n_docs
)
y = (
np.arange(0, n_pos_docs + 1)
if y_absolute
else np.arange(0, n_pos_docs + 1) / n_pos_docs
)

# Plot the stepwise perfect curve
ax.step(x, y, color="grey", where="post")

return ax



def _add_wss_curve(ax, labels, x_absolute=False, y_absolute=False, legend_label=None):
x, y = _wss_values(labels, x_absolute=x_absolute, y_absolute=y_absolute)
ax.step(x, y, where="post", label=legend_label)
Expand Down
1 change: 0 additions & 1 deletion docs/stats_explainer.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@
python docs/stats_explainer.py
"""


import matplotlib.pyplot as plt
import numpy as np

Expand Down
4 changes: 2 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ packages = ["asreviewcontrib"]
[tool.setuptools_scm]
write_to = "asreviewcontrib/insights/_version.py"

[tool.ruff]
[tool.ruff.lint]
select = ["E", "F", "UP", "I", "B"]

[tool.ruff.isort]
[tool.ruff.lint.isort]
force-single-line = true
2 changes: 1 addition & 1 deletion tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@
```
asreview simulate benchmark:van_de_schoot_2017 -s sim_van_de_schoot_2017_stop_if_min.asreview --init_seed 535 --seed 400 --stop_if min
asreview simulate benchmark:van_de_schoot_2017 -s sim_van_de_schoot_2017_stop_if_full.asreview --init_seed 535 --seed 400 --stop_if -1
```
```

0 comments on commit a552978

Please sign in to comment.