Merge pull request #57 from DSD-DBS/new_main_version

Version 4.0

.github/workflows/lint.yml

@@ -15,28 +15,20 @@ jobs:
       - uses: actions/setup-python@v4
         with:
           python-version: "3.12"
+      - name: Set up virtual environment
+        run: |-
+          python -m venv .venv
+          source .venv/bin/activate
       - name: Upgrade pip
         run: |-
-          python -m pip install -U pip
+          pip install -U pip
+      - name: Install packages
+        run: |-
+          pip install '.[test]'
+          ls .venv/bin
       - name: Install pre-commit
         run: |-
-          python -m pip install pre-commit types-docutils
+          pip install pre-commit types-docutils
       - name: Run Pre-Commit
         run: |-
           pre-commit run --all-files
-  pylint:
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@v4
-      - uses: actions/setup-python@v4
-        with:
-          python-version: "3.12"
-      - name: Upgrade pip
-        run: |-
-          python -m pip install -U pip
-      - name: Install pylint
-        run: |-
-          python -m pip install pylint
-      - name: Run pylint
-        run: |-
-          pylint -dfixme raillabel || exit $(($? & ~24))

.pre-commit-config.yaml

@@ -25,21 +25,27 @@ repos:
       - id: end-of-file-fixer
       - id: fix-byte-order-marker
       - id: trailing-whitespace
-  - repo: https://github.com/psf/black
-    rev: 24.10.0
-    hooks:
-      - id: black
-  - repo: https://github.com/PyCQA/isort
-    rev: 5.13.2
+
+  - repo: https://github.com/astral-sh/ruff-pre-commit
+    rev: v0.7.0
     hooks:
-      - id: isort
-  - repo: https://github.com/PyCQA/pydocstyle
-    rev: 6.3.0
+      - id: ruff-format
+        name: Run Formatter
+      - id: ruff
+        name: Run Linter
+        args: [ --fix ]
+
+  - repo: https://github.com/pre-commit/mirrors-mypy
+    rev: v1.12.1
     hooks:
-      - id: pydocstyle
-        exclude: '^tests/'
-        additional_dependencies:
-          - pydocstyle[toml]
+    - id: mypy
+      name: Run Type Checker
+      types_or: [python, pyi, toml, yaml]
+      pass_filenames: false
+      args: [raillabel]
+      additional_dependencies:
+        - "pydantic<3.0.0"
+
   - repo: https://github.com/Lucas-C/pre-commit-hooks
     rev: v1.5.5
     hooks:
@@ -83,11 +89,13 @@ repos:
           - LICENSES/.license_header.txt
           - --comment-style
           - '..|  |'
+
   - repo: https://github.com/fsfe/reuse-tool
     rev: v4.0.3
     hooks:
       - id: reuse
+
   - repo: https://github.com/qoomon/git-conventional-commits
-    rev: v2.1.1
+    rev: v2.6.7
     hooks:
       - id: conventional-commits

CHANGELOG.md

@@ -70,7 +70,7 @@ Release
   - ```frame_data``` can only contain ```Num``` instances
   - ```object_data``` can not contain ```Num``` instances anymore
 - Major restructuring of the project directories
-- ```FrameInterval.from_frame_uids()```: create ```FrameIntervals``` by providing a list of frame uids
+- ```FrameInterval.from_frame_ids()```: create ```FrameIntervals``` by providing a list of frame uids
 - ```Object.object_data_pointers()```: generate ```ElementDataPointers```
 - ```Scene.frame_intervals()```, ```Object.frame_intervals()```: generate ```FrameIntervals```
 - ```Object.asdict()``` now provides also frame intervals and object data pointers, if the frames from the scene are provided
@@ -92,3 +92,25 @@ Release
 
 ## 3.3.0
 - Introduce support for Python 3.13
+
+# 4.0.0
+New Major changes to RailLabel! Over the time two major use cases for this package have crystalized. The first one are the users of the data sets published by Deutsche Bahn. The second one are the contractors providing the annotations. Since the two groups have vastly different requirements for this package, we decided to split it accordingly.
+
+If you just want to work with out data, then you are in luck. You can just continue using this package with even more focus on your needs.
+
+If you are building raillabel scenes yourself or want to manipulate the data in a safe way, then the [raillabel_providerkit](https://github.com/DSD-DBS/raillabel-providerkit) is for you. All functionality you may be missing in this new raillabel version will be provided over there with even better APIs.
+
+Functionality, that has been **moved** to the `raillabel_providerkit`:
+- loading annotations in formats other than raillabel itself
+- validating the content of files
+
+Other breaking changes:
+- the `fromdict()` and `asdict()` methods in `raillabel.format` classes have been replaced with `from_json()` and `to_json` respectively
+- `raillabel.format.FrameInterval` fields have been changed by `frame_start -> start` and `frame_end -> end` to make it more concise
+- all uid fields of classes have been removed (like `raillabel.format.Frame.uid`) have been removed to avoid redundant information
+- `raillabel.format.Sensor` has been removed in favor of the different sensor type classes `raillabel.format.Camera`, `raillabel.format.Lidar`, `raillabel.format.Radar` and `raillabel.format.GpsImu`
+- `raillabel.filter()` has been removed in favor of `raillabel.Scene.filter()` with different input arguments
+
+New features:
+- `raillabel.json_format` has been introduced as an interface between the JSON format and the `raillabel` classes
+- `raillabel.scene_builder.SceneBuilder` is now available to easily build scenes for testing purposes

CONTRIBUTING.md

@@ -138,7 +138,7 @@ The key differences are:
     etc.
   - For classes that are not builtin (e.g. `Iterable`),
     `import collections.abc as cabc` and then use them like `cabc.Iterable`.
-  - Use [PEP-604-style unions], e.g. `int | float` instead of
+  - Use [PEP-604-style unions], e.g. `float` instead of
     `t.Union[int, float]`.
   - Use `... | None` (with `None` always as the last union member) instead of
     `t.Optional[...]` and always explicitly annotate where `None` is possible.

README.md

@@ -40,6 +40,44 @@ pip install -e '.[docs,test]'
 pre-commit install
 ```
 
+# Usage
+
+The first step in using `raillabel` is downloading a desired dataset (like [OSDaR23](https://data.fid-move.de/dataset/osdar23)). You can then load any scene by running
+```python
+import raillabel
+
+scene = raillabel.load("path/to/annotation_file.json")
+```
+
+This returns a [`raillabel.Scene`](https://dsd-dbs.github.io/raillabel/code/raillabel.html#raillabel.Scene), which is the root class for the annotations.
+
+If a file is too extensive for your use-case you can filter out certain parts of a scene like this
+```python
+from raillabel.filter import (
+    IncludeObjectTypeFilter,
+    ExcludeAnnotationTypeFilter,
+    StartTimeFilter, ExcludeFrameIdFilter,
+    IncludeAttributesFilter
+)
+
+scene_with_only_trains = scene.filter([IncludeObjectTypeFilter(["rail_vehicle"])])
+
+scene_without_bboxs = scene.filter([ExcludeAnnotationTypeFilter(["bbox"])])
+
+cut_scene_with_only_red_trains = scene.filter([
+    StartTimeFilter("1587349200.004200000"),
+    ExcludeFrameIdFilter([2, 4]),
+    IncludeObjectTypeFilter(["rail_vehicle"]),
+    IncludeAttributesFilter({"color": "red"}),
+])
+```
+An overview of all available filters can be found [here](https://dsd-dbs.github.io/raillabel/code/raillabel.filter.html#module-raillabel.filter).
+
+If you then want to save your changes, then use
+```python
+raillabel.save(cut_scene_with_only_red_trains, "/path/to/target.json")
+```
+
 # Contributing
 
 We'd love to see your bug reports and improvement suggestions! Please take a

docs/source/conf.py

@@ -66,9 +66,7 @@
 rst_epilog = """
 .. |Project| replace:: {project}
 .. |Version| replace:: {version}
-""".format(
-    project=project, version=version
-)
+""".format(project=project, version=version)
 
 
 # -- Options for copy-button -------------------------------------------------