Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added substation segementation dataset #2352

Open
wants to merge 73 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
7dff61c
Added substation segementation dataset
rijuld Oct 17, 2024
10637af
resolved bugs
rijuld Oct 21, 2024
2cb0842
a
rijuld Oct 21, 2024
608f76a
Resolved error
rijuld Oct 21, 2024
288e8b1
fixed ruff errors
rijuld Oct 21, 2024
2e9bf83
fixed mypy errors for substation seg py file
rijuld Oct 21, 2024
78c494d
removed more errors
rijuld Oct 21, 2024
75ca32c
resolved ruff errors and mypy errors
rijuld Oct 24, 2024
e2326cc
fixed length and data size along with ruff and mypy errors
rijuld Oct 25, 2024
9832db4
resolved float error
rijuld Oct 25, 2024
ef79cd7
organized imports
rijuld Oct 25, 2024
83f2eb4
changed to float
rijuld Oct 25, 2024
69f5815
resolved mypy errors
rijuld Oct 27, 2024
898e6b3
resolved further tests
rijuld Oct 27, 2024
d14eca6
sorted imports
rijuld Oct 27, 2024
d6ae700
more test coverage
rijuld Oct 30, 2024
8892f0d
ruff format
rijuld Oct 30, 2024
3f135b4
increased test code coverage
rijuld Oct 30, 2024
9a05811
added formatting
rijuld Oct 30, 2024
4e65b04
removed transformations so that I can add them in data module
rijuld Oct 30, 2024
9a9d555
increased underline length
rijuld Oct 30, 2024
3e12e7e
corrected csv row length
rijuld Oct 30, 2024
bbba17b
Update datasets.rst
zijinyin Nov 24, 2024
4fffc1f
Update non_geo_datasets.csv
zijinyin Nov 24, 2024
598c4be
Merge pull request #3 from zijinyin/patch-4
rijuld Nov 25, 2024
15a8881
Merge pull request #1 from zijinyin/patch-2
rijuld Nov 25, 2024
095b7dd
added comment for dataset
rijuld Nov 25, 2024
b503817
changed name to substation
rijuld Nov 25, 2024
f28e30c
added copyright
rijuld Nov 25, 2024
fe1761d
corrected issues
rijuld Nov 25, 2024
c4c3545
added plot and tests
rijuld Nov 25, 2024
1817132
removed pytest
rijuld Nov 25, 2024
28377f8
ruff format
rijuld Nov 25, 2024
5af4e0f
Merge branch 'main' into main
rijuld Nov 26, 2024
a3b95ba
added extract function
rijuld Dec 2, 2024
1216da4
added import
rijuld Dec 2, 2024
b0c3c90
Merge branch 'main' into main
rijuld Dec 2, 2024
545ff66
added datamodule
rijuld Dec 5, 2024
4a6e349
addressed few comments
rijuld Jan 1, 2025
dcc98ef
changed image size
rijuld Jan 1, 2025
d8147ed
removed argument for image files
rijuld Jan 1, 2025
23adef5
added homepage for dataset
rijuld Jan 1, 2025
14e3e51
added ruff format
rijuld Jan 1, 2025
fe12d52
removed mypy errors
rijuld Jan 1, 2025
337b002
fixed the remaining mypy errors
rijuld Jan 1, 2025
4aedf93
Merge branch 'main' into main
rijuld Jan 1, 2025
c7fc761
fixed all the existing tests
rijuld Jan 5, 2025
8e09e8a
added datamodule testing files
rijuld Jan 5, 2025
7626f28
Merge branch 'main' into main
rijuld Jan 8, 2025
d35b435
changed the datatype of bands to list[int] form int
Jan 8, 2025
173a915
changed bands datatype from datamodule
rijuld Jan 8, 2025
cfe800d
changed num of bands variables
rijuld Jan 8, 2025
ebcc36f
Added substation in datamodules.rst and resolved datasets.rst length …
rijuld Jan 8, 2025
f1fcdf0
added substation datamodule in init
rijuld Jan 8, 2025
f00bcd2
chanded the data type of normalizing factor to Any
rijuld Jan 8, 2025
280e32a
[just for testing]
rijuld Jan 8, 2025
743113e
[for testing]
rijuld Jan 8, 2025
85bb9c9
Added parent class
rijuld Jan 8, 2025
de5b337
removed patch size
rijuld Jan 8, 2025
3285346
removed unwanted key
rijuld Jan 8, 2025
d1f062f
resolved errors and tested data module using conf file
rijuld Jan 19, 2025
aebe183
resolved some ruff issues
rijuld Jan 19, 2025
a01c3b4
Merge branch 'main' into main
rijuld Jan 19, 2025
5de36d4
fixed another ruff error
rijuld Jan 19, 2025
7c8c71a
fixed ruff issue
rijuld Jan 19, 2025
6c2b1cb
added more test coverage for extract and verify
rijuld Jan 19, 2025
d4bf9fb
organized imports
rijuld Jan 19, 2025
9a050bd
added more tests for dataset
rijuld Jan 19, 2025
8c918a8
added identity for init values
rijuld Jan 19, 2025
39668ca
ruff format
rijuld Jan 19, 2025
b3af64a
removed pytest command from test file
rijuld Jan 19, 2025
8355860
ruff format
rijuld Jan 19, 2025
5091e16
Merge branch 'main' into main
rijuld Jan 20, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Resolved error
rijuld committed Oct 21, 2024
commit 608f76a8119a7a83ca12d67b113f9cf417eeef5e
2 changes: 1 addition & 1 deletion torchgeo/datasets/__init__.py
Original file line number Diff line number Diff line change
@@ -3,150 +3,150 @@

"""TorchGeo datasets."""

from .advance import ADVANCE
from .agb_live_woody_density import AbovegroundLiveWoodyBiomassDensity
from .agrifieldnet import AgriFieldNet
from .airphen import Airphen
from .astergdem import AsterGDEM
from .benin_cashews import BeninSmallHolderCashews
from .bigearthnet import BigEarthNet
from .biomassters import BioMassters
from .cabuar import CaBuAr
from .cbf import CanadianBuildingFootprints
from .cdl import CDL
from .chabud import ChaBuDx
from .chabud import ChaBuD
from .chesapeake import (
Chesapeake,
ChesapeakeCVPR,
ChesapeakeDC,
ChesapeakeDE,
ChesapeakeMD,
ChesapeakeNY,
ChesapeakePA,
ChesapeakeVA,
ChesapeakeWV,
)
from .cloud_cover import CloudCoverDetection
from .cms_mangrove_canopy import CMSGlobalMangroveCanopy
from .cowc import COWC, COWCCounting, COWCDetection
from .cropharvest import CropHarvest
from .cv4a_kenya_crop_type import CV4AKenyaCropType
from .cyclone import TropicalCyclone
from .deepglobelandcover import DeepGlobeLandCover
from .dfc2022 import DFC2022
from .digital_typhoon import DigitalTyphoon
from .eddmaps import EDDMapS
from .enviroatlas import EnviroAtlas
from .errors import DatasetNotFoundError, DependencyNotFoundError, RGBBandsMissingError
from .esri2020 import Esri2020
from .etci2021 import ETCI2021
from .eudem import EUDEM
from .eurocrops import EuroCrops
from .eurosat import EuroSAT, EuroSAT100, EuroSATSpatial
from .fair1m import FAIR1M
from .fire_risk import FireRisk
from .forestdamage import ForestDamage
from .ftw import FieldsOfTheWorld
from .gbif import GBIF
from .geo import (
GeoDataset,
IntersectionDataset,
NonGeoClassificationDataset,
NonGeoDataset,
RasterDataset,
UnionDataset,
VectorDataset,
)
from .geonrw import GeoNRW
from .gid15 import GID15
from .globbiomass import GlobBiomass
from .idtrees import IDTReeS
from .inaturalist import INaturalist
from .inria import InriaAerialImageLabeling
from .iobench import IOBench
from .l7irish import L7Irish
from .l8biome import L8Biome
from .landcoverai import LandCoverAI, LandCoverAI100, LandCoverAIBase, LandCoverAIGeo
from .landsat import (
Landsat,
Landsat1,
Landsat2,
Landsat3,
Landsat4MSS,
Landsat4TM,
Landsat5MSS,
Landsat5TM,
Landsat7,
Landsat8,
Landsat9,
)
from .levircd import LEVIRCD, LEVIRCDBase, LEVIRCDPlus
from .loveda import LoveDA
from .mapinwild import MapInWild
from .millionaid import MillionAID
from .mmearth import MMEarth
from .naip import NAIP
from .nasa_marine_debris import NASAMarineDebris
from .nccm import NCCM
from .nlcd import NLCD
from .openbuildings import OpenBuildings
from .oscd import OSCD
from .pastis import PASTIS
from .substation_seg import SubstationDataset
from .patternnet import PatternNet
from .potsdam import Potsdam2D
from .prisma import PRISMA
from .quakeset import QuakeSet
from .reforestree import ReforesTree
from .resisc45 import RESISC45
from .rwanda_field_boundary import RwandaFieldBoundary
from .seasonet import SeasoNet
from .seco import SeasonalContrastS2
from .sen12ms import SEN12MS
from .sentinel import Sentinel, Sentinel1, Sentinel2
from .skippd import SKIPPD
from .skyscript import SkyScript
from .so2sat import So2Sat
from .south_africa_crop_type import SouthAfricaCropType
from .south_america_soybean import SouthAmericaSoybean
from .spacenet import (
SpaceNet,
SpaceNet1,
SpaceNet2,
SpaceNet3,
SpaceNet4,
SpaceNet5,
SpaceNet6,
SpaceNet7,
SpaceNet8,
)
from .splits import (
random_bbox_assignment,
random_bbox_splitting,
random_grid_cell_assignment,
roi_split,
time_series_split,
)
from .ssl4eo import SSL4EO, SSL4EOL, SSL4EOS12
from .ssl4eo_benchmark import SSL4EOLBenchmark
from .sustainbench_crop_yield import SustainBenchCropYield
from .ucmerced import UCMerced
from .usavars import USAVars
from .utils import (
BoundingBox,
concat_samples,
merge_samples,
stack_samples,
unbind_samples,
)
from .vaihingen import Vaihingen2D
from .vhr10 import VHR10
from .western_usa_live_fuel_moisture import WesternUSALiveFuelMoisture
from .xview import XView2
from .zuericrop import ZueriCrop

__all__ = (
# GeoDataset

Check failure on line 149 in torchgeo/datasets/__init__.py

GitHub Actions / ruff

Ruff (I001)

torchgeo/datasets/__init__.py:6:1: I001 Import block is un-sorted or un-formatted
'AbovegroundLiveWoodyBiomassDensity',
'AgriFieldNet',
'Airphen',

Unchanged files with check annotations Beta

import os
import shutil

Check failure on line 2 in tests/datasets/test_substation_seg.py

GitHub Actions / ruff

Ruff (F401)

tests/datasets/test_substation_seg.py:2:8: F401 `shutil` imported but unused
import pytest
import numpy as np
import torch
from pathlib import Path
from pytest import MonkeyPatch
from torchgeo.datasets import SubstationDataset, DatasetNotFoundError
class TestSubstationDataset:

Check failure on line 12 in tests/datasets/test_substation_seg.py

GitHub Actions / ruff

Ruff (I001)

tests/datasets/test_substation_seg.py:1:1: I001 Import block is un-sorted or un-formatted
@pytest.fixture(
params=[
{
}
]
)
def dataset(self, monkeypatch: MonkeyPatch, tmp_path: Path, request) -> SubstationDataset:

Check failure on line 24 in tests/datasets/test_substation_seg.py

GitHub Actions / ruff

Ruff (ANN001)

tests/datasets/test_substation_seg.py:24:65: ANN001 Missing type annotation for function argument `request`
args = lambda: None

Check failure on line 25 in tests/datasets/test_substation_seg.py

GitHub Actions / ruff

Ruff (E731)

tests/datasets/test_substation_seg.py:25:9: E731 Do not assign a `lambda` expression, use a `def`
args.data_dir = tmp_path
args.in_channels = 4
args.use_timepoints = False
def test_already_downloaded(self, tmp_path: Path) -> None:
# Test to ensure dataset initialization doesn't download if data already exists
args = lambda: None

Check failure on line 68 in tests/datasets/test_substation_seg.py

GitHub Actions / ruff

Ruff (E731)

tests/datasets/test_substation_seg.py:68:9: E731 Do not assign a `lambda` expression, use a `def`
args.data_dir = tmp_path
args.in_channels = 4
args.use_timepoints = False
os.makedirs(os.path.join(tmp_path, 'image_stack'))
os.makedirs(os.path.join(tmp_path, 'mask'))
dataset = SubstationDataset(args, image_files=[])

Check failure on line 82 in tests/datasets/test_substation_seg.py

GitHub Actions / ruff

Ruff (F841)

tests/datasets/test_substation_seg.py:82:9: F841 Local variable `dataset` is assigned to but never used
assert os.path.exists(os.path.join(tmp_path, 'image_stack'))
assert os.path.exists(os.path.join(tmp_path, 'mask'))
def test_not_downloaded(self, tmp_path: Path) -> None:
args = lambda: None

Check failure on line 87 in tests/datasets/test_substation_seg.py

GitHub Actions / ruff

Ruff (E731)

tests/datasets/test_substation_seg.py:87:9: E731 Do not assign a `lambda` expression, use a `def`
args.data_dir = tmp_path
args.in_channels = 4
args.use_timepoints = False
# No assertion, just ensuring that the plotting does not throw any exceptions.
def test_corrupted(self, tmp_path: Path) -> None:
args = lambda: None

Check failure on line 106 in tests/datasets/test_substation_seg.py

GitHub Actions / ruff

Ruff (E731)

tests/datasets/test_substation_seg.py:106:9: E731 Do not assign a `lambda` expression, use a `def`
args.data_dir = tmp_path
args.in_channels = 4
args.use_timepoints = False
import os

Check failure on line 1 in torchgeo/datasets/substation_seg.py

GitHub Actions / ruff

Ruff (D100)

torchgeo/datasets/substation_seg.py:1:1: D100 Missing docstring in public module
from collections.abc import Callable, Sequence
from typing import ClassVar