Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pull] master from OSGeo:master #135

Merged
merged 45 commits into from
Jan 6, 2025
Merged

[pull] master from OSGeo:master #135

merged 45 commits into from
Jan 6, 2025

Conversation

pull[bot]
Copy link

@pull pull bot commented Jan 6, 2025

See Commits and Changes for more details.


Created by pull[bot] (v2.0.0-alpha.1)

Can you help keep this open source service alive? 💖 Please sponsor : )

```
-  .. oo:: FOREIGN_MEMBERS
      :choices: AUTO, ALL, NONE, STAC
      :default: AUTO
      :since: 3.11.0

      Whether and how foreign members at the feature level should be
      processed as OGR fields:

      - ``AUTO`` mode behaves like ``STAC`` mode if a ``stac_version`` member is found at
        the Feature level, otherwise it behaves as ``NONE`` mode.

      - In ``ALL`` mode, all foreign members at the feature level are added.
        Whether to recursively explore nested objects and produce flatten OGR attributes
        or not is decided by the ``FLATTEN_NESTED_ATTRIBUTES`` open option.

      - In ``NONE`` mode, no foreign members at the feature level are added.

      - ``STAC`` mode (Spatio-Temporal Asset Catalog) behaves the same as ``ALL``,
        except content under the ``assets`` member is by default flattened
        as ``assets.{asset_name}.{asset_property}`` fields.
```
This driver is based on the libertiff library for IFD and tag parsing.
It uses in a very controlled (and isolated) way internal libtiff LZW,
PackBits and LERC codecs (even if using external libtiff for the GTiff
driver). For Deflate/GZip, LZMA, ZSTD, it defers to GDAL
cpl_compressor.h API. For JPEG, WEBP, JPEGXL, it uses the corresponding
GDAL drivers to open strip/tile blobs as temporary datasets.

Note that at current time the driver isn't necessarily optimized to
minimize the number of network requests for COG layout.

The driver supports most TIFF formulations, but not necessarily the most
exotic ones that the GTiff libtiff based might support. In particular it
does not support non-power-of-two BitsPerSample value (could be added
with extra coding). On the plus side, JPEGXL is supported as soon as the
JPEGXL driver is available at runtime (contrary to the GTiff driver which requires
using internal libtiff since the tif_jxl.c codec hasn't been upstreamed
to libtiff yet).

As noted in the documentation, no side-car file at all are currently
used by this driver.
…ompression when RasterIO() requests intersect several tiles
e.g.:
```
$ gdal
ERROR 1: gdal: Missing command name.
Usage: gdal <COMMAND> [OPTIONS]
where <COMMAND> is one of:
  - convert:  Convert a dataset (shortcut for 'gdal raster convert' or 'gdal vector convert').
  - info:     Return information on a dataset (shortcut for 'gdal raster info' or 'gdal vector info').
  - pipeline: Execute a pipeline (shortcut for 'gdal raster pipeline' or 'gdal vector pipeline').
  - raster:   Raster commands.
  - vector:   Vector commands.

Try 'gdal --help' for help.

'gdal <FILENAME>' can also be used as a shortcut for 'gdal info <FILENAME>'.
And 'gdal read <FILENAME> ! ...' as a shortcut for 'gdal pipeline <FILENAME> ! ...'.

For more details, consult https://gdal.org/programs/index.html

WARNING: the gdal command is provisionally provided as an alternative interface to GDAL and OGR command line utilities.
The project reserves the right to modify, rename, reorganize, and change the behavior of the utility
until it is officially frozen in a future feature release of GDAL.
```
Examples:
```
$ gdal <TAB><TAB>
convert   info      pipeline  raster    vector
```

```
$ gdal r<TAB>
==>
$ gdal raster
```

```
$ gdal raster<TAB><TAB>
convert    edit       info       pipeline   reproject
```

```
$ gdal raster info -<TAB><TAB>
--approx-stats   -f               --help           --if             --json-usage     --min-max        --no-fl          --no-md          --oo             --stats
--checksum       --format         --hist           --input          --list-mdd       --mm             --no-gcp         --no-nodata      --open-option    --subdataset
--drivers        -h               -i               --input-format   --mdd            --no-ct          --no-mask        --of             --output-format  --version
```

```
$ gdal raster info --of <TAB><TAB>
json  text
```

```
$ gdal raster info --of=<TAB><TAB>
json  text
```

```
$ gdal raster info --of=j<TAB>
==>
$ gdal raster info --of=json
```

```
$ gdal raster convert --of <TAB><TAB>
AAIGrid          CALS             ERS              GSAG             ILWIS            KEA              MFF              OpenFileGDB      R                SQLite           WMTS
ADRG             COG              EXR              GSBG             ISCE             KMLSUPEROVERLAY  MFF2             PAux             Rasterlite       SRTMHGT          XPM
AVIF             CTable2          FIT              GTA              ISIS2            KRO              MRF              PCIDSK           RMF              Terragen         XYZ
BAG              DDS              FITS             GTiff            ISIS3            KTX2             netCDF           PCRaster         ROI_PAC          TileDB           Zarr
BASISU           DTED             GeoRaster        GTX              JP2ECW           LAN              NGW              PDF              RRASTER          USGSDEM          ZMap
BLX              ECW              GIF              HDF4Image        JP2KAK           LCP              NITF             PDS4             RST              VICAR
BMP              EHdr             GPKG             HEIF             JP2OpenJPEG      Leveller         NTv2             PNG              SAGA             VRT
BT               ELAS             GRIB             HF2              JPEG             MBTiles          NULL             PNM              SGI              WEBP
BYN              ENVI             GS7BG            HFA              JPEGXL           MEM              NWT_GRD          PostGISRaster    SIGDEM           WMS
```

```
$ gdal raster convert in.tif out.tif --co <TAB><TAB>
ALPHA=                           ENDIANNESS=                      JXL_EFFORT=                      PIXELTYPE=                       SOURCE_PRIMARIES_RED=            TIFFTAG_TRANSFERRANGE_BLACK=
BIGTIFF=                         GEOTIFF_KEYS_FLAVOR=             JXL_LOSSLESS=                    PREDICTOR=                       SOURCE_WHITEPOINT=               TIFFTAG_TRANSFERRANGE_WHITE=
BLOCKXSIZE=                      GEOTIFF_VERSION=                 LZMA_PRESET=                     PROFILE=                         SPARSE_OK=                       TILED=
[ ... snip ... ]
```

```
$ gdal raster convert in.tif out.tif --co COMP<TAB>
==>
$ gdal raster convert in.tif out.tif --co COMPRESS=
```

```
$ gdal raster convert in.tif out.tif --co COMPRESS=<TAB><TAB>
CCITTFAX3     CCITTRLE      JPEG          LERC          LERC_ZSTD     LZW           PACKBITS      ZSTD
CCITTFAX4     DEFLATE       JXL           LERC_DEFLATE  LZMA          NONE          WEBP
```

```
$ gdal raster convert in.tif out.tif --co TILED=<TAB><TAB>
NO   YES
```

```
$ gdal raster convert in.tif out.tif --co ZLEVEL=<TAB><TAB>
1   10  11  12  2   3   4   5   6   7   8   9
```

```
$ gdal raster convert --of COG --co <TAB><TAB>
ADD_ALPHA=             EXTENT=                JXL_LOSSLESS=          NUM_THREADS=           OVERVIEW_RESAMPLING=   RESAMPLING=            WARP_RESAMPLING=
ALIGNED_LEVELS=        GEOTIFF_VERSION=       LEVEL=                 OVERVIEW_COMPRESS=     OVERVIEWS=             SPARSE_OK=             ZOOM_LEVEL=
BIGTIFF=               JXL_ALPHA_DISTANCE=    MAX_Z_ERROR=           OVERVIEW_COUNT=        PREDICTOR=             STATISTICS=            ZOOM_LEVEL_STRATEGY=
BLOCKSIZE=             JXL_DISTANCE=          MAX_Z_ERROR_OVERVIEW=  OVERVIEW_PREDICTOR=    QUALITY=               TARGET_SRS=
COMPRESS=              JXL_EFFORT=            NBITS=                 OVERVIEW_QUALITY=      RES=                   TILING_SCHEME=
```

```
$ gdal --config <TAB><TAB>
... long list of known configuration options ...
```

```
$ gdal --config AWS_<TAB><TAB>
AWS_ACCESS_KEY_ID=                       AWS_DEFAULT_REGION=                      AWS_REQUEST_PAYER=                       AWS_STS_ENDPOINT=
AWS_CONFIG_FILE=                         AWS_HTTPS=                               AWS_ROLE_ARN=                            AWS_STS_REGION=
AWS_CONTAINER_AUTHORIZATION_TOKEN=       AWS_MAX_KEYS=                            AWS_ROLE_SESSION_NAME=                   AWS_STS_REGIONAL_ENDPOINTS=
AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE=  AWS_NO_SIGN_REQUEST=                     AWS_S3_ENDPOINT=                         AWS_TIMESTAMP=
AWS_CONTAINER_CREDENTIALS_FULL_URI=      AWS_PROFILE=                             AWS_SECRET_ACCESS_KEY=                   AWS_VIRTUAL_HOSTING=
AWS_DEFAULT_PROFILE=                     AWS_REGION=                              AWS_SESSION_TOKEN=                       AWS_WEB_IDENTITY_TOKEN_FILE=
```

```
$ gdal raster reproject --dst-crs
EPSG:      ESRI:      IAU_2015:  IGNF:      NKG:       OGC:
```

```
$ gdal raster reproject --dst-crs EP<TAB>
==>
$ gdal raster reproject --dst-crs EPSG:
```

```
$ gdal raster reproject --dst-crs EPSG:<TAB>
10150 -- MSL UK & Ireland VORF08 depth
10151 -- CD UK & Ireland VORF08 depth
10156 -- ETRS89 + MSL UK & Ireland VORF08 depth
10157 -- ETRS89 + CD UK & Ireland VORF08 depth
[ ... snip ... ]
```

```
$ gdal raster reproject --dst-crs EPSG:432<TAB>
4322 -- WGS 72                  4324 -- WGS 72BE                4326 -- WGS 84                  4327 -- WGS 84 (geographic 3D)  4328 -- WGS 84 (geocentric)     4329 -- WGS 84 (3D)
```

Last but not least: autocompletion of VSI files
```
$ gdal raster info /vsis3/my_bucket/b<TAB><TAB>
/vsis3/my_bucket/byte.tif      /vsis3/my_bucket/byte2.tif
```
…omain

Add a BAND_IMAGERY_METADATA options to papszOptions:

JSON serialized object defining which arrays of the
poRootGroup, indexed by non-X and Y dimensions,
should be mapped as band metadata items in the
band IMAGERY domain.
The object currently accepts 2 members:
- "CENTRAL_WAVELENGTH_UM": Central Wavelength in
  micrometers.
- "FWHM_UM": Full-width half-maximum
  in micrometers.

The value of each member should be an object with the
following members:
- "array": (required) full name of a band parameter
  array.
  Such array must be a one dimensional array, and its
  dimension must be one of the dimensions of the
  array on which the method is called
  (excluding the X and Y dimensons).
- "unit": (optional) unit of the values pointed in
  the above array.
  Can be a literal string or a string of the form
  "${attribute_name}" to point to an attribute for
  the array.
  Accepted values are "um", "micrometer"
  (with UK vs US spelling, singular or plural), "nm",
  "nanometer" (with UK vs US spelling, singular or
  plural)
  If not provided, micrometer is assumed.

Example for EMIT datasets:
```json
{
   "CENTRAL_WAVELENGTH_UM": {
       "array": "/sensor_band_parameters/wavelengths",
       "unit": "${units}"
   },
   "FWHM_UM": {
       "array": "/sensor_band_parameters/fwhm",
       "unit": "${units}"
   }
}
```
rouault added 15 commits January 5, 2025 21:49
Fix CPLFormFilename(absolute_path, ../something, NULL) to strip the relative path
Add LIBERTIFF driver: native thread-safe read-only GeoTIFF reader
/vsicurl?: accept header.<key>=<value> to specify HTTP request headers
GeoJSON: add a FOREIGN_MEMBERS=AUTO/ALL/NONE/STAC open option
Doc: faq.rst: add a paragraph about 3D models
'gdal' CLI: add a warning about experimental status in help message
…ption

VRTProcessedDataset: reduce block size when amount of RAM for temp buffers would be too large
…METADATA

GDALMDArray::AsClassicDataset(): allow to set band IMAGERY metadata domain
Add GDALTranspose2D() for fast 2D matrix transposition
@pull pull bot added the ⤵️ pull label Jan 6, 2025
@pull pull bot merged commit 44ff97a into innFactory:master Jan 6, 2025
1 check passed
@coveralls
Copy link

Pull Request Test Coverage Report for Build 12639806163

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 3400 of 3649 (93.18%) changed or added relevant lines in 46 files are covered.
  • 18506 unchanged lines in 35 files lost coverage.
  • Overall coverage increased (+0.1%) to 70.067%

Changes Missing Coverage Covered Lines Changed/Added Lines %
gcore/gdalallvalidmaskband.cpp 7 8 87.5%
port/cpl_path.cpp 36 37 97.3%
frmts/libertiff/libtiff_codecs.h 37 39 94.87%
gcore/gdalmultidim.cpp 123 125 98.4%
gcore/gdalrasterband.cpp 1 3 33.33%
frmts/gtiff/libtiff/tif_lerc.c 7 10 70.0%
apps/gdalalg_raster_buildvrt.cpp 133 137 97.08%
autotest/cpp/test_gdal.cpp 79 85 92.94%
gcore/tiff_common.cpp 94 100 94.0%
ogr/ogrsf_frmts/geojson/ogrgeojsonreader.cpp 90 96 93.75%
Files with Coverage Reduction New Missed Lines %
port/cpl_vsil_unix_stdio_64.cpp 1 90.75%
port/cpl_conv.cpp 1 77.36%
ogr/ogrlinestring.cpp 1 86.15%
alg/gdalwarpkernel.cpp 1 80.97%
port/cpl_vsil_curl_streaming.cpp 1 86.64%
frmts/gtiff/libgeotiff/geo_tiffp.c 1 97.83%
port/cplstringlist.cpp 1 91.21%
frmts/wms/gdalhttp.cpp 1 84.38%
third_party/fast_float/parse_number.h 1 62.07%
alg/gdalgrid.cpp 1 89.61%
Totals Coverage Status
Change from base Build 12634258852: 0.1%
Covered Lines: 563101
Relevant Lines: 803660

💛 - Coveralls

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants