0.3.0

0.3.0 - Summary Of Changes

Improved Segmentation Map Augmentation (#302)

The segmentation map augmentation was previously previously a wrapper
around heatmap augmentation. This patch introduces independent methods
for segmentation map augmentation. This makes the augmentation of such
inputs faster and more memory efficient. The internal representation (int
instead of floats) also becomes more intuitive.

This improvement leads to some breaking changes. To adapt to the new
version, the following steps should be sufficient for most users:

• Rename all calls of SegmentationMapOnImage to SegmentationMapsOnImage
  (Map -> Maps).
• Rename all calls of SegmentationMapsOnImage.get_arr_int() to
  SegmentationMapsOnImage.get_arr().
• Remove the argument nb_classes from all calls of SegmentationMapsOnImage.
• Remove the argument background_threshold from all
  calls as it is no longer supported.
• Remove the argument draw_foreground_mask from all calls of
  SegmentationMapsOnImage.draw_on_image() as it is no longer supported.
• Ensure that the input array to SegmentationMapsOnImage is always an
  int-like (int, uint or bool). Float arrays are now deprecated.
• Adapt all calls SegmentationMapsOnImage.draw() and
  SegmentationMapsOnImage.draw_on_image(), as both of these now return a
  list of drawn images instead of a single array. (For a segmentation map
  array of shape (H,W,C) they return C drawn images. In most cases C=1,
  so simply call draw()[0] or draw_on_image()[0].)
• Ensure that if SegmentationMapsOnImage.arr is accessed anywhere, the
  respective code can handle the new int32 (H,W,#maps) array form.
  Previously, it was float32 and the channel-axis had the same size as the
  max class id (+1) that could appear in the map.
• Ensure that calls of <augmenter>.augment() or <augmenter>() that
  provide segmentation maps as numpy arrays (i.e. bypassing
  SegmentationMapsOnImage) use the shape (N,H,W,#maps) as
  (N,H,W) is no longer supported.

New RNG System (#375, #408)

numpy 1.17 introduces a new API for random number generation. This patch
adapts imgaug to automatically use the new API if it is available and
fall back to the old one otherwise. To achieve that, the module
imgaug.random is introduced, containing the new standard random number
generator imgaug.random.RNG. You can create a new RNG using a seed value
via RNG(seed) and it will take care of the rest. It supports all sampling
functions that numpy.random.RandomState and numpy.random.Generator
support. This new random number generator is now supposed to be used
wherever previously numpy.random.RandomState would have been used.
(For most users, this shouldn't change anything. Integer seeds are
still supported. If you used RandomState anywhere, that is also still
supported.)

Breaking changes related to this patch:

• imgaug now uses a different seed at each run of the library. Previously,
  a fixed seed was used for each run, leading to the same agumentations. That
  confused some users as it differed from numpy's behaviour.
  The new "dynamic" seed is derived from numpy's seed and hence seeding numpy
  will also lead to imgaug being seeded. (It is not recommended to rely on
  that behaviour as it might be changed in the future. Use
  imgaug.random.seed() to set a custom seed.)
• The constants imgaug.SEED_MIN_VALUE and imgaug.SEED_MAX_VALUE were
  removed. They are now in imgaug.random.
• The constant imgaug.CURRENT_RANDOM_STATE was removed.
  Use imgaug.random.get_global_rng() instead.

Other Changes Related to numpy 1.17 (#302)

numpy 1.17 uses a new implementation of clip(), which turns int64 values
into float64 values. As a result, it is no longer safe to use int64 in
many augmenters and other functions/methods and hence these inputs are now
rejected. This affects at least ReplaceElementwise and thereby Dropout,
CoarseDropout, Salt, Pepper, SaltAndPepper, CoarseSalt,
CoarsePepper and CoarseSaltAndPepper. See the ReadTheDocs documentation
page about dtype support for more details.

In relation to this change, parameters in imgaug.parameters that previously
returned int64 were modified to now return int32 instead. Analogously,
float64 results were changed to float32.

New Augmenters

The following new augmenters were added to the library:

Canny edge detection (#316):

• imgaug.augmenters.edges.Canny. Performs canny edge detection and colorizes
  the resulting binary image in random ways.

Pooling (#317):

• imgaug.augmenters.edges.AveragePooling. Performs average pooling using a
  given kernel size. Very similar to AverageBlur.
• imgaug.augmenters.edges.MaxPooling. Performs maximum pooling using a
  given kernel size.
• imgaug.augmenters.edges.MinPooling. Analogous.
• imgaug.augmenters.edges.MedianPooling. Analogous.

Hue and Saturation (#210, #319):

• imgaug.augmenters.color.WithHueAndSaturation. Apply child augmenters to
  images in HSV colorspace. Automatically accounts for the hue being in
  angular representation.
• imgaug.augmenters.color.AddToHue. Adds a defined value to the hue of each
  pixel in input images.
• imgaug.augmenters.color.AddToSaturation. Adds a defined value to the
  saturation of each pixel in input images.
• imgaug.augmenters.color.MultiplyHueAndSaturation. Multiplies the hue and/or
  saturation of all pixels in input images.
• imgaug.augmenters.color.MultiplyHue. Analogous, affects always only the hue.
• imgaug.augmenters.color.MultiplySaturation. Analogous, affects always only
  the saturation.

Color Quantization (#347):

• imgaug.augmenters.color.UniformColorQuantization. Uniformly splits all
  possible colors into N different ones, then finds for each pixel in an
  image among the N colors the most similar one and replaces that pixel's
  color with the quantized color.
• imgaug.augmenters.color.KMeansColorQuantization. Groups all colors in an
  each into N different ones using k-Means clustering. Then replaces each
  pixel'S color, analogously to UniformColorQuantization.

Voronoi (#348):

• imgaug.augmenters.segmentation.Voronoi. Queries a point sampler to
  generate a large number of (x,y) coordinates on an image. Each such
  coordinate becomes a voronoi cell. All pixels within the voronoi cell
  are replaced by their average color. (Similar to Superpixels, this
  augmenter also supports to only replace p% of all cells with their
  average color.)
• imgaug.augmenters.segmentation.UniformVoronoi. Shortcut to call Voronoi
  with a uniform points sampler. That sampler places N points on an image
  using uniform distributions (i.e. they are randomly spread over the image.)
• imgaug.augmenters.segmentation.RegularGridVoronoi. Shortcut to call
  Voronoi with a regular grid points sampler. That points sampler generates
  coordinate on a regular grid with H rows and W cols. Some of these points
  can be randomly dropped to generate a less regular pattern.
• imgaug.augmenters.segmentation.RelativeRegularGridVoronoi. Same as
  RegularGridVoronoi, but instead of using absolute numbers for H and W,
  they are defined as relative amounts w.r.t. image shapes, leading to more
  rows/cols on larger images.

New Augmentation Functions

One of the long term goals of the library is to move as much augmentation
logic as possible out of Augmenter instances and into functions. This
patch therefore adds several new augmentation functions:

• imgaug.min_pool(). #369
• imgaug.median_pool(). #369
• augmenters.segmentation.segment_voronoi(). #348
• augmenters.flip.fliplr(). #385
• augmenters.flip.flipud(). #385
• augmenters.color.change_colorspace_(). #409
• augmenters.color.change_colorspace_batch_(). #409
• augmenters.arithmetic.add_scalar(). #411
• augmenters.arithmetic.add_elementwise(). #411
• augmenters.arithmetic.replace_elementwise_(). #411
• augmenters.arithmetic.compress_jpg(). #411

Colorspace Changes (#409)

The color space naming within the library had become rather messy in the
past as there were many colorspace-related augmenters, with some of them
not using constants for colorspace names/IDs and others defining their own
ones. This patch introduces a unified colorspace naming system for which the
following constants were added:

• imgaug.CSPACE_RGB
• imgaug.CSPACE_BGR
• imgaug.CSPACE_GRAY
• imgaug.CSPACE_CIE
• imgaug.CSPACE_YCrCb
• imgaug.CSPACE_HSV
• imgaug.CSPACE_HLS
• imgaug.CSPACE_Lab
• imgaug.CSPACE_Luv
• imgaug.CSPACE_YUV
• imgaug.CSPACE_ALL

All colorspace-related augmenters should now support these constants.

Additionally, support for rarely used colorspaces -- mainly CIE, YCrCb,
Luv and YUV -- was previously unverified or non-existent. These colorspaces
are now tested for the underlying transformation functions and should be
supported by most colorspace-related augmenters. (Some augmenters may still
define their own subset of actually sensible colorspaces and only accept
these.)

Setting limits on memory usage of background augmentation (#305, #417)

The methods imap_batches() and imap_batches_unordered() of
imgaug.multicore.Pool have now the new argument output_buffer_size.
The argument set the maximum number of batches that may be handled anywhere
in the augmentation pipeline at a given time (i.e. in the steps "loaded and
waiting", "in augmentation" or "augmented and waiting"). It denotes the
total number of batches over all processes. Setting this argument to
an integer value avoids situations where Pool eats up all the available
memory due to the data loading and augmentation running faster than the
training.

Augmenter.augment_batches() now uses a default value of 10*C
for output_buffer_size, where C is the number of available logical CPU
cores.

Performance Related Changes

The algorithms for Fliplr and Flipud were reworked to be as fast as
possible. In practice this should have no noticeable effects as both augmenters
were already very fast. (#385)

Furthermore, all assert statements within the library were changed from
do_assert() to standard assert statements. This is a bit less secure
(as assert statements can be optimized away), but should have a small
positive impact on the performance. (#387)

Large parts of the library were also refactored to reduce code duplication
and decrease the complexity of many functions. This should make future
improvements easier, but is expected to have a very small negative impact on
the performance due to an increased number of function calls.
It is also expected that numpy 1.17 can make some operations slower. This
is because (a) creating and copying random number generaters has become slower
and (b) clip() overall seems to be slower.

Improved Error Messages (#366, #367, #387)

imgaug uses quite many assert statements and other checks on input data
to fail early instead of late. This is supposed to improve usability, but that
goal was not always reached as many errors had no associated error
messages. This patch changes that. Now, all assert statements and other
checks have an associated error message. This should protect users from having
to wade through the library's code in order to understand the root cause of
errors.

(Almost) All Augmenters Are Now Classes (#396)

Some augmenters were previously defined as functions returning other
augmenters with appropriate settings. This could lead to confusing effects,
where seemingly instantiating an augmenters would lead to the instantiation
of a completely different augmenter. Hence, most of these augmenters were
switched from functions to classes. (The classes are now inheriting from the
previously returned augmenters, i.e. instanceof checks should still work.)
This affects: AdditiveGaussianNoise, AdditiveLaplaceNoise,
AdditivePoissonNoise, Dropout, CoarseDropout, ImpulseNoise,
SaltAndPepper, CoarseSaltAndPepper, Salt, CoarseSalt, Pepper,
CoarsePepper, SimplexNoiseAlpha, FrequencyNoiseAlpha, MotionBlur,
MultiplyHueAndSaturation, MultiplyHue, MultiplySaturation, AddToHue,
AddToSaturation, Grayscale, GammaContrast, SigmoidContrast,
LogContrast, LinearContrast, Sharpen, Emboss, EdgeDetect,
DirectedEdgeDetect, OneOf, AssertLambda, AssertShape, Pad, Crop,
Clouds, Fog and Snowflakes.

Not yet switched are: InColorspace (deprecated),
ContrastNormalization (deprecated), HorizontalFlip (pure alias
for Fliplr), VerticalFlip (pure alias for Flipud)
and Scale (deprecated).

Augmenters are now more robust towards unusual axis-sizes (#428, #433)

Feeding images with height and/or width of 0 or a channel axis of size 0
into augmenters would previously often result in crashes. This was also the
case for input arrays with more than 512 channels. Some of these errors
also included segmentation faults or endlessly hanging programs. Most
augmenters and helper functions were modified to be more robust towards
such unusual inputs and will no longer crash.

It is still good practice to avoid such inputs. Note e.g. that some helper
functions -- like drawing routines -- may still crash. The unittests
corresponding to this change also only cover image data. Using other inputs,
e.g. segmentation maps, might still induce problems.

Other New Functions

The following (public) functions were added to the library (not listing
functions that were already mentioned above):

• Added imgaug.is_np_scalar(). #366
• Added dtypes.normalize_dtypes(). #366
• Added dtypes.normalize_dtype(). #366
• Added dtypes.change_dtypes_(). #366
• Added dtypes.change_dtype_(). #366
• Added dtypes.increase_itemsize_of_dtype(). #366
• Added imgaug.warn() function. #367
• Added imgaug.compute_paddings_to_reach_multiples_of(). #369
• Added imgaug.pad_to_multiples_of(). #369
• Added augmentables.utils.copy_augmentables. #410
• Added validation.convert_iterable_to_string_of_types(). #413
• Added validation.is_iterable_of(). #413
• Added validation.assert_is_iterable_of(). #413
• Added random.supports_new_rng_style(). #375
• Added random.get_global_rng(). #375
• Added random.seed(). #375
• Added random.normalize_generator(). #375
• Added random.normalize_generator_(). #375
• Added random.convert_seed_to_generator(). #375
• Added random.convert_seed_sequence_to_generator(). #375
• Added random.create_pseudo_random_generator_(). #375
• Added random.create_fully_random_generator(). #375
• Added random.generate_seed_(). #375
• Added random.generate_seeds_(). #375
• Added random.copy_generator(). #375
• Added random.copy_generator_unless_global_generator(). #375
• Added random.reset_generator_cache_(). #375
• Added random.derive_generator_(). #375
• Added random.derive_generators_(). #375
• Added random.get_generator_state(). #375
• Added random.set_generator_state_(). #375
• Added random.is_generator_equal_to(). #375
• Added random.advance_generator_(). #375
• Added random.polyfill_integers(). #375
• Added random.polyfill_random(). #375

Other New Classes and Interfaces

The following (public) classes were added (not listing classes that were
already mentioned above):

• Added augmenters.edges.IBinaryImageColorizer. #316
• Added augmenters.edges.RandomColorsBinaryImageColorizer. #316
• Added augmenters.segmentation.IPointsSampler. #348
• Added augmenters.segmentation.RegularGridPointsSampler. #348
• Added augmenters.segmentation.RelativeRegularGridPointsSampler. #348
• Added augmenters.segmentation.DropoutPointsSampler. #348
• Added augmenters.segmentation.UniformPointsSampler. #348
• Added augmenters.segmentation.SubsamplingPointsSampler. #348
• Added testutils.ArgCopyingMagicMock. #413

The image colorization is used for Canny to turn binary images into color
images.
The points samplers are currently used within Voronoi.

Refactorings

Due to fast growth of the library in the past, a significant amount of messy
code had accumulated. To fix that, a lot of time was spend to refactor the code
throughout the whole library to reduce code duplication and improve the
general quality. This also included a rewrite of many outdated docstrings.
There is still quite some mess remaining, but the current state should make
it somewhat easier to add future improvements.

As part of the refactorings, a few humongously large unittests were also
split up into many smaller tests. The library has now around 3000
unique unittests (i.e. each unittest function is counted once, even it is
called many times with different parameters).

Related PRs:

• #302, #319, #328, #329, #330, #331, #332, #333, #334, #335, #336, #351,
  #352, #353, #354, #355, #356, #359, #362, #366, #367, #368, #369, #389,
  #397, #401, #402, #403, #407, #409, #410, #411, #413, #419

Deprecated

The following functions/classes/arguments are now deprecated:

• Function imgaug.augmenters.meta.clip_augmented_image_.
  Use imgaug.dtypes.clip_() or numpy.clip() instead. #398
• Function imgaug.augmenters.meta.clip_augmented_image.
  Use imgaug.dtypes.clip_() or numpy.clip() instead. #398
• Function imgaug.augmenters.meta.clip_augmented_images_.
  Use imgaug.dtypes.clip_() or numpy.clip() instead. #398
• Function imgaug.augmenters.meta.clip_augmented_images.
  Use imgaug.dtypes.clip_() or numpy.clip() instead. #398
• Function imgaug.normalize_random_state.
  Use imgaug.random.normalize_generator instead. #375
• Function imgaug.current_random_state.
  Use imgaug.random.get_global_rng instead. #375
• Function imgaug.new_random_state.
  Use class imgaug.random.RNG instead. #375
• Function imgaug.dummy_random_state.
  Use imgaug.random.RNG(1) instead. #375
• Function imgaug.copy_random_state.
  Use imgaug.random.copy_generator instead.
• Function imgaug.derive_random_state.
  Use imgaug.random.derive_generator_ instead. #375
• Function imgaug.normalize_random_states.
  Use imgaug.random.derive_generators_ instead. #375
• Function imgaug.forward_random_state.
  Use imgaug.random.advance_generator_ instead. #375
• Augmenter imgaug.augmenters.arithmetic.ContrastNormalization.
  Use imgaug.augmenters.contrast.LinearContrast instead. #396
• Argument X in imgaug.augmentables.kps.compute_geometric_median().
  Use argument points instead. #402
• Argument cval in imgaug.pool(), imgaug.avg_pool() and
  imgaug.max_pool(). Use pad_cval instead. #369

Dependencies

The following changes were made to the dependencies of the library:

• Increased minimum version requirement for scikit-image to
  0.14.2. #377, #399
• Changed dependency opencv-python to opencv-python-headless.
  This should improve support for some system without GUIs. #324
• Added dependency pytest-subtests for the library's unittests. #366

conda-forge

The library was added to conda-forge so that it can now be installed via
conda install imgaug. (The conda-forge channel must be added first,
see installation docs or README.) #320 #339

Fixes

• Fixed an issue with Polygon.clip_out_of_image(),
  which would lead to exceptions if a polygon had overlap with an image,
  but not a single one of its points was inside that image plane.
• Fixed multicore methods falsely not accepting
  augmentables.batches.UnnormalizedBatch.
• Rot90 now uses subpixel-based coordinate remapping.
  I.e. any coordinate (x, y) will be mapped to (H-y, x) for a rotation by
  90deg.
  Previously, an integer-based remapping to (H-y-1, x) was used.
  Coordinates are e.g. used by keypoints, bounding boxes or polygons.
• augmenters.arithmetic.Invert
  • [rarely breaking] If min_value and/or max_value arguments were
    set, uint64 is no longer a valid input array dtype for Invert.
    This is due to a conversion to float64 resulting in loss of resolution.
  • Fixed Invert in rare cases restoring dtypes improperly.
• Fixed dtypes.gate_dtypes() crashing if the input was one or more numpy
  scalars instead of numpy arrays or dtypes.
• Fixed augmenters.geometric.PerspectiveTransform producing invalid
  polygons (more often with higher scale values). #338
• Fixed errors caused by external/poly_point_isect_py2py3.py related to
  floating point inaccuracies (changed an epsilon from 1e-10 to 1e-4,
  rounded some floats). #338
• Fixed Superpixels breaking when a sampled n_segments was <=0.
  n_segments is now treated as 1 in these cases.
• Fixed ReplaceElementwise both allowing and disallowing dtype int64. #346
• Fixed BoundingBox.deepcopy() creating only shallow copies of labels. #356
• Fixed dtypes.change_dtypes_() #366
  • Fixed argument round being ignored if input images were a list.
  • Fixed failure if input images were a list and dtypes a single numpy
    dtype function.
• Fixed dtypes.get_minimal_dtype() failing if argument arrays contained
  not exactly two items. #366
• Fixed calls of CloudLayer.get_parameters() resulting in errors. #309
• Fixed SimplexNoiseAlpha and FrequencyNoiseAlpha not handling
  sigmoid argument correctly. #343
• Fixed SnowflakesLayer crashing for grayscale images. #345
• Fixed Affine heatmap augmentation crashing for arrays with more than
  four channels and order!=0. #381
• Fixed an outdated error message in Affine. #381
• Fixed Polygon.clip_out_of_image() crashing if the intersection between
  polygon and image plane was an edge or point. #382
• Fixed Polygon.clip_out_of_image() potentially failing for polygons
  containing two or fewer points. #382
• Fixed Polygon.is_out_of_image() returning wrong values if the image plane
  was fully contained inside the polygon with no intersection between the
  image plane and the polygon edge. #382
• Fixed Fliplr and Flipud using for coordinate-based inputs and image-like
  inputs slightly different conditions for when to actually apply
  augmentations. #385
• Fixed Convolve using an overly restrictive check when validating inputs
  for matrix w.r.t. whether they are callables. The check should now also
  support class methods (and possibly various other callables). #407
• Fixed CropAndPad, Pad and PadToFixedSize still clipping cval samples
  to the uint8. They now clip to the input array's dtype's value range. #407
• Fixed WithColorspace not propagating polygons to child augmenters. #409
• Fixed WithHueAndSaturation not propagating segmentation maps and polygons
  to child augmenters. #409
• Fixed AlphaElementwise to blend coordinates (for keypoints, polygons,
  line strings) on a point-by-point basis following the image's average
  alpha value in the sampled alpha mask of the point's coordinate.
  Previously, the average over the whole mask was used and then either all
  points of the first branch or all of the second branch were used as the
  augmentation output. This also affects SimplexNoiseAlpha and
  FrequencyNoiseAlpha. #410
• Fixed many augmenters and helper functions producing errors if the height,
  width and/or channels of input arrays were exactly 0 or the channels
  were >512. See further above for more details. #433
• Fixed Rot90 not supporting imgaug.ALL. #434
• Fixed PiecewiseAffine possibly generating samples for non-image data
  when using absolute_scale=True that were not well aligned with the
  corresponding images. #437