sleap.nn.data.resizing

Transformers for image resizing and padding.

class sleap.nn.data.resizing.PointsRescaler(points_key: str = 'predicted_instances', scale_key: str = 'scale', invert: bool = True)

Transformer to apply or invert scaling operations on points.

property input_keys

Return the keys that incoming elements are expected to have.

property output_keys

Return the keys that outgoing elements will have.

transform_dataset(input_ds: tensorflow.python.data.ops.dataset_ops.DatasetV2) → tensorflow.python.data.ops.dataset_ops.DatasetV2

Create a dataset that contains instance cropped data.

class sleap.nn.data.resizing.Resizer(image_key: str = 'image', scale_key: str = 'scale', points_key: Optional[str] = 'instances', scale: float = 1.0, pad_to_stride: int = 1, keep_full_image: bool = False, full_image_key: str = 'full_image')

Data transformer to resize or pad images.

This is useful as a transformation to data streams that require resizing or padding in order to be downsampled or meet divisibility criteria.

image_key

String name of the key containing the images to resize.

scale_key

String name of the key containing the scale of the images.

points_key

String name of the key containing points to adjust for the resizing operation.

scale

Scalar float specifying scaling factor to resize images by.

pad_to_stride

Maximum stride in a model that the images must be divisible by. If > 1, this will pad the bottom and right of the images to ensure they meet this divisibility criteria. Padding is applied after the scaling specified in the scale attribute.

keep_full_image

If True, keeps the (original size) full image in the examples. This is useful for multi-scale inference.

full_image_key

String name of the key containing the full images.

classmethod from_config(config: sleap.nn.config.data.PreprocessingConfig, image_key: str = 'image', scale_key: str = 'scale', pad_to_stride: Optional[int] = None, keep_full_image: bool = False, full_image_key: str = 'full_image', points_key: Optional[str] = 'instances') → sleap.nn.data.resizing.Resizer

Build an instance of this class from its configuration options.

Parameters

• config – An PreprocessingConfig instance with the desired parameters. If config.pad_to_stride is not an explicit integer, the pad_to_stride parameter must be provided.

• image_key – String name of the key containing the images to resize.

• scale_key – String name of the key containing the scale of the images.

• pad_to_stride – An integer specifying the pad_to_stride if config.pad_to_stride is not an explicit integer (e.g., set to None).

• keep_full_image – If True, keeps the (original size) full image in the examples. This is useful for multi-scale inference.

• full_image_key – String name of the key containing the full images.

• points_key – String name of the key containing points to adjust for the resizing operation.

Returns

An instance of this class.

Raises

ValueError – If config.pad_to_stride is not set to an integer and the pad_to_stride argument is not provided.

property input_keys

Return the keys that incoming elements are expected to have.

property output_keys

Return the keys that outgoing elements will have.

transform_dataset(ds_input: tensorflow.python.data.ops.dataset_ops.DatasetV2) → tensorflow.python.data.ops.dataset_ops.DatasetV2

Create a dataset that contains centroids computed from the inputs.

Parameters

ds_input – A dataset with the image specified in the image_key attribute, points specified in the points_key attribute, and the “scale” key for tracking scaling transformations.

Returns

A tf.data.Dataset with elements containing the same images and points with resizing applied.

The “scale” key of the example will be multipled by the scale attribute of this transformer.

If the keep_full_image attribute is True, a key specified by full_image_key will be added with the to the example containing the image before any processing.

sleap.nn.data.resizing.find_padding_for_stride(image_height: int, image_width: int, max_stride: int) → Tuple[int, int]

Compute padding required to ensure image is divisible by a stride.

This function is useful for determining how to pad images such that they will not have issues with divisibility after repeated pooling steps.

Parameters

• image_height – Scalar integer specifying the image height (rows).

• image_width – Scalar integer specifying the image height (columns).

• max_stride – Scalar integer specifying the maximum stride that the image must be divisible by.

Returns

A tuple of (pad_bottom, pad_right), integers with the number of pixels that the image would need to be padded by to meet the divisibility requirement.

sleap.nn.data.resizing.pad_to_stride(image: tensorflow.python.framework.ops.Tensor, max_stride: int) → tensorflow.python.framework.ops.Tensor

Pad an image to meet a max stride constraint.

This is useful for ensuring there is no size mismatch between an image and the output tensors after multiple downsampling and upsampling steps.

Parameters

• image – Single image tensor of shape (height, width, channels).

• max_stride – Scalar integer specifying the maximum stride that the image must be divisible by. This is the ratio between the length of the image and the length of the smallest tensor it is converted to. This is typically 2 ** n_down_blocks, where n_down_blocks is the number of 2-strided reduction layers in the model.

Returns

The input image with 0-padding applied to the bottom and/or right such that the new shape’s height and width are both divisible by max_stride.

sleap.nn.data.resizing.resize_image(image: tensorflow.python.framework.ops.Tensor, scale: tensorflow.python.framework.ops.Tensor) → tensorflow.python.framework.ops.Tensor

Rescale an image by a scale factor.

This function is primarily a convenience wrapper for tf.image.resize that calculates the new shape from the scale factor.

Parameters

• image – Single image tensor of shape (height, width, channels).

• scale – Factor to resize the image dimensions by, specified as either a float scalar or as a 2-tuple of [scale_x, scale_y]. If a scalar is provided, both dimensions are resized by the same factor.

Returns

The resized image tensor of the same dtype but scaled height and width.

See also: tf.image.resize