sleap.nn.data.resizing

Transformers for image resizing and padding.

class sleap.nn.data.resizing.PointsRescaler(points_key: str = 'predicted_instances', scale_key: str = 'scale', invert: bool = True)[source]

Transformer to apply or invert scaling operations on points.

property input_keys

Return the keys that incoming elements are expected to have.

property output_keys

Return the keys that outgoing elements will have.

transform_dataset(input_ds: tensorflow.python.data.ops.dataset_ops.DatasetV2)tensorflow.python.data.ops.dataset_ops.DatasetV2[source]

Create a dataset that contains instance cropped data.

class sleap.nn.data.resizing.Resizer(image_key: str = 'image', scale_key: str = 'scale', points_key: Optional[str] = 'instances', scale: float = 1.0, pad_to_stride: int = 1, keep_full_image: bool = False, full_image_key: str = 'full_image')[source]

Data transformer to resize or pad images.

This is useful as a transformation to data streams that require resizing or padding in order to be downsampled or meet divisibility criteria.

image_key

String name of the key containing the images to resize.

Type

str

scale_key

String name of the key containing the scale of the images.

Type

str

points_key

String name of the key containing points to adjust for the resizing operation.

Type

Optional[str]

scale

Scalar float specifying scaling factor to resize images by.

Type

float

pad_to_stride

Maximum stride in a model that the images must be divisible by. If > 1, this will pad the bottom and right of the images to ensure they meet this divisibility criteria. Padding is applied after the scaling specified in the scale attribute.

Type

int

keep_full_image

If True, keeps the (original size) full image in the examples. This is useful for multi-scale inference.

Type

bool

full_image_key

String name of the key containing the full images.

Type

str

classmethod from_config(config: sleap.nn.config.data.PreprocessingConfig, image_key: str = 'image', scale_key: str = 'scale', pad_to_stride: Optional[int] = None, keep_full_image: bool = False, full_image_key: str = 'full_image', points_key: Optional[str] = 'instances')sleap.nn.data.resizing.Resizer[source]

Build an instance of this class from its configuration options.

Parameters
  • config – An PreprocessingConfig instance with the desired parameters. If config.pad_to_stride is not an explicit integer, the pad_to_stride parameter must be provided.

  • image_key – String name of the key containing the images to resize.

  • scale_key – String name of the key containing the scale of the images.

  • pad_to_stride – An integer specifying the pad_to_stride if config.pad_to_stride is not an explicit integer (e.g., set to None).

  • keep_full_image – If True, keeps the (original size) full image in the examples. This is useful for multi-scale inference.

  • full_image_key – String name of the key containing the full images.

  • points_key – String name of the key containing points to adjust for the resizing operation.

Returns

An instance of this class.

Raises

ValueError – If config.pad_to_stride is not set to an integer and the pad_to_stride argument is not provided.

property input_keys

Return the keys that incoming elements are expected to have.

property output_keys

Return the keys that outgoing elements will have.

transform_dataset(ds_input: tensorflow.python.data.ops.dataset_ops.DatasetV2)tensorflow.python.data.ops.dataset_ops.DatasetV2[source]

Create a dataset that contains centroids computed from the inputs.

Parameters

ds_input – A dataset with the image specified in the image_key attribute, points specified in the points_key attribute, and the “scale” key for tracking scaling transformations.

Returns

A tf.data.Dataset with elements containing the same images and points with resizing applied.

The “scale” key of the example will be multipled by the scale attribute of this transformer.

If the keep_full_image attribute is True, a key specified by full_image_key will be added with the to the example containing the image before any processing.

class sleap.nn.data.resizing.SizeMatcher(image_key: str = 'image', scale_key: str = 'scale', points_key: Optional[str] = 'instances', keep_full_image: bool = False, full_image_key: str = 'full_image', max_image_height: Optional[int] = None, max_image_width: Optional[int] = None)[source]

Data transformer that ensures output images have uniform shape by resizing/padding smaller images.

image_key

String name of the key containing the images to resize.

Type

str

scale_key

String name of the key containing the scale of the images.

Type

str

points_key

String name of the key containing points to adjust for the resizing operation.

Type

Optional[str]

keep_full_image

If True, keeps the (original size) full image in the examples. This is useful for multi-scale inference.

Type

bool

full_image_key

String name of the key containing the full images.

Type

str

max_image_height

int The target height to which all smaller images will be resized/padded to.

Type

int

max_image_width

int The target width to which all smaller images will be resized/padded to.

Type

int

classmethod from_config(config: sleap.nn.config.data.PreprocessingConfig, provider: Optional[Provider] = None, update_config: bool = True, image_key: str = 'image', scale_key: str = 'scale', keep_full_image: bool = False, full_image_key: str = 'full_image', points_key: Optional[str] = 'instances')SizeMatcher[source]

Build an instance of this class from configuration.

Parameters
  • config – An PreprocessingConfig instance with the desired parameters. If config.resize_and_pad_to_target is True and ‘target_height’ / ‘target_width’ are not set, provider needs to be set that implements ‘max_height_and_width’.

  • provider – Data provider.

  • update_config – If True, the input model configuration will be updated with values inferred from other fields.

  • image_key – String name of the key containing the images to resize.

  • scale_key – String name of the key containing the scale of the images.

  • pad_to_stride – An integer specifying the pad_to_stride if config.pad_to_stride is not an explicit integer (e.g., set to None).

  • keep_full_image – If True, keeps the (original size) full image in the examples. This is useful for multi-scale inference.

  • full_image_key – String name of the key containing the full images.

  • points_key – String name of the key containing points to adjust for the resizing operation.

Returns

An instance of this class.

property input_keys

Return the keys that incoming elements are expected to have.

property output_keys

Return the keys that outgoing elements will have.

transform_dataset(ds_input: tensorflow.python.data.ops.dataset_ops.DatasetV2)tensorflow.python.data.ops.dataset_ops.DatasetV2[source]

Transform a dataset with variable size images into one with fixed sizes.

Parameters

ds_input – A dataset with the image specified in the image_key attribute, points specified in the points_key attribute, and the “scale” key for tracking scaling transformations.

Returns

A tf.data.Dataset with elements containing the same images and points of equal size.

If the keep_full_image attribute is True, a key specified by full_image_key will be added to the example containing the image before any processing.

sleap.nn.data.resizing.find_padding_for_stride(image_height: int, image_width: int, max_stride: int)Tuple[int, int][source]

Compute padding required to ensure image is divisible by a stride.

This function is useful for determining how to pad images such that they will not have issues with divisibility after repeated pooling steps.

Parameters
  • image_height – Scalar integer specifying the image height (rows).

  • image_width – Scalar integer specifying the image height (columns).

  • max_stride – Scalar integer specifying the maximum stride that the image must be divisible by.

Returns

A tuple of (pad_bottom, pad_right), integers with the number of pixels that the image would need to be padded by to meet the divisibility requirement.

sleap.nn.data.resizing.pad_to_stride(image: tensorflow.python.framework.ops.Tensor, max_stride: int)tensorflow.python.framework.ops.Tensor[source]

Pad an image to meet a max stride constraint.

This is useful for ensuring there is no size mismatch between an image and the output tensors after multiple downsampling and upsampling steps.

Parameters
  • image – Single image tensor of shape (height, width, channels).

  • max_stride – Scalar integer specifying the maximum stride that the image must be divisible by. This is the ratio between the length of the image and the length of the smallest tensor it is converted to. This is typically 2 ** n_down_blocks, where n_down_blocks is the number of 2-strided reduction layers in the model.

Returns

The input image with 0-padding applied to the bottom and/or right such that the new shape’s height and width are both divisible by max_stride.

sleap.nn.data.resizing.resize_image(image: tensorflow.python.framework.ops.Tensor, scale: tensorflow.python.framework.ops.Tensor)tensorflow.python.framework.ops.Tensor[source]

Rescale an image by a scale factor.

This function is primarily a convenience wrapper for tf.image.resize that calculates the new shape from the scale factor.

Parameters
  • image – Single image tensor of shape (height, width, channels).

  • scale – Factor to resize the image dimensions by, specified as either a float scalar or as a 2-tuple of [scale_x, scale_y]. If a scalar is provided, both dimensions are resized by the same factor.

Returns

The resized image tensor of the same dtype but scaled height and width.

See also: tf.image.resize