sleap.nn.data.identity

Utilities for generating data for track identity models.

class sleap.nn.data.identity.ClassMapGenerator(sigma: float = 2.0, output_stride: int = 1, centroids: bool = False, class_map_threshold: float = 0.2)

Transformer to generate class maps from track indices.

sigma

Standard deviation of the 2D Gaussian distribution sampled to generate confidence maps for masking the identity maps. This defines the spread in units of the input image’s grid, i.e., it does not take scaling in previous steps into account.

Type:

float

output_stride

Relative stride of the generated maps. This is effectively the reciprocal of the output scale, i.e., increase this to generate maps that are smaller than the input images.

Type:

int

centroids

If True, generate masking confidence maps for centroids rather than instance points.

Type:

bool

class_map_threshold

Minimum confidence map value below which map values will be replaced with zeros.

Type:

float

property input_keys: List[str]

Return the keys that incoming elements are expected to have.

property output_keys: List[str]

Return the keys that outgoing elements will have.

transform_dataset(input_ds: DatasetV2) → DatasetV2

Create a dataset that contains the generated class identity maps.

Parameters:

input_ds – A dataset with elements that contain the keys "image", "track_inds", "n_tracks" and either "instances" or "centroids" depending on whether the centroids attribute is set to True.

Returns:

A tf.data.Dataset with the same keys as the input, as well as a "class_maps" key containing the generated class maps.

class sleap.nn.data.identity.ClassVectorGenerator

Transformer to generate class probability vectors from track indices.

property input_keys: List[str]

Return the keys that incoming elements are expected to have.

property output_keys: List[str]

Return the keys that outgoing elements will have.

transform_dataset(input_ds: DatasetV2) → DatasetV2

Create a dataset that contains the generated class identity vectors.

Parameters:

input_ds – A dataset with elements that contain the keys`”track_inds”` and "n_tracks".

Returns:

A tf.data.Dataset with the same keys as the input, as well as a "class" key containing the generated class vectors.

sleap.nn.data.identity.make_class_maps(confmaps: Tensor, class_inds: Tensor, n_classes: int, threshold: float = 0.2) → Tensor

Generate identity class maps using instance-wise confidence maps.

This is useful for making class maps defined on local neighborhoods around the peaks.

Parameters:

• confmaps – Confidence maps for the same points as the offset maps as a tf.Tensor of shape (grid_height, grid_width, n_instances) and dtype tf.float32. This can be generated by sleap.nn.data.confidence_maps.make_confmaps.

• class_inds – Class indices as tf.int32 tensor of shape (n_instances).

• n_classes – Integer number of maximum classes.

• threshold – Minimum confidence map value below which map values will be replaced with zeros.

Returns:

The class maps with shape (grid_height, grid_width, n_classes) and dtype tf.float32 where each channel will be a binary mask with 1 where the instance confidence maps were higher than the threshold.

Notes

Pixels that have confidence map values from more than one animal will have the class vectors weighed by the relative contribution of each instance.

See also: make_class_vectors, sleap.nn.data.confidence_maps.make_confmaps

sleap.nn.data.identity.make_class_vectors(class_inds: Tensor, n_classes: int) → Tensor

Make a binary class vectors from class indices.

Parameters:

• class_inds – Class indices as tf.Tensor of dtype tf.int32 and shape (n_instances,). Indices of -1 will be interpreted as having no class.

• n_classes – Integer number of maximum classes.

Returns:

A tensor with binary class vectors of shape (n_instances, n_classes) of dtype tf.int32. Instances with no class will have all zeros in their row.

Notes: A class index can be used to represent a track index.