sleap.nn.heads#

Model head definitions for defining model output types.

class sleap.nn.heads.CenteredInstanceConfmapsHead(part_names: List[str], anchor_part: Optional[str] = None, sigma: float = 5.0, output_stride: int = 1, loss_weight: float = 1.0)[source]#

Head for specifying centered instance confidence maps.

part_names#

List of strings specifying the part names associated with channels.

Type

List[str]

anchor_part#

Name of the part to use as an anchor node. If not specified, the bounding box centroid will be used.

Type

Optional[str]

sigma#

Spread of the confidence maps.

Type

float

output_stride#

Stride of the output head tensor. The input tensor is expected to be at the same stride.

Type

int

loss_weight#

Weight of the loss term for this head during optimization.

Type

float

property channels: int#

Return the number of channels in the tensor output by this head.

classmethod from_config(config: sleap.nn.config.model.CenteredInstanceConfmapsHeadConfig, part_names: Optional[List[str]] = None) sleap.nn.heads.CenteredInstanceConfmapsHead[source]#

Create this head from a set of configurations.

config#

A CenteredInstanceConfmapsHeadConfig instance specifying the head parameters.

part_names#

Text name of the body parts (nodes) that the head will be configured to produce. The number of parts determines the number of channels in the output. This must be provided if the part_names attribute of the configuration is not set.

Returns

The instantiated head with the specified configuration options.

class sleap.nn.heads.CentroidConfmapsHead(anchor_part: Optional[str] = None, sigma: float = 5.0, output_stride: int = 1, loss_weight: float = 1.0)[source]#

Head for specifying instance centroid confidence maps.

anchor_part#

Name of the part to use as an anchor node. If not specified, the bounding box centroid will be used.

Type

Optional[str]

sigma#

Spread of the confidence maps.

Type

float

output_stride#

Stride of the output head tensor. The input tensor is expected to be at the same stride.

Type

int

loss_weight#

Weight of the loss term for this head during optimization.

Type

float

property channels: int#

Return the number of channels in the tensor output by this head.

classmethod from_config(config: sleap.nn.config.model.CentroidsHeadConfig) sleap.nn.heads.CentroidConfmapsHead[source]#

Create this head from a set of configurations.

config#

A CentroidsHeadConfig instance specifying the head parameters.

Returns

The instantiated head with the specified configuration options.

class sleap.nn.heads.ClassMapsHead(classes: List[str], sigma: float = 5.0, output_stride: int = 1, loss_weight: float = 1.0)[source]#

Head for specifying class identity maps.

classes#

List of string names of the classes.

Type

List[str]

sigma#

Spread of the class maps around each node.

Type

float

output_stride#

Stride of the output head tensor. The input tensor is expected to be at the same stride.

Type

int

loss_weight#

Weight of the loss term for this head during optimization.

Type

float

property activation: str#

Return the activation function of the head output layer.

property channels: int#

Return the number of channels in the tensor output by this head.

classmethod from_config(config: sleap.nn.config.model.ClassMapsHeadConfig, classes: Optional[List[str]] = None) sleap.nn.heads.ClassMapsHead[source]#

Create this head from a set of configurations.

config#

A ClassMapsHeadConfig instance specifying the head parameters.

classes#

List of string names of the classes that this head will predict. This must be set if the classes attribute of the configuration is not set.

Returns

The instantiated head with the specified configuration options.

class sleap.nn.heads.ClassVectorsHead(classes: List[str], num_fc_layers: int = 1, num_fc_units: int = 64, global_pool: bool = True, output_stride: int = 1, loss_weight: float = 1.0)[source]#

Head for specifying classification heads.

classes#

List of string names of the classes.

Type

List[str]

num_fc_layers#

Number of fully connected layers after flattening input features.

Type

int

num_fc_units#

Number of units (dimensions) in fully connected layers prior to classification output.

Type

int

output_stride#

Stride of the output head tensor. The input tensor is expected to be at the same stride.

Type

int

loss_weight#

Weight of the loss term for this head during optimization.

Type

float

property activation: str#

Return the activation function of the head output layer.

property channels: int#

Return the number of channels in the tensor output by this head.

classmethod from_config(config: sleap.nn.config.model.ClassVectorsHeadConfig, classes: Optional[List[str]] = None) sleap.nn.heads.ClassVectorsHead[source]#

Create this head from a set of configurations.

config#

A ClassVectorsHeadConfig instance specifying the head parameters.

classes#

List of string names of the classes that this head will predict. This must be set if the classes attribute of the configuration is not set.

Returns

The instantiated head with the specified configuration options.

property loss_function: str#

Return the name of the loss function to use for this head.

make_head(x_in: tensorflow.python.framework.ops.Tensor, name: Optional[str] = None) tensorflow.python.framework.ops.Tensor[source]#

Make head output tensor from input feature tensor.

Parameters
  • x_in – An input tf.Tensor.

  • name – If provided, specifies the name of the output layer. If not (the default), uses the name of the head as the layer name.

Returns

A tf.Tensor with the correct shape for the head.

class sleap.nn.heads.Head(output_stride: int = 1, loss_weight: float = 1.0)[source]#

Base class for model output heads.

property activation: str#

Return the activation function of the head output layer.

abstract property channels: int#

Return the number of channels in the tensor output by this head.

property loss_function: str#

Return the name of the loss function to use for this head.

make_head(x_in: tensorflow.python.framework.ops.Tensor, name: Optional[str] = None) tensorflow.python.framework.ops.Tensor[source]#

Make head output tensor from input feature tensor.

Parameters
  • x_in – An input tf.Tensor.

  • name – If provided, specifies the name of the output layer. If not (the default), uses the name of the head as the layer name.

Returns

A tf.Tensor with the correct shape for the head.

class sleap.nn.heads.MultiInstanceConfmapsHead(part_names: List[str], sigma: float = 5.0, output_stride: int = 1, loss_weight: float = 1.0)[source]#

Head for specifying multi-instance confidence maps.

part_names#

List of strings specifying the part names associated with channels.

Type

List[str]

sigma#

Spread of the confidence maps.

Type

float

output_stride#

Stride of the output head tensor. The input tensor is expected to be at the same stride.

Type

int

loss_weight#

Weight of the loss term for this head during optimization.

Type

float

property channels: int#

Return the number of channels in the tensor output by this head.

classmethod from_config(config: sleap.nn.config.model.MultiInstanceConfmapsHeadConfig, part_names: Optional[List[str]] = None) sleap.nn.heads.MultiInstanceConfmapsHead[source]#

Create this head from a set of configurations.

config#

A MultiInstanceConfmapsHeadConfig instance specifying the head parameters.

part_names#

Text name of the body parts (nodes) that the head will be configured to produce. The number of parts determines the number of channels in the output. This must be provided if the part_names attribute of the configuration is not set.

Returns

The instantiated head with the specified configuration options.

class sleap.nn.heads.OffsetRefinementHead(part_names: List[str], output_stride: int = 1, sigma_threshold: float = 0.2, loss_weight: float = 1.0)[source]#

Head for specifying offset refinement maps.

part_names#

List of strings specifying the part names associated with channels.

Type

List[str]

sigma_threshold#

Threshold of confidence map values to use for defining the boundary of the offset maps.

Type

float

output_stride#

Stride of the output head tensor. The input tensor is expected to be at the same stride.

Type

int

loss_weight#

Weight of the loss term for this head during optimization.

Type

float

property channels: int#

Return the number of channels in the tensor output by this head.

classmethod from_config(config: Union[sleap.nn.config.model.CentroidsHeadConfig, sleap.nn.config.model.SingleInstanceConfmapsHeadConfig, sleap.nn.config.model.CenteredInstanceConfmapsHeadConfig, sleap.nn.config.model.MultiInstanceConfmapsHeadConfig], part_names: Optional[List[str]] = None, sigma_threshold: float = 0.2, loss_weight: float = 1.0) sleap.nn.heads.OffsetRefinementHead[source]#

Create this head from a set of configurations.

config#

A ConfmapConfig instance specifying the head parameters.

part_names#

Text name of the body parts (nodes) that the head will be configured to produce. The number of parts determines the number of channels in the output. This must be provided if the part_names attribute of the configuration is not set.

sigma_threshold#

Minimum confidence map value below which offsets will be replaced with zeros.

loss_weight#

Weight of the loss associated with this head.

Returns

The instantiated head with the specified configuration options.

class sleap.nn.heads.PartAffinityFieldsHead(edges: Sequence[Tuple[str, str]], sigma: float = 15.0, output_stride: int = 1, loss_weight: float = 1.0)[source]#

Head for specifying multi-instance part affinity fields.

edges#

List of tuples of (source, destination) node names.

Type

Sequence[Tuple[str, str]]

sigma#

Spread of the part affinity fields.

Type

float

output_stride#

Stride of the output head tensor. The input tensor is expected to be at the same stride.

Type

int

loss_weight#

Weight of the loss term for this head during optimization.

Type

float

property channels: int#

Return the number of channels in the tensor output by this head.

classmethod from_config(config: sleap.nn.config.model.PartAffinityFieldsHeadConfig, edges: Optional[Sequence[Tuple[str, str]]] = None) sleap.nn.heads.PartAffinityFieldsHead[source]#

Create this head from a set of configurations.

config#

A PartAffinityFieldsHeadConfig instance specifying the head parameters.

edges#

List of 2-tuples of the form (source_node, destination_node) that define pairs of text names of the directed edges of the graph. This must be set if the edges attribute of the configuration is not set.

Returns

The instantiated head with the specified configuration options.

class sleap.nn.heads.SingleInstanceConfmapsHead(part_names: List[str], sigma: float = 5.0, output_stride: int = 1, loss_weight: float = 1.0)[source]#

Head for specifying single instance confidence maps.

part_names#

List of strings specifying the part names associated with channels.

Type

List[str]

sigma#

Spread of the confidence maps.

Type

float

output_stride#

Stride of the output head tensor. The input tensor is expected to be at the same stride.

Type

int

loss_weight#

Weight of the loss term for this head during optimization.

Type

float

property channels: int#

Return the number of channels in the tensor output by this head.

classmethod from_config(config: sleap.nn.config.model.SingleInstanceConfmapsHeadConfig, part_names: Optional[List[str]] = None) sleap.nn.heads.SingleInstanceConfmapsHead[source]#

Create this head from a set of configurations.

config#

A SingleInstanceConfmapsHeadConfig instance specifying the head parameters.

part_names#

Text name of the body parts (nodes) that the head will be configured to produce. The number of parts determines the number of channels in the output. This must be provided if the part_names attribute of the configuration is not set.

Returns

The instantiated head with the specified configuration options.