sleap.nn.data.pipelines

This module defines high level pipeline configurations from providers/transformers.

The Pipeline class has the capability to create sequences of data I/O and processing operations wrapped in a tf.data-based pipeline.

This allows for convenient ways to configure individual variants of common pipelines, as well as to define training vs inference versions based on the same configurations.

class sleap.nn.data.pipelines.BottomUpPipeline(data_config: sleap.nn.config.data.DataConfig, optimization_config: sleap.nn.config.optimization.OptimizationConfig, confmaps_head: sleap.nn.heads.MultiInstanceConfmapsHead, pafs_head: sleap.nn.heads.PartAffinityFieldsHead)

Pipeline builder for confidence maps + part affinity fields models.

data_config

Data-related configuration.

optimization_config

Optimization-related configuration.

confmaps_head

Instantiated head describing the output confidence maps tensor.

pafs_head

Instantiated head describing the output PAFs tensor.

make_base_pipeline(data_provider: Provider) → sleap.nn.data.pipelines.Pipeline

Create base pipeline with input data only.

Parameters

data_provider – A Provider that generates data examples, typically a LabelsReader instance.

Returns

A Pipeline instance configured to produce input examples.

make_training_pipeline(data_provider: Provider) → sleap.nn.data.pipelines.Pipeline

Create full training pipeline.

Parameters

data_provider – A Provider that generates data examples, typically a LabelsReader instance.

Returns

A Pipeline instance configured to produce all data keys required for training.

Notes

This does not remap keys to model outputs. Use KeyMapper to pull out keys with the appropriate format for the instantiated tf.keras.Model.

make_viz_pipeline(data_provider: Provider, keras_model: tensorflow.python.keras.engine.training.Model) → sleap.nn.data.pipelines.Pipeline

Create visualization pipeline.

Parameters

• data_provider – A Provider that generates data examples, typically a LabelsReader instance.

• keras_model – A tf.keras.Model that can be used for inference.

Returns

A Pipeline instance configured to fetch data and run inference to generate predictions useful for visualization during training.

class sleap.nn.data.pipelines.CentroidConfmapsPipeline(data_config: sleap.nn.config.data.DataConfig, optimization_config: sleap.nn.config.optimization.OptimizationConfig, centroid_confmap_head: sleap.nn.heads.CentroidConfmapsHead)

Pipeline builder for centroid confidence map models.

data_config

Data-related configuration.

optimization_config

Optimization-related configuration.

centroid_confmap_head

Instantiated head describing the output centroid confidence maps tensor.

make_base_pipeline(data_provider: Provider) → sleap.nn.data.pipelines.Pipeline

Create base pipeline with input data only.

Parameters

data_provider – A Provider that generates data examples, typically a LabelsReader instance.

Returns

A Pipeline instance configured to produce input examples.

make_training_pipeline(data_provider: Provider) → sleap.nn.data.pipelines.Pipeline

Create full training pipeline.

Parameters

data_provider – A Provider that generates data examples, typically a LabelsReader instance.

Returns

A Pipeline instance configured to produce all data keys required for training.

Notes

This does not remap keys to model outputs. Use KeyMapper to pull out keys with the appropriate format for the instantiated tf.keras.Model.

make_viz_pipeline(data_provider: Provider, keras_model: tensorflow.python.keras.engine.training.Model) → sleap.nn.data.pipelines.Pipeline

Create visualization pipeline.

Parameters

• data_provider – A Provider that generates data examples, typically a LabelsReader instance.

• keras_model – A tf.keras.Model that can be used for inference.

Returns

A Pipeline instance configured to fetch data and run inference to generate predictions useful for visualization during training.

class sleap.nn.data.pipelines.Pipeline(providers=NOTHING, transformers=NOTHING)

Pipeline composed of providers and transformers.

providers

A single or a list of data providers.

transformers

A single or a list of transformers.

append(other: Union[Pipeline, Transformer, List[Transformer]])

Append one or more blocks to this pipeline instance.

Parameters

other – A single Pipeline, Transformer or list of `Transformer`s to append to the end of this pipeline.

Raises

ValueError – If blocks provided are not a Pipeline, Transformer or list of `Transformer`s.

classmethod from_blocks(blocks: Union[Provider, Transformer, Sequence[Union[Provider, Transformer]]]) → sleap.nn.data.pipelines.Pipeline

Create a pipeline from a sequence of providers and transformers.

Parameters

sequence – List or tuple of providers and transformer instances.

Returns

An instantiated pipeline with all blocks chained.

classmethod from_pipelines(pipelines: Sequence[Pipeline]) → sleap.nn.data.pipelines.Pipeline

Create a new pipeline instance by chaining together multiple pipelines.

Parameters

pipelines – A sequence of Pipeline instances.

Returns

A new Pipeline instance formed by concatenating the individual pipelines.

make_dataset() → tensorflow.python.data.ops.dataset_ops.DatasetV2

Create a dataset instance that generates examples from the pipeline.

Returns

The instantiated tf.data.Dataset pipeline that generates examples with the keys in the output_keys attribute.

property output_keys

Return the keys in examples from a dataset generated from this pipeline.

run() → List[Dict[str, tensorflow.python.framework.ops.Tensor]]

Build and evaluate the pipeline.

Returns

List of example dictionaries after processing the pipeline.

validate_pipeline() → List[str]

Check that all pipeline blocks meet the data requirements.

Returns

The final keys that will be present in each example.

Raises

ValueError – If keys required for a block are dropped at some point in the pipeline.

class sleap.nn.data.pipelines.SingleInstanceConfmapsPipeline(data_config: sleap.nn.config.data.DataConfig, optimization_config: sleap.nn.config.optimization.OptimizationConfig, single_instance_confmap_head: sleap.nn.heads.SingleInstanceConfmapsHead)

Pipeline builder for single-instance confidence map models.

data_config

Data-related configuration.

optimization_config

Optimization-related configuration.

single_instance_confmap_head

Instantiated head describing the output confidence maps tensor.

make_base_pipeline(data_provider: Provider) → sleap.nn.data.pipelines.Pipeline

Create base pipeline with input data only.

Parameters

data_provider – A Provider that generates data examples, typically a LabelsReader instance.

Returns

A Pipeline instance configured to produce input examples.

make_training_pipeline(data_provider: Provider) → sleap.nn.data.pipelines.Pipeline

Create full training pipeline.

Parameters

data_provider – A Provider that generates data examples, typically a LabelsReader instance.

Returns

A Pipeline instance configured to produce all data keys required for training.

Notes

This does not remap keys to model outputs. Use KeyMapper to pull out keys with the appropriate format for the instantiated tf.keras.Model.

make_viz_pipeline(data_provider: Provider, keras_model: tensorflow.python.keras.engine.training.Model) → sleap.nn.data.pipelines.Pipeline

Create visualization pipeline.

Parameters

• data_provider – A Provider that generates data examples, typically a LabelsReader instance.

• keras_model – A tf.keras.Model that can be used for inference.

Returns

A Pipeline instance configured to fetch data and run inference to generate predictions useful for visualization during training.

class sleap.nn.data.pipelines.TopdownConfmapsPipeline(data_config: sleap.nn.config.data.DataConfig, optimization_config: sleap.nn.config.optimization.OptimizationConfig, instance_confmap_head: sleap.nn.heads.CenteredInstanceConfmapsHead)

Pipeline builder for instance-centered confidence map models.

data_config

Data-related configuration.

optimization_config

Optimization-related configuration.

instance_confmap_head

Instantiated head describing the output centered confidence maps tensor.

make_base_pipeline(data_provider: Provider) → sleap.nn.data.pipelines.Pipeline

Create base pipeline with input data only.

Parameters

data_provider – A Provider that generates data examples, typically a LabelsReader instance.

Returns

A Pipeline instance configured to produce input examples.

make_training_pipeline(data_provider: Provider) → sleap.nn.data.pipelines.Pipeline

Create full training pipeline.

Parameters

data_provider – A Provider that generates data examples, typically a LabelsReader instance.

Returns

A Pipeline instance configured to produce all data keys required for training.

Notes

This does not remap keys to model outputs. Use KeyMapper to pull out keys with the appropriate format for the instantiated tf.keras.Model.

make_viz_pipeline(data_provider: Provider, keras_model: tensorflow.python.keras.engine.training.Model) → sleap.nn.data.pipelines.Pipeline

Create visualization pipeline.

Parameters

• data_provider – A Provider that generates data examples, typically a LabelsReader instance.

• keras_model – A tf.keras.Model that can be used for inference.

Returns

A Pipeline instance configured to fetch data and run inference to generate predictions useful for visualization during training.