sleap.gui.overlays.base

Base classes for overlays.

Overlays are used for showing additional visuals on top of a video frame (i.e., a QtVideoPlayer object). Overlay objects are created in the main GUI app, which then automatically calls the add_to_scene for each loaded overlay after drawing a frame (i.e., when user navigates to a new frame or something changes so that current frame must be redrawn).

class sleap.gui.overlays.base.BaseOverlay(labels: sleap.io.dataset.Labels = None, player: sleap.gui.widgets.video.QtVideoPlayer = None)

Abstract base class for overlays.

Most overlays need rely on the Labels from which to get data and need the QtVideoPlayer to which a QGraphicsObject item will be added, so these attributes are included in the base class.

class sleap.gui.overlays.base.DataOverlay(labels: sleap.io.dataset.Labels = None, player: sleap.gui.widgets.video.QtVideoPlayer = None, data: Sequence = None, overlay_class: Optional[Union[ConfMapsPlot, MultiQuiverPlot]] = None)

Base class for confidence maps/part affinity fields overlays.

These overlays use a ModelData class which provides the confidence maps/ part affinity fields for the frame (by running inference with a model). They could easily be modified to use another “data” class, e.g., one which load saved confidence maps/part affinity fields from a file.

data

instance of a class such that you can use data[frame_idx] to get the data (e.g., confmaps) for a given frame.

overlay_class

determines how the data will be shown, i.e., as confidence maps or as a quiver plot (for part affinity fields).

class sleap.gui.overlays.base.ModelData(predictor: sleap.nn.inference.VisualPredictor, result_key: str, video: sleap.io.video.Video, output_scale: float = 1.0, adjust_vals: bool = True)

Sequence-type object which generates predictions for specified frames.