sleap.nn.tracker.components#

Functions/classes used by multiple trackers.

Main types of functions:

Calculate pair-wise instance similarity; used for populating similarity/cost matrix.
Pick matches based on cost matrix.
Other clean-up (e.g., cull instances, connect track breaks).

class sleap.nn.tracker.components.FrameMatches(matches: List[sleap.nn.tracker.components.Match], cost_matrix: numpy.ndarray, unmatched_instances: List[sleap.nn.tracker.components.InstanceType] = NOTHING)[source]#

Calculates (and stores) matches for a frame.

This class encapsulates the logic to generate matches (using a custom matching function) from a cost matrix. One key feature is that it retains additional information, such as whether all the matches were first-choice (i.e., if each instance got the instance it would have matched to if there weren’t other instances).

Typically this will be created using the from_candidate_instances method which creates the cost matrix and then uses the matching function to find matches.

matches#

the list of Match objects.

Type: List[sleap.nn.tracker.components.Match]

cost_matrix#

the cost matrix, shape is (number of untracked instances, number of candidate tracks).

Type: numpy.ndarray

unmatched_instances#

the instances for which we are finding matches.

Type: List[sleap.nn.tracker.components.InstanceType]

classmethod from_candidate_instances(untracked_instances: List[sleap.nn.tracker.components.InstanceType], candidate_instances: List[sleap.nn.tracker.components.InstanceType], similarity_function: Callable, matching_function: Callable, robust_best_instance: float = 1.0)[source]#

Calculates (and stores) matches for a frame from candidate instance.

Parameters

untracked_instances – list of untracked instances in the frame.
candidate_instances – list of instances use as match.
similarity_function – a function that returns the similarity between two instances (untracked and candidate).
matching_function – function used to find the best match from the cost matrix. See the classmethod from_cost_matrix.
robust_best_instance (float) – if the value is between 0 and 1 (excluded), use a robust quantile similarity score for the track. If the value is 1, use the max similarity (non-robust). For selecting a robust score, 0.95 is a good value.

property has_only_first_choice_matches: bool#

Whether all the matches were first-choice.

A match is a ‘first-choice’ for an instance if that instance would have matched to the same track even if there were no other instances.

class sleap.nn.tracker.components.Match(track: sleap.instance.Track, instance: sleap.instance.Instance, score: Optional[float] = None, is_first_choice: bool = False)[source]#: Stores a match between a specific instance and specific track.

sleap.nn.tracker.components.centroid_distance(ref_instance: sleap.nn.tracker.components.InstanceType, query_instance: sleap.nn.tracker.components.InstanceType, cache: dict = {}) → float[source]#

Returns the negative distance between the centroids of two instances.

Uses cache dictionary (created with function so it persists between calls) since without cache this method is significantly slower than others.

sleap.nn.tracker.components.connect_single_track_breaks(frames: List[LabeledFrame], instance_count: int) → List[LabeledFrame][source]#

Merges breaks in tracks by connecting single lost with single new track.

Parameters

frames – The list of LabeledFrame objects with predictions.
instance_count – The maximum number of instances we want per frame.

Returns

Updated list of frames, also modifies frames in place.

sleap.nn.tracker.components.cull_frame_instances(instances_list: List[sleap.nn.tracker.components.InstanceType], instance_count: int, iou_threshold: Optional[float] = None) → List[LabeledFrame][source]#

Removes instances (for single frame) over instance per frame threshold.

Parameters

instances_list – The list of instances for a single frame.
instance_count – The maximum number of instances we want per frame.
iou_threshold – Intersection over Union (IOU) threshold to use when removing overlapping instances over target count; if None, then only use score to determine which instances to remove.

Returns

Updated list of frames, also modifies frames in place.

sleap.nn.tracker.components.cull_instances(frames: List[LabeledFrame], instance_count: int, iou_threshold: Optional[float] = None)[source]#

Removes instances from frames over instance per frame threshold.

Parameters

frames – The list of LabeledFrame objects with predictions.
instance_count – The maximum number of instances we want per frame.
iou_threshold – Intersection over Union (IOU) threshold to use when removing overlapping instances over target count; if None, then only use score to determine which instances to remove.

Returns

None; modifies frames in place.

sleap.nn.tracker.components.first_choice_matching(cost_matrix: numpy.ndarray) → List[Tuple[int, int]][source]#

Returns match indices where each row gets matched to best column.

The means that multiple rows might be matched to the same column.

sleap.nn.tracker.components.greedy_matching(cost_matrix: numpy.ndarray) → List[Tuple[int, int]][source]#: Performs greedy bipartite matching.

sleap.nn.tracker.components.hungarian_matching(cost_matrix: numpy.ndarray) → List[Tuple[int, int]][source]#: Wrapper for Hungarian matching algorithm in scipy.

sleap.nn.tracker.components.instance_iou(ref_instance: sleap.nn.tracker.components.InstanceType, query_instance: sleap.nn.tracker.components.InstanceType, cache: dict = {}) → float[source]#: Computes IOU between bounding boxes of instances.

sleap.nn.tracker.components.instance_similarity(ref_instance: sleap.nn.tracker.components.InstanceType, query_instance: sleap.nn.tracker.components.InstanceType) → float[source]#: Computes similarity between instances.

sleap.nn.tracker.components.nms_fast(boxes, scores, iou_threshold, target_count=None) → List[int][source]#: https://www.pyimagesearch.com/2015/02/16/faster-non-maximum-suppression-python/