sleap.nn.tracker.components

Functions/classes used by multiple trackers.

Main types of functions:

  1. Calculate pair-wise instance similarity; used for populating similarity/cost matrix.

  2. Pick matches based on cost matrix.

  3. Other clean-up (e.g., cull instances, connect track breaks).

class sleap.nn.tracker.components.FrameMatches(matches: List[sleap.nn.tracker.components.Match], cost_matrix: numpy.ndarray, unmatched_instances: List[InstanceType] = NOTHING)[source]

Calculates (and stores) matches for a frame.

This class encapsulates the logic to generate matches (using a custom matching function) from a cost matrix. One key feature is that it retains additional information, such as whether all the matches were first-choice (i.e., if each instance got the instance it would have matched to if there weren’t other instances).

Typically this will be created using the from_candidate_instances method which creates the cost matrix and then uses the matching function to find matches.

matches

the list of Match objects.

cost_matrix

the cost matrix, shape is (number of untracked instances, number of candidate tracks).

unmatched_instances

the instances for which we are finding matches.

property has_only_first_choice_matches

Whether all the matches were first-choice.

A match is a ‘first-choice’ for an instance if that instance would have matched to the same track even if there were no other instances.

class sleap.nn.tracker.components.Match(track: sleap.instance.Track, instance: sleap.instance.Instance, score: Optional[float] = None, is_first_choice: bool = False)[source]

Stores a match between a specific instance and specific track.

sleap.nn.tracker.components.centroid_distance(ref_instance: InstanceType, query_instance: InstanceType, cache: dict = {}) → float[source]

Returns the negative distance between the centroids of two instances.

Uses cache dictionary (created with function so it persists between calls) since without cache this method is significantly slower than others.

sleap.nn.tracker.components.connect_single_track_breaks(frames: List[LabeledFrame], instance_count: int) → List[sleap.instance.LabeledFrame][source]

Merges breaks in tracks by connecting single lost with single new track.

Parameters
  • frames – The list of LabeledFrame objects with predictions.

  • instance_count – The maximum number of instances we want per frame.

Returns

Updated list of frames, also modifies frames in place.

sleap.nn.tracker.components.cull_frame_instances(instances_list: List[InstanceType], instance_count: int, iou_threshold: Optional[float] = None) → List[sleap.instance.LabeledFrame][source]

Removes instances (for single frame) over instance per frame threshold.

Parameters
  • instances_list – The list of instances for a single frame.

  • instance_count – The maximum number of instances we want per frame.

  • iou_threshold – Intersection over Union (IOU) threshold to use when removing overlapping instances over target count; if None, then only use score to determine which instances to remove.

Returns

Updated list of frames, also modifies frames in place.

sleap.nn.tracker.components.cull_instances(frames: List[LabeledFrame], instance_count: int, iou_threshold: Optional[float] = None)[source]

Removes instances from frames over instance per frame threshold.

Parameters
  • frames – The list of LabeledFrame objects with predictions.

  • instance_count – The maximum number of instances we want per frame.

  • iou_threshold – Intersection over Union (IOU) threshold to use when removing overlapping instances over target count; if None, then only use score to determine which instances to remove.

Returns

None; modifies frames in place.

sleap.nn.tracker.components.first_choice_matching(cost_matrix: numpy.ndarray) → List[Tuple[int, int]][source]

Returns match indices where each row gets matched to best column.

The means that multiple rows might be matched to the same column.

sleap.nn.tracker.components.greedy_matching(cost_matrix: numpy.ndarray) → List[Tuple[int, int]][source]

Performs greedy bipartite matching.

sleap.nn.tracker.components.hungarian_matching(cost_matrix: numpy.ndarray) → List[Tuple[int, int]][source]

Wrapper for Hungarian matching algorithm in scipy.

sleap.nn.tracker.components.instance_iou(ref_instance: InstanceType, query_instance: InstanceType, cache: dict = {}) → float[source]

Computes IOU between bounding boxes of instances.

sleap.nn.tracker.components.instance_similarity(ref_instance: InstanceType, query_instance: InstanceType) → float[source]

Computes similarity between instances.

sleap.nn.tracker.components.nms_fast(boxes, scores, iou_threshold, target_count=None) → List[int][source]

https://www.pyimagesearch.com/2015/02/16/faster-non-maximum-suppression-python/