sleap.nn.tracker.kalman

Module to use Kalman filters for tracking instance identities.

The Kalman filters needs a small number of frames already tracked in order to initialize the filters. Then you can use the module for tracking on the remaining frames.

It’s a good idea to cull the instances (i.e., N best instances per frame) before trying to track with the Kalman filter, since the skeleton fragments can mess up the filters.

sleap.nn.tracker.kalman.get_track_instance_matches(cost_matrix: numpy.ndarray, instances: List[InstanceType], tracks: List[sleap.instance.Track], are_too_close_function: Callable) → List[sleap.nn.tracker.components.Match]

Matches track identities (from filters) to instances in frame.

Algorithm is modified greedy matching.

Standard greedy matching: 0. Start with list of all possible matches, sorted by ascending cost. 1. Find the match with minimum cost. 2. Use this match. 3. Remove other matches from list with same row or same column. 4. Go back to (1).

The algorithm implemented here replaces step (2) with this:

2’. If the instance for the match would have preferred a track which

was already matched by another instance, then only use the match under consideration now if these instances aren’t “too close”.

Whenever the greedy matching would assign an instance to a track that wasn’t its first choice (i.e., another instance was a better match for the track that would have been our current instance’s first choice), then we make sure that the two instances aren’t too close together.

The upshot is that if two nearby instances are competing for the same track, then the looser won’t be given its second choice (since it’s more likely to be a redundant instance), but if nearby instances both match best to distinct tracks, both matches are used.

What counts as “too close” is determined by the are_too_close_function argument, which should have this signature:

are_too_close_function(Instance, Instance) -> bool

sleap.nn.tracker.kalman.match_dict_from_match_function(cost_matrix: numpy.ndarray, row_items: List[Any], column_items: List[Any], match_function: Callable, key_by_column: bool = True) → Dict[Any, Any]

Dict keys are from column (tracks), values are from row (instances).

If multiple rows (instances) match on the same column (track), then dict will just contain the best match.

sleap.nn.tracker.kalman.remove_second_bests_from_cost_matrix(cost_matrix: numpy.ndarray, thresh: float, invalid_val: float = nan) → numpy.ndarray

Removes unclear matches from cost matrix.

If the best match for a given track is too close to the second best match, then this will clear all the matches for that track (and ensure that any instance where that track was the best match won’t be matched to another track).

It removes the matches by setting the appropriate rows/columns to the specified invalid_val (usually nan or inf).

Parameters

• cost_matrix – Cost matrix for matching, lower means better match.

• thresh – Best match must be better than second best + threshold to be valid.

• invalid_val – Value to set invalid rows/columns to.

Returns

cost matrix with invalid matches set to specified invalid value.