sleap.info.feature_suggestions

Module for generating lists of frames using frame features, pca, kmeans, etc.

class sleap.info.feature_suggestions.FrameGroupSet(method: str, item_group: Dict[FrameItem, int] = NOTHING, group_data: Dict[int, dict] = NOTHING, groupset_data: Dict = NOTHING)

Class for a set of groups of FrameItem objects.

Each item can have at most one group; each group is represented as an int.

method

Label for the method used to generate group set.

Type:

str

item_group

Dictionary which maps each item to its group.

Type:

Dict[sleap.info.feature_suggestions.FrameItem, int]

group_data

Dictionary of any extra data for each group; keys are group ids, values are dictionaries of data.

Type:

Dict[int, dict]

groupset_data

Dictionary for any data about the entire set of groups.

Type:

Dict

property all_items

Gets list of all items.

append_to_group(group: int, item: FrameItem)

Adds item to group.

extend_group_items(group: int, item_list: List[FrameItem])

Adds all items in list to group.

get_item_group(item: FrameItem)

Returns group that contain item.

property groups

Iterate over groups, yielding group and list of items.

sample(per_group: int, unique_samples: bool = True)

Returns new FrameGroupSet with groups sampled from current groups.

Note that the order of items in the new groups will not match order of items in the groups from which samples are drawn.

Parameters:

• per_group – The number of samples to take from each group.

• unique_samples – Whether to ensure that there are no shared items in the resulting groups.

Returns:

New FrameGroupSet.

class sleap.info.feature_suggestions.FrameItem(video: Video, frame_idx: int)

Just a simple wrapper for (video, frame_idx), plus method to get image.

class sleap.info.feature_suggestions.ItemStack(items: List = NOTHING, data: ndarray | None = None, ownership: List[tuple] | None = None, meta: List = NOTHING, group_sets: List[FrameGroupSet] = NOTHING)

Container for items, each item can “own” one or more rows of data.

items

The list of items

Type:

List

data

An ndarray with rows of data corresponding to items.

Type:

numpy.ndarray | None

ownership

List which specifies which rows of data correspond to which items.

Type:

List[tuple] | None

meta

List which stores metadata about each operation on stack.

Type:

List

group_sets

List of GroupSets of items.

Type:

List[sleap.info.feature_suggestions.FrameGroupSet]

brisk_bag_of_features(brisk_threshold=40, vocab_size=20)

Transform data using bag of features based on brisk features.

property current_groupset

Gets current (most recent) group set.

extend_ownership(ownership, row_count)

Extends an ownership list with number of rows owned by next item.

flatten()

Flattens each row of data to 1-d array.

get_all_items_from_group()

Sets items for Stack to all items from current GroupSet.

get_item_data(item)

Returns rows of data which belong to item.

get_item_data_idxs(item)

Returns indexes of rows in data which belong to item.

get_raw_images(scale=0.5)

Sets data to raw image for each FrameItem.

hog_bag_of_features(brisk_threshold=40, vocab_size=20)

Transforms data into bag of features vector of hog descriptors.

kmeans(n_clusters: int)

Adds GroupSet using k-means clustering on data.

make_sample_group(videos: List[Video], samples_per_video: int, sample_method: str = 'stride')

Adds GroupSet by sampling frames from each video.

pca(n_components: int)

Transforms data by applying PCA.

sample_groups(samples_per_group: int)

Adds GroupSet by sampling items from current GroupSet.

class sleap.info.feature_suggestions.ParallelFeaturePipeline(pipeline: FeatureSuggestionPipeline, videos_as_dicts: List[Dict])

Enables easy per-video pipeline parallelization for feature suggestions.

Create a FeatureSuggestionPipeline with the desired parameters, and then call ParallelFeaturePipeline.run() with the pipeline and the list of videos to process in parallel. This will take care of serializing the videos, running the pipelines in a process pool, and then deserializing the results back into a single list of SuggestionFrame objects.

get(video_idx)

Apply pipeline to single video by idx. Can be called in process.

classmethod make(pipeline, videos)

Make class object from pipeline and list of videos.

classmethod run(pipeline, videos, parallel=True)

Runs pipeline on all videos in parallel and returns suggestions.

classmethod tuples_to_suggestions(tuples, videos)

Converts serialized data from processes back into SuggestionFrames.