sleap.skeleton

Implementation of skeleton data structure and API.

This module implements and API for creating animal skeletons. The goal is to provide a common interface for defining the parts of the animal, their connection to each other, and needed meta-data.

class sleap.skeleton.EdgeType(value)

Type of edge in the skeleton graph.

The skeleton graph can store different types of edges to represent different things. All edges must specify one or more of the following types:

• BODY - these edges represent connections between parts or landmarks.

• SYMMETRY - these edges represent symmetrical relationships between parts (e.g. left and right arms)

class sleap.skeleton.Node(name: str, weight: float = 1.0)

This class represents node in the skeleton graph, i.e., a body part.

Note: Nodes can exist without being part of a skeleton.

name

String name of the node.

Type:

str

weight

Weight of the node (not currently used).

Type:

float

classmethod as_node(node: Union[str, Node]) → Node

Convert given node to Node object (if not already).

static from_names(name_list: str) → List[Node]

Convert list of node names to list of nodes objects.

matches(other: Node) → bool

Check whether all attributes match between two nodes.

Parameters:

other – The Node to compare to this one.

Returns:

True if all attributes match, False otherwise.

class sleap.skeleton.Skeleton(name: Optional[str] = None)

The main object for representing animal skeletons.

The skeleton represents the constituent parts of the animal whose pose is being estimated.

_skeleton_idx

An index variable used to give skeletons a default name that should be unique across all skeletons.

preview_image

A byte string containing an encoded preview image for the skeleton. Used only for templates.

Type:

Optional[bytes]

description

A text description of the skeleton. Used only for templates.

Type:

Optional[str]

_is_template

Whether this skeleton is a template. Used only for templates.

Type:

bool

add_edge(source: str, destination: str)

Add an edge between two nodes.

Parameters:

• source – The name of the source node.

• destination – The name of the destination node.

Raises:

ValueError – If source or destination nodes cannot be found, or if edge already exists between those nodes.

Returns:

None.

add_node(name: str)

Add a node representing an animal part to the skeleton.

Parameters:

name – The name of the node to add to the skeleton. This name must be unique within the skeleton.

Raises:

ValueError – If name is not unique.

add_nodes(name_list: List[str])

Add a list of nodes representing animal parts to the skeleton.

Parameters:

name_list – List of strings representing the nodes.

add_symmetry(node1: str, node2: str)

Specify that two parts (nodes) in skeleton are symmetrical.

Certain parts of an animal body can be related as symmetrical parts in a pair. For example, left and right hands of a person.

Parameters:

• node1 – The name of the first part in the symmetric pair

• node2 – The name of the second part in the symmetric pair

Raises:

ValueError – If node1 and node2 match, or if there is already a symmetry between them.

Returns:

None

clear_edges()

Delete all edges in skeleton.

delete_edge(source: str, destination: str)

Delete an edge between two nodes.

Parameters:

• source – The name of the source node.

• destination – The name of the destination node.

Raises:

ValueError – If skeleton does not have either source node, destination node, or edge between them.

Returns:

None

delete_node(name: str)

Remove a node from the skeleton.

The method removes a node from the skeleton and any edge that is connected to it.

Parameters:

name – The name of the node to remove

Raises:

ValueError – If node cannot be found.

Returns:

None

delete_symmetry(node1: Union[str, Node], node2: Union[str, Node])

Delete a previously established symmetry between two nodes.

Parameters:

• node1 – One node (by Node object or name) in symmetric pair.

• node2 – Other node in symmetric pair.

Raises:

ValueError – If there’s no symmetry between node1 and node2.

Returns:

None

property edge_inds: List[Tuple[int, int]]

Get a list of edges as node indices.

Returns:

A list of (src_node_ind, dst_node_ind), where indices are subscripts into the Skeleton.nodes list.

property edge_names: List[Tuple[str, str]]

Get a list of edge name tuples.

Returns:

list of (src_node.name, dst_node.name)

edge_to_index(source: Union[str, Node], destination: Union[str, Node]) → int

Return the index of edge from source to destination.

property edges: List[Tuple[Node, Node]]

Get a list of edge tuples.

Returns:

list of (src_node, dst_node)

property edges_full: List[Tuple[Node, Node, Any, Any]]

Get a list of edge tuples with keys and attributes.

Returns:

list of (src_node, dst_node, key, attributes)

find_neighbors(node: Union[str, Node]) → List[Node]

Find nodes that are predecessors or successors from a node.

Parameters:

node – Name or Node instance.

Returns:

A list of Node objects that are neighbors to the node.

find_node(name: Union[str, Node]) → Node

Find node in skeleton by name of node.

Parameters:

name – The name of the Node (or a Node)

Returns:

Node, or None if no match found

static find_unique_nodes(skeletons: List[Skeleton]) → List[Node]

Find all unique nodes from a list of skeletons.

Parameters:

skeletons – The list of skeletons.

Returns:

A list of unique Node objects.

classmethod from_dict(d: Dict, node_to_idx: Optional[Dict[Node, int]] = None) → Skeleton

Create skeleton from dict; used for loading from JSON.

Parameters:

• d – the dict from which to deserialize

• node_to_idx – optional dict which maps Node`sto index in some list. This is used when saving :class:`Labels`where we want to serialize the :class:`Nodes outside the Skeleton object. If given, then we replace each Node with specified index before converting Skeleton. Otherwise, we convert Node objects with the rest of the Skeleton.

Returns:

Skeleton.

classmethod from_json(json_str: str, idx_to_node: Optional[Dict[int, Node]] = None) → Skeleton

Instantiate Skeleton from JSON string.

Parameters:

• json_str – The JSON encoded Skeleton.

• idx_to_node – optional dict which maps an int (indexing a list of Node objects) to the already deserialized Node. This should invert node_to_idx we used when saving. If not given, then we’ll assume each Node was left in the Skeleton when it was saved.

Returns:

An instance of the Skeleton object decoded from the JSON.

classmethod from_names_and_edge_inds(node_names: List[str], edge_inds: Optional[List[Tuple[int, int]]] = None) → Skeleton

Create skeleton from a list of node names and edge indices.

Parameters:

• node_names – List of strings defining the nodes.

• edge_inds – List of tuples in the form (src_node_ind, dst_node_ind). If not specified, the resulting skeleton will have no edges.

Returns:

The instantiated skeleton.

get_symmetry(node: Union[str, Node]) → Optional[Node]

Return the node symmetric with the specified node.

Parameters:

node – Node (by Node object or name) to query.

Raises:

ValueError – If node has more than one symmetry.

Returns:

The symmetric Node, None if no symmetry.

get_symmetry_name(node: Union[str, Node]) → Optional[str]

Return the name of the node symmetric with the specified node.

Parameters:

node – Node (by Node object or name) to query.

Returns:

Name of symmetric node, None if no symmetry.

property graph

Return a view on the subgraph of body nodes and edges for skeleton.

property graph_symmetry

Return subgraph of symmetric edges for skeleton.

has_edge(source_name: str, dest_name: str) → bool

Check whether the skeleton has an edge.

Parameters:

• source_name – The name of the source node for the edge.

• dest_name – The name of the destination node for the edge.

Returns:

True is yes, False if no.

has_node(name: str) → bool

Check whether the skeleton has a node.

Parameters:

name – The name of the node to check for.

Returns:

True for yes, False for no.

has_nodes(names: Iterable[str]) → bool

Check whether the skeleton has a list of nodes.

Parameters:

names – The list names of the nodes to check for.

Returns:

True for yes, False for no.

property is_arborescence: bool

Return whether this skeleton graph forms an arborescence.

property is_template: bool

Return whether this skeleton is a template.

If is_template is True, then the preview image and description are saved. If is_template is False, then the preview image and description are not saved.

Only provided template skeletons are considered templates. To save a new template skeleton, change this to True before saving.

classmethod load_all_hdf5(file: Union[str, File], return_dict: bool = False) → Union[List[Skeleton], Dict[str, Skeleton]]

Load all skeletons found in the HDF5 file.

Parameters:

• file – The file name or open h5py.File

• return_dict – Whether the the return value should be a dict where the keys are skeleton names and values the corresponding skeleton. If False, then method will return just a list of the skeletons.

Returns:

The skeleton instances stored in the HDF5 file. Either in List or Dict form.

classmethod load_hdf5(file: Union[str, File], name: str) → List[Skeleton]

Load a specific skeleton (by name) from the HDF5 file.

Parameters:

• file – The file name or open h5py.File

• name – The name of the skeleton.

Returns:

The specified Skeleton instance stored in the HDF5 file.

classmethod load_json(filename: str, idx_to_node: Optional[Dict[int, Node]] = None) → Skeleton

Load a skeleton from a JSON file.

This method will load the Skeleton from JSON file saved with; save_json()

Parameters:

• filename – The file that contains the JSON.

• idx_to_node – optional dict which maps an int (indexing a list of Node objects) to the already deserialized Node. This should invert node_to_idx we used when saving. If not given, then we’ll assume each Node was left in the Skeleton when it was saved.

Returns:

The Skeleton object stored in the JSON filename.

classmethod load_mat(filename: str) → Skeleton

Load the skeleton from a Matlab MAT file.

This is to support backwards compatibility with old LEAP MATLAB code and datasets.

Parameters:

filename – The name of the skeleton file

Returns:

An instance of the skeleton.

static make_cattr(idx_to_node: Optional[Dict[int, Node]] = None) → Converter

Make cattr.Convert() for Skeleton.

Make a cattr.Converter() that registers structure/unstructure hooks for Skeleton objects to handle serialization of skeletons.

Parameters:

idx_to_node – A dict that maps node index to Node objects.

Returns:

A cattr.Converter() instance for skeleton serialization and deserialization.

matches(other: Skeleton) → bool

Compare this Skeleton to another, ignoring name and node identities.

Parameters:

other – The other skeleton.

Returns:

True if the skeleton graphs are isomorphic and node names.

property name: str

Get the name of the skeleton.

Returns:

A string representing the name of the skeleton.

property node_names: List[str]

Get a list of node names.

Returns:

A list of node names.

node_to_index(node: Union[str, Node]) → int

Return the index of the node, accepts either Node or name.

Parameters:

node – The name of the node or the Node object.

Raises:

ValueError if node cannot be found in skeleton. –

Returns:

The index of the node in the graph.

property nodes: List[Node]

Get a list of :class:`Node`s.

Returns:

A list of :class:`Node`s

relabel_node(old_name: str, new_name: str)

Relabel a single node to a new name.

Parameters:

• old_name – The old name of the node.

• new_name – The new name of the node.

Returns:

None

relabel_nodes(mapping: Dict[str, str])

Relabel the nodes of the skeleton.

Parameters:

mapping – A dictionary with the old labels as keys and new labels as values. A partial mapping is allowed.

Raises:

ValueError – If node already present with one of the new names.

Returns:

None

classmethod rename_skeleton(skeleton: Skeleton, name: str) → Skeleton

Make copy of skeleton with new name.

This property is immutable because it is used to hash skeletons. If you want to rename a Skeleton you must use this class method.

>>> new_skeleton = Skeleton.rename_skeleton(
>>>     skeleton=old_skeleton, name="New Name")

Parameters:

• skeleton – The skeleton to copy.

• name – The name of the new skeleton.

Returns:

A new deep copied skeleton with the changed name.

classmethod save_all_hdf5(file: Union[str, File], skeletons: List[Skeleton])

Convenience method to save a list of skeletons to HDF5 file.

Skeletons are saved as attributes of a /skeleton group in the file.

Parameters:

• file – The filename or the open h5py.File object.

• skeletons – The list of skeletons to save.

Raises:

ValueError – If multiple skeletons have the same name.

Returns:

None

save_hdf5(file: Union[str, File])

Wrapper for HDF5 saving which takes either filename or h5py.File.

Parameters:

file – can be filename (string) or h5py.File object

Returns:

None

save_json(filename: str, node_to_idx: Optional[Dict[Node, int]] = None)

Save the Skeleton as JSON file.

Output the complete skeleton to a file in JSON format.

Parameters:

• filename – The filename to save the JSON to.

• node_to_idx – optional dict which maps Node`sto index in some list. This is used when saving :class:`Labels`where we want to serialize the :class:`Nodes outside the Skeleton object. If given, then we replace each Node with specified index before converting Skeleton. Otherwise, we convert Node objects with the rest of the Skeleton.

Returns:

None

property symmetric_inds: ndarray

Return the symmetric nodes as an array of indices.

property symmetries: List[Tuple[Node, Node]]

Get a list of all symmetries without duplicates.

Returns:

list of (node1, node2)

property symmetries_full: List[Tuple[Node, Node, Any, Any]]

Get a list of all symmetries with keys and attributes.

Note: The returned list will contain duplicates (node1, node2) and (node2, node1).

Returns:

list of (node1, node2, key, attr)

property symmetry_names: List[Tuple[str, str]]

List of symmetry edges as tuples of node names.

static to_dict(obj: Skeleton, node_to_idx: Optional[Dict[Node, int]] = None) → Dict

Convert skeleton to dict; used for saving as JSON.

Parameters:

• obj – the :object:`Skeleton` to convert

• node_to_idx – optional dict which maps Node`sto index in some list. This is used when saving :class:`Labels`where we want to serialize the :class:`Nodes outside the Skeleton object. If given, then we replace each Node with specified index before converting Skeleton. Otherwise, we convert Node objects with the rest of the Skeleton.

Returns:

dict with data from skeleton

to_json(node_to_idx: Optional[Dict[Node, int]] = None) → str

Convert the Skeleton to a JSON representation.

Parameters:

node_to_idx – optional dict which maps Node`sto index in some list. This is used when saving :class:`Labels`where we want to serialize the :class:`Nodes outside the Skeleton object. If given, then we replace each Node with specified index before converting Skeleton. Otherwise, we convert Node objects with the rest of the Skeleton.

Returns:

A string containing the JSON representation of the skeleton.

class sleap.skeleton.SkeletonDecoder

Replace jsonpickle.decode with our own decoder.

This function will decode the following from jsonpickle’s encoded format:

Node objects from

{

“py/object”: “sleap.skeleton.Node”, “py/state”: { “py/tuple”: [“thorax1”, 1.0] }

}

to Node(name="thorax1", weight=1.0)

EdgeType objects from

{

“py/reduce”: [

{ “py/type”: “sleap.skeleton.EdgeType” }, { “py/tuple”: [1] }

]

}

to EdgeType(1)

bytes from

{

“py/b64”: “aVZC…”

}

to b"iVBO..."

and any repeated objects from

{

“py/id”: 1

}

to the object with the same reconstruction id (from top to bottom).

classmethod decode(json_str: str) → Dict

Decode the given json string into a dictionary.

Returns:

A dict with Node`s, `EdgeType`s, and `bytes decoded/reconstructed.

static decode_preview_image(img_b64: bytes, return_bytes: bool = False) → Union[Image, bytes]

Decode a skeleton preview image byte string representation to a PIL.Image

Parameters:

• img_b64 – a byte string representation of a skeleton preview image

• return_bytes – whether to return the decoded image as bytes

Returns:

Either a PIL.Image of the skeleton preview image or the decoded image as bytes (if return_bytes is True).

class sleap.skeleton.SkeletonEncoder

Replace jsonpickle.encode with our own encoder.

The input is a dictionary containing python objects that need to be encoded as JSON strings. The output is a JSON string that represents the input dictionary.

Node(name='neck', weight=1.0) =>

{

“py/object”: “sleap.Skeleton.Node”, “py/state”: {“py/tuple” [“neck”, 1.0]}

}

<EdgeType.BODY: 1> =>

{“py/reduce”: [

{“py/type”: “sleap.Skeleton.EdgeType”}, {“py/tuple”: [1] } ]

}`

Where name and weight are the attributes of the Node class; weight is always 1.0. EdgeType is an enum with values BODY = 1 and SYMMETRY = 2.

See sleap.skeleton.Node and sleap.skeleton.EdgeType.

If the object has been “seen” before, it will not be encoded as the full JSON string but referenced by its py/id, which starts at 1 and indexes the objects in the order they are seen so that the second time the first object is used, it will be referenced as {"py/id": 1}.

classmethod encode(data: Dict[str, Any]) → str

Encodes the input dictionary as a JSON string.

Parameters:

data – The data to encode.

Returns:

The JSON string representation of the data.

Return type:

json_str