Open In Colab

Post-inference tracking#

In tracking, we associate the poses (that were detected within individual frames) across time such that they belong to the same individual.

Since we typically do this separately, it is sometimes desirable to tweak the tracking parameters to optimize accuracy without having to re-run inference (e.g., sleap-track).

In this notebook, we will explore how to re-run the tracking given an existing predictions SLP file.

Note: Tracking does not run on the GPU, so this notebook can be run locally on your computer without the hassle of uploading your data if desired.

1. Setup SLEAP#

Run this cell first to install SLEAP. If you get a dependency error in subsequent cells, just click Runtime → Restart runtime to reload the packages.

Install#

# This should take care of all the dependencies on colab:
!pip uninstall -y opencv-python opencv-contrib-python && pip install sleap

# But to do it locally, we'd recommend the conda package (available on Windows + Linux):
# conda create -n sleap -c sleap -c conda-forge -c nvidia sleap

Found existing installation: opencv-python 4.1.2.30
Uninstalling opencv-python-4.1.2.30:
  Successfully uninstalled opencv-python-4.1.2.30
Found existing installation: opencv-contrib-python 4.1.2.30
Uninstalling opencv-contrib-python-4.1.2.30:
  Successfully uninstalled opencv-contrib-python-4.1.2.30
Collecting sleap
  Downloading sleap-1.2.2-py3-none-any.whl (62.0 MB)
     |████████████████████████████████| 62.0 MB 19 kB/s 
?25hCollecting pykalman==0.9.5
  Downloading pykalman-0.9.5.tar.gz (228 kB)
     |████████████████████████████████| 228 kB 21.7 MB/s 
?25hRequirement already satisfied: certifi<=2021.10.8,>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from sleap) (2021.10.8)
Requirement already satisfied: h5py<=3.6.0,>=3.1.0 in /usr/local/lib/python3.7/dist-packages (from sleap) (3.1.0)
Collecting opencv-python-headless<=4.5.5.62,>=4.2.0.34
  Downloading opencv_python_headless-4.5.5.62-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (47.7 MB)
     |████████████████████████████████| 47.7 MB 1.4 MB/s 
?25hCollecting jsonpickle==1.2
  Downloading jsonpickle-1.2-py2.py3-none-any.whl (32 kB)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.7/dist-packages (from sleap) (3.13)
Requirement already satisfied: scikit-learn==1.0.* in /usr/local/lib/python3.7/dist-packages (from sleap) (1.0.2)
Collecting imgstore==0.2.9
  Downloading imgstore-0.2.9-py2.py3-none-any.whl (904 kB)
     |████████████████████████████████| 904 kB 44.2 MB/s 
?25hRequirement already satisfied: networkx in /usr/local/lib/python3.7/dist-packages (from sleap) (2.6.3)
Requirement already satisfied: pyzmq in /usr/local/lib/python3.7/dist-packages (from sleap) (22.3.0)
Requirement already satisfied: scipy<=1.7.3,>=1.4.1 in /usr/local/lib/python3.7/dist-packages (from sleap) (1.4.1)
Requirement already satisfied: psutil in /usr/local/lib/python3.7/dist-packages (from sleap) (5.4.8)
Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from sleap) (1.3.5)
Collecting segmentation-models==1.0.1
  Downloading segmentation_models-1.0.1-py3-none-any.whl (33 kB)
Collecting rich==10.16.1
  Downloading rich-10.16.1-py3-none-any.whl (214 kB)
     |████████████████████████████████| 214 kB 53.6 MB/s 
?25hRequirement already satisfied: numpy<=1.21.5,>=1.19.5 in /usr/local/lib/python3.7/dist-packages (from sleap) (1.21.5)
Collecting qimage2ndarray<=1.8.3,>=1.8.2
  Downloading qimage2ndarray-1.8.3-py3-none-any.whl (11 kB)
Collecting python-rapidjson
  Downloading python_rapidjson-1.6-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB)
     |████████████████████████████████| 1.6 MB 21.9 MB/s 
?25hCollecting attrs==21.2.0
  Downloading attrs-21.2.0-py2.py3-none-any.whl (53 kB)
     |████████████████████████████████| 53 kB 1.6 MB/s 
?25hRequirement already satisfied: tensorflow<2.9.0,>=2.6.3 in /usr/local/lib/python3.7/dist-packages (from sleap) (2.8.0)
Requirement already satisfied: scikit-image in /usr/local/lib/python3.7/dist-packages (from sleap) (0.18.3)
Collecting cattrs==1.1.1
  Downloading cattrs-1.1.1-py3-none-any.whl (16 kB)
Collecting jsmin
  Downloading jsmin-3.0.1.tar.gz (13 kB)
Collecting scikit-video
  Downloading scikit_video-1.1.11-py2.py3-none-any.whl (2.3 MB)
     |████████████████████████████████| 2.3 MB 61.6 MB/s 
?25hRequirement already satisfied: imageio<=2.15.0 in /usr/local/lib/python3.7/dist-packages (from sleap) (2.4.1)
Requirement already satisfied: seaborn in /usr/local/lib/python3.7/dist-packages (from sleap) (0.11.2)
Collecting PySide2<=5.14.1,>=5.13.2
  Downloading PySide2-5.14.1-5.14.1-cp35.cp36.cp37.cp38-abi3-manylinux1_x86_64.whl (165.5 MB)
     |████████████████████████████████| 165.5 MB 69 kB/s 
?25hCollecting imgaug==0.4.0
  Downloading imgaug-0.4.0-py2.py3-none-any.whl (948 kB)
     |████████████████████████████████| 948 kB 27.9 MB/s 
?25hRequirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from imgaug==0.4.0->sleap) (1.15.0)
Collecting opencv-python
  Downloading opencv_python-4.5.5.64-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (60.5 MB)
     |████████████████████████████████| 60.5 MB 1.1 MB/s 
?25hRequirement already satisfied: Shapely in /usr/local/lib/python3.7/dist-packages (from imgaug==0.4.0->sleap) (1.8.1.post1)
Requirement already satisfied: Pillow in /usr/local/lib/python3.7/dist-packages (from imgaug==0.4.0->sleap) (7.1.2)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from imgaug==0.4.0->sleap) (3.2.2)
Requirement already satisfied: tzlocal in /usr/local/lib/python3.7/dist-packages (from imgstore==0.2.9->sleap) (1.5.1)
Requirement already satisfied: python-dateutil in /usr/local/lib/python3.7/dist-packages (from imgstore==0.2.9->sleap) (2.8.2)
Requirement already satisfied: pytz in /usr/local/lib/python3.7/dist-packages (from imgstore==0.2.9->sleap) (2018.9)
Collecting colorama<0.5.0,>=0.4.0
  Downloading colorama-0.4.4-py2.py3-none-any.whl (16 kB)
Requirement already satisfied: pygments<3.0.0,>=2.6.0 in /usr/local/lib/python3.7/dist-packages (from rich==10.16.1->sleap) (2.6.1)
Collecting commonmark<0.10.0,>=0.9.0
  Downloading commonmark-0.9.1-py2.py3-none-any.whl (51 kB)
     |████████████████████████████████| 51 kB 5.9 MB/s 
?25hRequirement already satisfied: typing-extensions<5.0,>=3.7.4 in /usr/local/lib/python3.7/dist-packages (from rich==10.16.1->sleap) (3.10.0.2)
Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn==1.0.*->sleap) (3.1.0)
Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn==1.0.*->sleap) (1.1.0)
Collecting efficientnet==1.0.0
  Downloading efficientnet-1.0.0-py3-none-any.whl (17 kB)
Collecting image-classifiers==1.0.0
  Downloading image_classifiers-1.0.0-py3-none-any.whl (19 kB)
Collecting keras-applications<=1.0.8,>=1.0.7
  Downloading Keras_Applications-1.0.8-py3-none-any.whl (50 kB)
     |████████████████████████████████| 50 kB 6.3 MB/s 
?25hRequirement already satisfied: cached-property in /usr/local/lib/python3.7/dist-packages (from h5py<=3.6.0,>=3.1.0->sleap) (1.5.2)
Collecting shiboken2==5.14.1
  Downloading shiboken2-5.14.1-5.14.1-cp35.cp36.cp37.cp38-abi3-manylinux1_x86_64.whl (847 kB)
     |████████████████████████████████| 847 kB 43.5 MB/s 
?25hRequirement already satisfied: PyWavelets>=1.1.1 in /usr/local/lib/python3.7/dist-packages (from scikit-image->sleap) (1.3.0)
Requirement already satisfied: tifffile>=2019.7.26 in /usr/local/lib/python3.7/dist-packages (from scikit-image->sleap) (2021.11.2)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->imgaug==0.4.0->sleap) (3.0.7)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->imgaug==0.4.0->sleap) (0.11.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->imgaug==0.4.0->sleap) (1.4.0)
Requirement already satisfied: gast>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.9.0,>=2.6.3->sleap) (0.5.3)
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.9.0,>=2.6.3->sleap) (57.4.0)
Requirement already satisfied: libclang>=9.0.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.9.0,>=2.6.3->sleap) (13.0.0)
Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.9.0,>=2.6.3->sleap) (0.24.0)
Requirement already satisfied: astunparse>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.9.0,>=2.6.3->sleap) (1.6.3)
Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.9.0,>=2.6.3->sleap) (3.3.0)
Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.9.0,>=2.6.3->sleap) (1.1.0)
Requirement already satisfied: keras<2.9,>=2.8.0rc0 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.9.0,>=2.6.3->sleap) (2.8.0)
Requirement already satisfied: wrapt>=1.11.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.9.0,>=2.6.3->sleap) (1.14.0)
Requirement already satisfied: keras-preprocessing>=1.1.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.9.0,>=2.6.3->sleap) (1.1.2)
Requirement already satisfied: google-pasta>=0.1.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.9.0,>=2.6.3->sleap) (0.2.0)
Collecting tf-estimator-nightly==2.8.0.dev2021122109
  Downloading tf_estimator_nightly-2.8.0.dev2021122109-py2.py3-none-any.whl (462 kB)
     |████████████████████████████████| 462 kB 49.8 MB/s 
?25hRequirement already satisfied: protobuf>=3.9.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.9.0,>=2.6.3->sleap) (3.17.3)
Requirement already satisfied: tensorboard<2.9,>=2.8 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.9.0,>=2.6.3->sleap) (2.8.0)
Requirement already satisfied: flatbuffers>=1.12 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.9.0,>=2.6.3->sleap) (2.0)
Requirement already satisfied: absl-py>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.9.0,>=2.6.3->sleap) (1.0.0)
Requirement already satisfied: grpcio<2.0,>=1.24.3 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.9.0,>=2.6.3->sleap) (1.44.0)
Requirement already satisfied: wheel<1.0,>=0.23.0 in /usr/local/lib/python3.7/dist-packages (from astunparse>=1.6.0->tensorflow<2.9.0,>=2.6.3->sleap) (0.37.1)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.7/dist-packages (from tensorboard<2.9,>=2.8->tensorflow<2.9.0,>=2.6.3->sleap) (0.4.6)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard<2.9,>=2.8->tensorflow<2.9.0,>=2.6.3->sleap) (1.8.1)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.7/dist-packages (from tensorboard<2.9,>=2.8->tensorflow<2.9.0,>=2.6.3->sleap) (3.3.6)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard<2.9,>=2.8->tensorflow<2.9.0,>=2.6.3->sleap) (0.6.1)
Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.7/dist-packages (from tensorboard<2.9,>=2.8->tensorflow<2.9.0,>=2.6.3->sleap) (1.0.1)
Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard<2.9,>=2.8->tensorflow<2.9.0,>=2.6.3->sleap) (2.23.0)
Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.7/dist-packages (from tensorboard<2.9,>=2.8->tensorflow<2.9.0,>=2.6.3->sleap) (1.35.0)
Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow<2.9.0,>=2.6.3->sleap) (4.8)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow<2.9.0,>=2.6.3->sleap) (0.2.8)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow<2.9.0,>=2.6.3->sleap) (4.2.4)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.7/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.9,>=2.8->tensorflow<2.9.0,>=2.6.3->sleap) (1.3.1)
Requirement already satisfied: importlib-metadata>=4.4 in /usr/local/lib/python3.7/dist-packages (from markdown>=2.6.8->tensorboard<2.9,>=2.8->tensorflow<2.9.0,>=2.6.3->sleap) (4.11.3)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard<2.9,>=2.8->tensorflow<2.9.0,>=2.6.3->sleap) (3.7.0)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow<2.9.0,>=2.6.3->sleap) (0.4.8)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard<2.9,>=2.8->tensorflow<2.9.0,>=2.6.3->sleap) (2.10)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard<2.9,>=2.8->tensorflow<2.9.0,>=2.6.3->sleap) (3.0.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard<2.9,>=2.8->tensorflow<2.9.0,>=2.6.3->sleap) (1.24.3)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.9,>=2.8->tensorflow<2.9.0,>=2.6.3->sleap) (3.2.0)
Building wheels for collected packages: pykalman, jsmin
  Building wheel for pykalman (setup.py) ... ?25l?25hdone
  Created wheel for pykalman: filename=pykalman-0.9.5-py3-none-any.whl size=48462 sha256=5de7d8c6487261ac5359426edf6b9d6ff977786a758424aaa6462a743fae77e4
  Stored in directory: /root/.cache/pip/wheels/6a/04/02/2dda6ea59c66d9e685affc8af3a31ad3a5d87b7311689efce6
  Building wheel for jsmin (setup.py) ... ?25l?25hdone
  Created wheel for jsmin: filename=jsmin-3.0.1-py3-none-any.whl size=13782 sha256=353b91b543700f74d4c7801c636ff32de6e99c9578162db575ea8d5e0b29d64e
  Stored in directory: /root/.cache/pip/wheels/a4/0b/64/fb4f87526ecbdf7921769a39d91dcfe4860e621cf15b8250d6
Successfully built pykalman jsmin
Installing collected packages: keras-applications, tf-estimator-nightly, shiboken2, opencv-python, image-classifiers, efficientnet, commonmark, colorama, attrs, segmentation-models, scikit-video, rich, qimage2ndarray, python-rapidjson, PySide2, pykalman, opencv-python-headless, jsonpickle, jsmin, imgstore, imgaug, cattrs, sleap
  Attempting uninstall: attrs
    Found existing installation: attrs 21.4.0
    Uninstalling attrs-21.4.0:
      Successfully uninstalled attrs-21.4.0
  Attempting uninstall: imgaug
    Found existing installation: imgaug 0.2.9
    Uninstalling imgaug-0.2.9:
      Successfully uninstalled imgaug-0.2.9
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible.
albumentations 0.1.12 requires imgaug<0.2.7,>=0.2.5, but you have imgaug 0.4.0 which is incompatible.
Successfully installed PySide2-5.14.1 attrs-21.2.0 cattrs-1.1.1 colorama-0.4.4 commonmark-0.9.1 efficientnet-1.0.0 image-classifiers-1.0.0 imgaug-0.4.0 imgstore-0.2.9 jsmin-3.0.1 jsonpickle-1.2 keras-applications-1.0.8 opencv-python-4.5.5.64 opencv-python-headless-4.5.5.62 pykalman-0.9.5 python-rapidjson-1.6 qimage2ndarray-1.8.3 rich-10.16.1 scikit-video-1.1.11 segmentation-models-1.0.1 shiboken2-5.14.1 sleap-1.2.2 tf-estimator-nightly-2.8.0.dev2021122109

Test#

#@title SLEAP and system versions: { display-mode: "form" }
import sleap
sleap.versions()
sleap.system_summary()

INFO:numexpr.utils:NumExpr defaulting to 2 threads.
SLEAP: 1.2.2
TensorFlow: 2.8.0
Numpy: 1.21.5
Python: 3.7.13
OS: Linux-5.4.144+-x86_64-with-Ubuntu-18.04-bionic
GPUs: None detected.

2. Setup data#

Here we’re downloading an existing .slp file with predictions and the corresponding .mp4 video.

You should replace this with Google Drive mounting if running this on Google Colab, or simply skip it altogether and just set the paths below if running locally.

!wget -O fly_clip.mp4 https://github.com/talmolab/sleap-tutorial-uo/blob/main/data/fly_clip.mp4?raw=true
!wget -O predictions.slp https://github.com/talmolab/sleap-tutorial-uo/blob/main/data/predictions.slp?raw=true

--2022-04-04 00:10:34--  https://github.com/talmolab/sleap-tutorial-uo/blob/main/data/fly_clip.mp4?raw=true
Resolving github.com (github.com)... 13.114.40.48
Connecting to github.com (github.com)|13.114.40.48|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github.com/talmolab/sleap-tutorial-uo/raw/main/data/fly_clip.mp4 [following]
--2022-04-04 00:10:34--  https://github.com/talmolab/sleap-tutorial-uo/raw/main/data/fly_clip.mp4
Reusing existing connection to github.com:443.
HTTP request sent, awaiting response... 302 Found
Location: https://raw.githubusercontent.com/talmolab/sleap-tutorial-uo/main/data/fly_clip.mp4 [following]
--2022-04-04 00:10:34--  https://raw.githubusercontent.com/talmolab/sleap-tutorial-uo/main/data/fly_clip.mp4
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.108.133, 185.199.109.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 676194 (660K) [application/octet-stream]
Saving to: ‘fly_clip.mp4’

fly_clip.mp4        100%[===================>] 660.35K  --.-KB/s    in 0.05s   

2022-04-04 00:10:36 (12.1 MB/s) - ‘fly_clip.mp4’ saved [676194/676194]

--2022-04-04 00:10:36--  https://github.com/talmolab/sleap-tutorial-uo/blob/main/data/predictions.slp?raw=true
Resolving github.com (github.com)... 52.69.186.44
Connecting to github.com (github.com)|52.69.186.44|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github.com/talmolab/sleap-tutorial-uo/raw/main/data/predictions.slp [following]
--2022-04-04 00:10:37--  https://github.com/talmolab/sleap-tutorial-uo/raw/main/data/predictions.slp
Reusing existing connection to github.com:443.
HTTP request sent, awaiting response... 302 Found
Location: https://raw.githubusercontent.com/talmolab/sleap-tutorial-uo/main/data/predictions.slp [following]
--2022-04-04 00:10:37--  https://raw.githubusercontent.com/talmolab/sleap-tutorial-uo/main/data/predictions.slp
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.108.133, 185.199.109.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 420976 (411K) [application/octet-stream]
Saving to: ‘predictions.slp’

predictions.slp     100%[===================>] 411.11K  --.-KB/s    in 0.04s   

2022-04-04 00:10:38 (9.66 MB/s) - ‘predictions.slp’ saved [420976/420976]

PREDICTIONS_FILE = "predictions.slp"

3. Track#

# Load predictions
labels = sleap.load_file(PREDICTIONS_FILE)

# Here I'm removing the tracks so we just have instances without any tracking applied.
for instance in labels.instances():
    instance.track = None
labels.tracks = []
labels

Labels(labeled_frames=1350, videos=1, skeletons=1, tracks=0)

Here we create a tracker with the options we want to experiment with. You can read more about tracking in the documentation or the parameters in the sleap-track CLI help.

# Create tracker
tracker = sleap.nn.tracking.Tracker.make_tracker_by_name(
    # General tracking options
    tracker="flow",
    track_window=3,

    # Matching options
    similarity="instance",
    match="greedy",
    min_new_track_points=1,
    min_match_points=1,

    # Optical flow options (only applies to "flow" tracker)
    img_scale=0.5,
    of_window_size=21,
    of_max_levels=3,

    # Pre-tracking filtering options
    target_instance_count=2,
    pre_cull_to_target=True,
    pre_cull_iou_threshold=0.8,

    # Post-tracking filtering options
    post_connect_single_breaks=True,
    clean_instance_count=0,
    clean_iou_threshold=None,
)

Next we’ll actually run the tracking on each frame. This might take a bit longer when using the "flow" method.

tracked_lfs = []
for lf in labels:
    lf.instances = tracker.track(lf.instances, img=lf.image)
    tracked_lfs.append(lf)
tracked_labels = sleap.Labels(tracked_lfs)
tracked_labels

Labels(labeled_frames=1350, videos=1, skeletons=1, tracks=2)

4. Inspect and save#

Let’s see the results and save out the tracked predictions.

tracked_labels[0].plot(scale=0.25)
../_images/Post_inference_tracking_17_0.png 
tracked_labels[100].plot(scale=0.25)
../_images/Post_inference_tracking_18_0.png 
tracked_labels.save("retracked.slp")