{ "nbformat": 4, "nbformat_minor": 0, "metadata": { "colab": { "name": "Training with custom hyperparameters", "provenance": [], "collapsed_sections": [], "toc_visible": true, "include_colab_link": true }, "kernelspec": { "name": "python3", "display_name": "Python 3" }, "accelerator": "GPU" }, "cells": [ { "cell_type": "markdown", "metadata": { "id": "view-in-github", "colab_type": "text" }, "source": [ "\"Open" ] }, { "cell_type": "markdown", "metadata": { "id": "K5xp-A8Oc80Q", "colab_type": "text" }, "source": [ "This notebook shows you how to use SLEAP on Colab to train a model with **custom hyperpameters**. Hyperparameters define things like the model architecture, the learning rate, and data augmentation methods. See our page about [choosing a set of models](https://sleap.ai/guides/choosing-models.html) for more information.\n", "\n", "This notebook assumes that you're already confortable using Google Drive to move your data to and from Colab. If you haven't done this already, take a look at our [other notebook](https://colab.research.google.com/github/murthylab/sleap-notebooks/blob/master/Training_and_inference_using_Google_Drive.ipynb) which walks you through that process in more detail." ] }, { "cell_type": "markdown", "metadata": { "id": "yX9noEb8m8re", "colab_type": "text" }, "source": [ "# Installation\n", "\n", "Before we install SLEAP we need to set Colab to use TensorFlow 2." ] }, { "cell_type": "code", "metadata": { "id": "1hJvyI24KXlu", "colab_type": "code", "colab": {} }, "source": [ "%tensorflow_version 2.x" ], "execution_count": 1, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "C5WL10cao-sc", "colab_type": "text" }, "source": [ "Let's confirm that we have a GPU available. This next line should return something like \"/device:GPU:0\". If you instead see an empty string as the result, go to \"Notebook settings\" in the \"Edit\" menu and select \"GPU\" as the hardware accelerator." ] }, { "cell_type": "code", "metadata": { "id": "9I12Lk6TKZnn", "colab": { "base_uri": "https://localhost:8080/", "height": 35 }, "outputId": "e3623a95-0b38-4df5-8f58-1df61cfc88ed" }, "source": [ "import tensorflow as tf\n", "tf.test.gpu_device_name()" ], "execution_count": 2, "outputs": [ { "output_type": "execute_result", "data": { "application/vnd.google.colaboratory.intrinsic+json": { "type": "string" }, "text/plain": [ "'/device:GPU:0'" ] }, "metadata": { "tags": [] }, "execution_count": 2 } ] }, { "cell_type": "markdown", "metadata": { "id": "pMjYpcVFqWgV" }, "source": [ "Now let's use `pip` to install SLEAP from PyPI." ] }, { "cell_type": "code", "metadata": { "id": "DUfnkxMtLcK3", "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, "outputId": "fce82767-991a-468e-a749-aa51ee0ecb9f" }, "source": [ "!pip install sleap==1.1.1" ], "execution_count": 3, "outputs": [ { "output_type": "stream", "text": [ "Collecting sleap==1.1.0\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/b9/52/e80875c50a649da56377c7884d5b937260b2310f8c2543dca9bf6c3c06d4/sleap-1.1.0a10-py3-none-any.whl (39.5MB)\n", "\u001b[K |████████████████████████████████| 39.5MB 84kB/s \n", "Collecting imgaug==0.3.0\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/11/df/5a3bba95b4600d5ca7aff072082ef0d9837056dd28cc4e738e7ce88dd8f8/imgaug-0.3.0-py2.py3-none-any.whl (819kB)\n", "\u001b[K |████████████████████████████████| 829kB 46.9MB/s \n", "\u001b[?25hCollecting rich==9.10.0\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/41/60/765dc36f3623c22fd08f152bfae65cc9f0d815c05503ea50f7c189486d10/rich-9.10.0-py3-none-any.whl (188kB)\n", "\u001b[K |████████████████████████████████| 194kB 53.0MB/s \n", "\u001b[?25hRequirement already satisfied: scipy<=1.4.1 in /usr/local/lib/python3.6/dist-packages (from sleap==1.1.0a10) (1.4.1)\n", "Collecting attrs==19.3\n", " Downloading https://files.pythonhosted.org/packages/a2/db/4313ab3be961f7a763066401fb77f7748373b6094076ae2bda2806988af6/attrs-19.3.0-py2.py3-none-any.whl\n", "Collecting segmentation-models==1.0.1\n", " Downloading https://files.pythonhosted.org/packages/da/b9/4a183518c21689a56b834eaaa45cad242d9ec09a4360b5b10139f23c63f4/segmentation_models-1.0.1-py3-none-any.whl\n", "Collecting qimage2ndarray==1.8\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/6c/16/fdd8930ef4f5795afc3cf89f056645ff012da828a4ddcc6a3d6d41edfb53/qimage2ndarray-1.8.tar.gz (183kB)\n", "\u001b[K |████████████████████████████████| 184kB 59.4MB/s \n", "\u001b[?25hRequirement already satisfied: pyzmq in /usr/local/lib/python3.6/dist-packages (from sleap==1.1.0a10) (22.0.2)\n", "Collecting opencv-python-headless==4.2.0.34\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/43/2c/909a04b07360516953beaf6f66480bb6b84b817c6b300c1235bfb2901ad8/opencv_python_headless-4.2.0.34-cp36-cp36m-manylinux1_x86_64.whl (21.6MB)\n", "\u001b[K |████████████████████████████████| 21.6MB 103kB/s \n", "\u001b[?25hCollecting python-rapidjson\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/a0/8f/50488049033aba6943659d7906a45947ea5b510b74e7d44ab9d7457e1710/python_rapidjson-1.0-cp36-cp36m-manylinux2010_x86_64.whl (1.4MB)\n", "\u001b[K |████████████████████████████████| 1.4MB 51.5MB/s \n", "\u001b[?25hRequirement already satisfied: seaborn in /usr/local/lib/python3.6/dist-packages (from sleap==1.1.0a10) (0.11.1)\n", "Collecting numpy<1.19.0,>=1.18.1\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/b3/a9/b1bc4c935ed063766bce7d3e8c7b20bd52e515ff1c732b02caacf7918e5a/numpy-1.18.5-cp36-cp36m-manylinux1_x86_64.whl (20.1MB)\n", "\u001b[K |████████████████████████████████| 20.1MB 1.4MB/s \n", "\u001b[?25hCollecting jsonpickle==1.2\n", " Downloading https://files.pythonhosted.org/packages/07/07/c157520a3ebd166c8c24c6ae0ecae7c3968eb4653ff0e5af369bb82f004d/jsonpickle-1.2-py2.py3-none-any.whl\n", "Collecting cattrs==1.0.0rc\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/01/f6/30e2b78920dae271ba94cb74a808021d88840b156c424c029f6255b72d3e/cattrs-1.0.0rc0-py2.py3-none-any.whl (59kB)\n", "\u001b[K |████████████████████████████████| 61kB 8.8MB/s \n", "\u001b[?25hRequirement already satisfied: pandas in /usr/local/lib/python3.6/dist-packages (from sleap==1.1.0a10) (1.1.5)\n", "Requirement already satisfied: pyyaml in /usr/local/lib/python3.6/dist-packages (from sleap==1.1.0a10) (3.13)\n", "Collecting imgstore==0.2.9\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/39/bb/a1099d1564c1f85752251a9549037ce36021f2b318c896be2f2b067d7e9a/imgstore-0.2.9-py2.py3-none-any.whl (904kB)\n", "\u001b[K |████████████████████████████████| 911kB 47.3MB/s \n", "\u001b[?25hRequirement already satisfied: h5py==2.10.0 in /usr/local/lib/python3.6/dist-packages (from sleap==1.1.0a10) (2.10.0)\n", "Requirement already satisfied: networkx in /usr/local/lib/python3.6/dist-packages (from sleap==1.1.0a10) (2.5)\n", "Collecting scikit-video\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/b1/a6/c69cad508139a342810ae46e946ebb3256aa6e42f690d901bb68f50582e3/scikit_video-1.1.11-py2.py3-none-any.whl (2.3MB)\n", "\u001b[K |████████████████████████████████| 2.3MB 53.4MB/s \n", "\u001b[?25hRequirement already satisfied: scikit-learn in /usr/local/lib/python3.6/dist-packages (from sleap==1.1.0a10) (0.22.2.post1)\n", "Collecting tensorflow==2.3.1\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/ad/ad/769c195c72ac72040635c66cd9ba7b0f4b4fc1ac67e59b99fa6988446c22/tensorflow-2.3.1-cp36-cp36m-manylinux2010_x86_64.whl (320.4MB)\n", "\u001b[K |████████████████████████████████| 320.4MB 51kB/s \n", "\u001b[?25hCollecting jsmin\n", " Downloading https://files.pythonhosted.org/packages/17/73/615d1267a82ed26cd7c124108c3c61169d8e40c36d393883eaee3a561852/jsmin-2.2.2.tar.gz\n", "Collecting pykalman==0.9.5\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/2f/62/a4adc4516bd5974aa5583090199dd4b78d1e87018d14e9279f72ccbf0b9b/pykalman-0.9.5.tar.gz (228kB)\n", "\u001b[K |████████████████████████████████| 235kB 50.4MB/s \n", "\u001b[?25hCollecting PySide2==5.14.1\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/d3/ea/4699f9a83069751b98b038733b3123f5774da939ebba29a601964280a756/PySide2-5.14.1-5.14.1-cp35.cp36.cp37.cp38-abi3-manylinux1_x86_64.whl (165.5MB)\n", "\u001b[K |████████████████████████████████| 165.5MB 79kB/s \n", "\u001b[?25hRequirement already satisfied: imageio>=2.3.0 in /usr/local/lib/python3.6/dist-packages (from scikit-image->sleap==1.1.0a10) (2.4.1)\n", "Collecting commonmark<0.10.0,>=0.9.0\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/b1/92/dfd892312d822f36c55366118b95d914e5f16de11044a27cf10a7d71bbbf/commonmark-0.9.1-py2.py3-none-any.whl (51kB)\n", "\u001b[K |████████████████████████████████| 51kB 8.9MB/s \n", "\u001b[?25hRequirement already satisfied: pygments<3.0.0,>=2.6.0 in /usr/local/lib/python3.6/dist-packages (from rich==9.10.0->sleap==1.1.0a10) (2.6.1)\n", "Requirement already satisfied: typing-extensions<4.0.0,>=3.7.4 in /usr/local/lib/python3.6/dist-packages (from rich==9.10.0->sleap==1.1.0a10) (3.7.4.3)\n", "Collecting colorama<0.5.0,>=0.4.0\n", " Downloading https://files.pythonhosted.org/packages/44/98/5b86278fbbf250d239ae0ecb724f8572af1c91f4a11edf4d36a206189440/colorama-0.4.4-py2.py3-none-any.whl\n", "Collecting keras-applications<=1.0.8,>=1.0.7\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/71/e3/19762fdfc62877ae9102edf6342d71b28fbfd9dea3d2f96a882ce099b03f/Keras_Applications-1.0.8-py3-none-any.whl (50kB)\n", "\u001b[K |████████████████████████████████| 51kB 8.6MB/s \n", "\u001b[?25hCollecting efficientnet==1.0.0\n", " Downloading https://files.pythonhosted.org/packages/97/82/f3ae07316f0461417dc54affab6e86ab188a5a22f33176d35271628b96e0/efficientnet-1.0.0-py3-none-any.whl\n", "Collecting image-classifiers==1.0.0\n", " Downloading https://files.pythonhosted.org/packages/81/98/6f84720e299a4942ab80df5f76ab97b7828b24d1de5e9b2cbbe6073228b7/image_classifiers-1.0.0-py3-none-any.whl\n", "Collecting tensorflow-estimator<2.4.0,>=2.3.0\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/e9/ed/5853ec0ae380cba4588eab1524e18ece1583b65f7ae0e97321f5ff9dfd60/tensorflow_estimator-2.3.0-py2.py3-none-any.whl (459kB)\n", "\u001b[K |████████████████████████████████| 460kB 57.5MB/s \n", "\u001b[?25hCollecting shiboken2==5.14.1\n", "\u001b[?25l Downloading https://files.pythonhosted.org/packages/b5/9b/afca43974697e7b50ec22d3ad0c878a3d07eb89bfe75d341707ff2421772/shiboken2-5.14.1-5.14.1-cp35.cp36.cp37.cp38-abi3-manylinux1_x86_64.whl (847kB)\n", "\u001b[K |████████████████████████████████| 849kB 52.7MB/s \n", "\u001b[?25hRequirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image->sleap==1.1.0a10) (0.10.0)\n", "Building wheels for collected packages: qimage2ndarray, jsmin, pykalman\n", " Building wheel for qimage2ndarray (setup.py) ... \u001b[?25l\u001b[?25hdone\n", " Created wheel for qimage2ndarray: filename=qimage2ndarray-1.8-cp36-none-any.whl size=11696 sha256=87f58ad6078cd1396d24422430460b40c72904451c6c180741ab5a5e54eaad96\n", " Stored in directory: /root/.cache/pip/wheels/52/ef/1d/c77ef6d3d3fe72aa88f822a05e29b17e79914af69ddeff4c02\n", " Building wheel for jsmin (setup.py) ... \u001b[?25l\u001b[?25hdone\n", " Created wheel for jsmin: filename=jsmin-2.2.2-cp36-none-any.whl size=13921 sha256=cd932542124debabaff96e2a186bec8649aa42982bec713e493291fb482d191e\n", " Stored in directory: /root/.cache/pip/wheels/64/f4/de/9667d84f759289edf5442220997c6d4334637a6bb2a7b90f73\n", " Building wheel for pykalman (setup.py) ... \u001b[?25l\u001b[?25hdone\n", " Created wheel for pykalman: filename=pykalman-0.9.5-cp36-none-any.whl size=48464 sha256=865a0eb1a1bf0bc6d0bfade2108d175d3ca466c0be66c2d2fb026ac0f612555e\n", " Stored in directory: /root/.cache/pip/wheels/d9/e8/6a/553d9832679cb74a8434fa597c3abdb07313e40054a0adf9ac\n", "Successfully built qimage2ndarray jsmin pykalman\n", "Installing collected packages: numpy, opencv-python-headless, imgaug, commonmark, colorama, rich, attrs, keras-applications, efficientnet, image-classifiers, segmentation-models, qimage2ndarray, python-rapidjson, jsonpickle, cattrs, imgstore, scikit-video, tensorflow-estimator, tensorflow, jsmin, pykalman, shiboken2, PySide2, sleap\n", " Found existing installation: numpy 1.19.5\n", " Uninstalling numpy-1.19.5:\n", " Successfully uninstalled numpy-1.19.5\n", " Found existing installation: imgaug 0.2.9\n", " Uninstalling imgaug-0.2.9:\n", " Successfully uninstalled imgaug-0.2.9\n", " Found existing installation: attrs 20.3.0\n", " Uninstalling attrs-20.3.0:\n", " Successfully uninstalled attrs-20.3.0\n", " Found existing installation: tensorflow-estimator 2.4.0\n", " Uninstalling tensorflow-estimator-2.4.0:\n", " Successfully uninstalled tensorflow-estimator-2.4.0\n", " Found existing installation: tensorflow 2.4.1\n", " Uninstalling tensorflow-2.4.1:\n", " Successfully uninstalled tensorflow-2.4.1\n", "Successfully installed PySide2-5.14.1 attrs-19.3.0 cattrs-1.0.0rc0 colorama-0.4.4 commonmark-0.9.1 efficientnet-1.0.0 image-classifiers-1.0.0 imgaug-0.3.0 imgstore-0.2.9 jsmin-2.2.2 jsonpickle-1.2 keras-applications-1.0.8 numpy-1.18.5 opencv-python-headless-4.2.0.34 pykalman-0.9.5 python-rapidjson-1.0 qimage2ndarray-1.8 rich-9.10.0 scikit-video-1.1.11 segmentation-models-1.0.1 shiboken2-5.14.1 sleap-1.1.0a10 tensorflow-2.3.1 tensorflow-estimator-2.3.0\n" ], "name": "stdout" }, { "output_type": "display_data", "data": { "application/vnd.colab-display-data+json": { "pip_warning": { "packages": [ "numpy", "tensorflow" ] } } }, "metadata": { "tags": [] } } ] }, { "cell_type": "markdown", "metadata": { "id": "iq7jrgUksLtR" }, "source": [ "## Getting your training data into Colab\n", "\n", "Mount your Google Drive so that it can be accessed from Colab. You'll be prompted to log into your Google account, give Colab access, and then copy an authorization code into a field below." ] }, { "cell_type": "code", "metadata": { "id": "GBWjF4jpMG2N", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "633f6ac1-8d43-49ca-f24b-75afecbcadb4" }, "source": [ "from google.colab import drive\n", "drive.mount('/content/drive/')" ], "execution_count": 4, "outputs": [ { "output_type": "stream", "text": [ "Mounted at /content/drive/\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "KQhv_gsdJzaq" }, "source": [ "I'll assume you already know how to train using a training package on Colab. If you haven't done this already, you should take a look at our [other notebook](https://colab.research.google.com/github/murthylab/sleap-notebooks/blob/master/Training_and_inference_using_Google_Drive.ipynb) which walks you through that process in more detail.\n", "\n", "For this notebook you'll use a custom **training profile**. Training profiles are JSON files which define the hyperparameters used for training. Go follow our guide to [creating custom training profiles](https://sleap.ai/guides/custom-training.html).\n", "\n", "For this demo, I'll assume you have a `sleap-custom-training` directory in the root of your Google drive. You should copy your training package and your custom training profile(s) into that directory." ] }, { "cell_type": "markdown", "metadata": { "id": "xZ-sr67av5uu" }, "source": [ "# Training\n", "\n", "Once you have the training package and custom training profile(s) ready, you can use the command-line interface to run training.\n", "\n", "If you haven't done this already, you should take a look at our [other notebook](https://colab.research.google.com/github/murthylab/sleap-notebooks/blob/master/Training_and_inference_using_Google_Drive.ipynb) which walks you through the basic training process in more detail.\n", "\n", "When training with a custom profile, the key difference when running training with a custom training profile is that the second parameter to `sleap-train` should be the filename of your custom training profile (you can specify it with a local path if it's in the same directory as your training package since we have a `cd` to use this as the working directory).\n", "\n", "If you used the \"**Save configuration files...**\" button in the Training dialog to export custom profile(s), the filename(s) will be:\n", "\n", "- for a **bottom-up** pipeline approach: `multi_instance.json`,\n", "\n", "- for a **top-down** pipeline, you'll have a different profile for each of the models: `centroid.json` and `centered_instance.json`,\n", "\n", "- for a **single animal** pipeline: `single_instance.json`.\n", "\n", "If you created the profile by hand, it will have whatever filename you gave it.\n", "\n", "Again, you should follow our guide to [creating custom training profiles](https://sleap.ai/guides/custom-training.html) in order to generate these custom training profiles." ] }, { "cell_type": "markdown", "metadata": { "id": "zDbIRjlw_fOF" }, "source": [ "The line below assumes your custom training profile is called `multi_instance.json` and that you've copied it into the `sleap-custom-training` directory in your Google Drive, along side your training package called `dataset.pkg.slp`. Edit the filenames in the command to match your files." ] }, { "cell_type": "code", "metadata": { "id": "QKf6qzMqNBUi" }, "source": [ "!cd \"/content/drive/My Drive/sleap-custom-training/\"; \\\n", "sleap-train multi_instance.json dataset.pkg.slp" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "whOf8PaFxYbt" }, "source": [ "Training should run just as it does when using a default training profile. The trained model will be in a subdirectory, and the name of the subdirectory is determined by the `run_name`, `run_name_prefix` and `run_name_suffix` specified in the training profile.\n", "\n", "Take a look at your Google Drive and you should see it. You can copy this subdirectory and use it for running inference on Colab, your local machine, or anywhere else you have SLEAP installed." ] } ] }