cvat/site/content/en/docs/contributing/development-environment.md

299 lines
9.9 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters!

This file contains ambiguous Unicode characters that may be confused with others in your current locale. If your use case is intentional and legitimate, you can safely ignore this warning. Use the Escape button to highlight these characters.

---
title: 'Development environment'
linkTitle: 'Development environment'
weight: 2
description: 'Installing a development environment for different operating systems.'
---
### Setup the dependencies:
- Install necessary dependencies:
Ubuntu 22.04/20.04
```bash
sudo apt-get update && sudo apt-get --no-install-recommends install -y build-essential curl git python3-dev python3-pip python3-venv python3-tk libldap2-dev libsasl2-dev libgeos-dev cargo
```
```bash
# Install Node.js 20
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo bash -
sudo apt-get install -y nodejs
sudo npm -g install corepack # ensure corepack is installed
```
MacOS 10.15
```bash
brew install git python pyenv redis curl openssl node sqlite3 geos rust
```
Arch Linux
```bash
# Update the system and AUR (you can use any other AUR helper of choice) first:
sudo pacman -Syyu
pikaur -Syu
```
```bash
# Install the required dependencies:
sudo pacman -S base-devel curl git redis cmake gcc python python-pip tk libldap libsasl pkgconf ffmpeg geos openldap rust
```
```bash
# CVAT supports only Python 3.10, so install it if you dont have it:
pikaur -S python310
```
```bash
# Install Node.js and npm
sudo pacman -S nodejs-lts-gallium npm
sudo npm -g install corepack # ensure corepack is installed
```
### Migration to Yarn Modern
We have updated our Yarn version from Classic (1.x) to Modern.
If you are still using CVAT with Yarn Classic you need to first migrate:
```bash
# If yarn --version shows 1.x
# remove old yarn
sudo npm uninstall -g yarn
# Ensure corepack is installed
sudo npm install -g corepack
# Enable new yarn
yarn --version # should show 4.x
```
- Install Chrome
- Install [VS Code](https://code.visualstudio.com/docs/setup/linux#_debian-and-ubuntu-based-distributions).
- Install the following VScode extensions:
- [JavaScript Debugger](https://marketplace.visualstudio.com/items?itemName=ms-vscode.js-debug)
- [Python](https://marketplace.visualstudio.com/items?itemName=ms-python.python)
- [ESLint](https://marketplace.visualstudio.com/items?itemName=dbaeumer.vscode-eslint)
- [Stylelint](https://marketplace.visualstudio.com/items?itemName=stylelint.vscode-stylelint)
- [Trailing Spaces](https://marketplace.visualstudio.com/items?itemName=shardulm94.trailing-spaces)
- [Code Spell Checker](https://marketplace.visualstudio.com/items?itemName=streetsidesoftware.code-spell-checker)
- Make sure to use Python 3.10.0 or higher
```bash
python3 --version
```
- Install CVAT on your local host:
```bash
git clone https://github.com/cvat-ai/cvat
cd cvat && mkdir logs keys
python3 -m venv .env
. .env/bin/activate
pip install -U pip wheel setuptools
pip install -r cvat/requirements/development.txt -r dev/requirements.txt
```
Note that the `.txt` files in the `cvat/requirements` directory
have pinned dependencies intended for the main target OS/Python version
(the one used in the main Dockerfile).
If you're unable to install those dependency versions,
you can substitute the corresponding `.in` files instead.
That way, you're more likely to be able to install the dependencies,
but their versions might not correspond to those used in production.
{{% alert title="Note for Mac users" color="primary" %}}
If you have any problems with installing dependencies from
`cvat/requirements/*.txt`, you may need to reinstall your system python
In some cases after system update it can be configured incorrectly and cannot compile
some native modules
Make sure Homebrew lib path is in `DYLD_LIBRARY_PATH`.
For Apple Silicon: `export DYLD_LIBRARY_PATH=/opt/homebrew/lib:$DYLD_LIBRARY_PATH`
Homebrew will install FFMpeg 5.0 by default, which does not work, so you should install 4.X.
You can install older 4.X FFMpeg using Homebrew like that:
```bash
cd "$(brew --repo homebrew/core)"
git checkout addd616edc9134f057e33694c420f4900be59db8
brew unlink ffmpeg
HOMEBREW_NO_AUTO_UPDATE=1 brew install ffmpeg
git checkout master
```
if you are still facing error `Running setup.py install for av ... error`, you may
try more radical variant
```bash
cd "$(brew --repo homebrew/core)"
git checkout addd616edc9134f057e33694c420f4900be59db8
brew uninstall ffmpeg --force
HOMEBREW_NO_AUTO_UPDATE=1 brew install ffmpeg
git checkout master
```
If you faced with error `Failed building wheel for h5py`, you may need install `hdf5`
```bash
brew install hdf5
export HDF5_DIR="$(brew --prefix hdf5)"
pip install --no-binary=h5py h5py
```
If you faced with error
`OSError: Could not find library geos_c or load any of its variants ['libgeos_c.so.1', 'libgeos_c.so']`.
You may fix this using
```bash
sudo ln -s /opt/homebrew/lib/libgeos_c.dylib /usr/local/lib
```
{{% /alert %}}
{{% alert title="Note for Arch Linux users" color="primary" %}}
Because PyAV as of version 10.0.0 already [works](https://github.com/PyAV-Org/PyAV/pull/910)
with FFMPEG5, you may consider changing the `av` version requirement
in `/cvat/cvat/requirements/base.txt` to 10.0.0 or higher.
Perform this action before installing cvat requirements from the list mentioned above.
{{% /alert %}}
- Install [Docker Engine](https://docs.docker.com/engine/install/ubuntu/) and [Docker Compose](https://docs.docker.com/compose/install/)
- Start service dependencies:
```bash
docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d --build \
cvat_opa cvat_db cvat_redis_inmem cvat_redis_ondisk cvat_server
```
Note: this runs an extra copy of the CVAT server in order to supply rules to OPA.
If you update the OPA rules, rerun this command to recreate the server image and container.
Note: to stop these services, use
`docker compose -f docker-compose.yml -f docker-compose.dev.yml down`.
You can add `-v` to remove the data, as well.
- Apply migrations and create a super user for CVAT:
```bash
python manage.py migrate
python manage.py migrateredis
python manage.py collectstatic
python manage.py syncperiodicjobs
python manage.py createsuperuser
```
- Install npm packages for UI (run the following command from CVAT root directory):
```bash
corepack enable yarn
yarn --immutable
```
{{% alert title="Note for Mac users" color="primary" %}}
If you faced with error
`Node Sass does not yet support your current environment: OS X 64-bit with Unsupported runtime (57)`,
read this article [Node Sass does not yet support your current environment](https://marketplace.visualstudio.com/items?itemName=msjsdiag.debugger-for-chrome)
{{% /alert %}}
### Run CVAT
- Start npm UI debug server (run the following command from CVAT root directory):
- If you want to run CVAT in localhost:
```sh
yarn run start:cvat-ui
```
- If you want to access CVAT from outside of your host:
```sh
CVAT_UI_HOST='<YOUR_HOST_IP>' CVAT_UI_PORT='<YOUR_PORT>' yarn run start:cvat-ui
```
- Open a new terminal window.
- Run VScode from the virtual environment (run the following command from CVAT root directory):
```sh
source .env/bin/activate && code
```
- Inside VScode, Open CVAT root dir
- Select `server: debug` configuration and run it (F5) to run REST server and its workers
- Make sure that `Uncaught Exceptions` option under breakpoints section is unchecked
- If you choose to run CVAT in localhost: Select `server: chrome` configuration and run it (F5) to open CVAT in Chrome
- Alternative: If you changed CVAT_UI_HOST just enter `<YOUR_HOST_IP>:3000` in your browser.
{{% alert title="Note for Mac users" color="primary" %}}
You may have a permission denied problem starting the server because **AirPlay Receiver** running on port 5000/7000.
Turn off AirPlay Receiver:
_Go to System Settings__General__AirDrop & Handoff__Untick Airplay Receiver_.
{{% /alert %}}
You have done! Now it is possible to insert breakpoints and debug server and client of the tool.
Instructions for running tests locally are available {{< ilink "/docs/contributing/running-tests" "here" >}}.
## Note for Windows users
You develop CVAT under WSL (Windows subsystem for Linux) following next steps.
- Install WSL using [this guide](https://docs.microsoft.com/en-us/windows/wsl/install-win10).
- Following this guide install Ubuntu 18.04 Linux distribution for WSL.
- Run Ubuntu using start menu link or execute next command
```powershell
wsl -d Ubuntu-18.04
```
- Install the VS Code extension for WSL, which helps you to open VS Code correctly inside WSL.
You can find the extension [here](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-wsl).
- Run all commands from this installation guide in WSL Ubuntu shell.
- You might have to manually start the redis server in wsl before you can start the configuration inside
Visual Studio Code. You can do this with `sudo service redis-server start`. Alternatively you can also
use a redis docker image instead of using the redis-server locally.
## Note for Mac users
- You might have to manually start the redis server. You can do this with `redis-server`.
Alternatively you can also use a redis docker image instead of using the redis-server locally.
## Note for Arch Linux users
- You need to start `redis` and `docker` services manually in order to begin debugging/running tests:
```bash
sudo systemctl start redis.service
sudo systemctl start docker.service
```
## CVAT Analytics Ports
In case you cannot access analytics, check if the following ports are open:
```yml
cvat_vector:
ports:
- '8282:80'
cvat_clickhouse:
ports:
- '8123:8123'
```
In addition, you can completely disable analytics if you don't need it by deleting the following data from
[launch.json](https://github.com/cvat-ai/cvat/blob/develop/.vscode/launch.json):
```json
"DJANGO_LOG_SERVER_HOST": "localhost",
"DJANGO_LOG_SERVER_PORT": "8282"
```
Analytics on GitHub:
[Analytics Components](https://github.com/cvat-ai/cvat/tree/develop/components/analytics)