2nd Workshop on Maritime Computer Vision (MaCVi)
MaCVi 2024 Challenges have concluded. Check the results on the leaderboards. Thank you for participating!
BoaTrack - Multi-Object Tracking of Boats
Ask for help
Download the BoaTrack dataset on the dataset page
. Upload a json-file with predictions on the test set on the
BoaTrack is a new MOT dataset from the viewpoint of USVs. The goal of this benchmark is to
advance computer vision algorithms in autonomous boating and boating assistance systems. It is
aimed at detecting and tracking boats and other objects (such as buoys) in open
water and dock scenarios. For this competition, only boats shall be tracked.
We provide tracking id labels alongside the bounding box labels for
several videos where the task is to track boats.
Create a multi-object tracker that, given a video, outputs bounding boxes and tracking ids for every object
(boat). This is a short-term tracking task, i.e. objects that disappear from the scene need not be re-identified if they reapper in the same video clip again at a later point.
We provide 3 clips in the test set. We withold the annotations. On the server-side, we compare your predictions against the annotations. Please note, that we do not provide any train or val videos. This means, you need to train a detector on other datasets, such as LaRS
(from the other competition). You are free to use any other publicly available dataset as long as you provide the link/source to it in the upload.
Please contact me any time if you find
some irregularities (email@example.com).
We evaluate your predictions on HOTA, MOTA, IDF1, MOTP, MT, ML, FP, FN, Recall, Precision, ID Switches, Frag. The
metric for winning is HOTA. In case of tie, MOTA is the tiebreaker.
Furthermore, we require every participant to submit information on the speed of their method measured in frames per
Please also indicate the hardware that you used. Lastly, you should indicate which data sets (also for pretraining) you
used during training
and whether you used meta data.
In order to participate, you can perform the following steps:
- Download the dataset BoaTrack (aka the three videos) on the dataset page.
- Train a detector+multi-object tracker of your choice on any publicly available dataset (e.g. LaRS).
You need to create a json-prediction file on the test set for upload. In the following, we will illustrate the structure of the
json based on an example:
See the BoaTrack_example_submission.json. You can load it in Python via
This yields you a list where you find that this list is of length = number of frames in entire test set (all
videos), i.e. 10,656.
The frames of the videos should be ordered according to the following list:
Each entry in the above list contains a single list. This list contains a single list or multiple lists.
If you found no object on that frame, then there is only a single empty list, so that the entry is:
For example, in the sample submission the first frame does not contain any prediction, i.e.
If you found objects, then there is one or more lists contained. For example, in the sample submission the 901th
frame contains three predicted objects:
data=[[[4, 372.8047619047619, 398.5, 393.4, 429.5, 1.0],
[5, 189.05714285714268, 374.0000000000001, 216.57142857142858, 408.42857142857144, 1.0],
[6, 131.71428571428578, 354.0, 151.39999999999998, 400.9, 1.0]]]
Each object is itself encoded as a list. For example, see the first object in the 901th frame:
data=[4, 372.8047619047619, 398.5, 393.4, 429.5, 1.0]
The numbers mean the following:
The object id must be an integer. The confidence should be a float between 0 and 1.
The coordinates should be self-explanatory.
Let us know if you need further information.
- Upload your json-file along with all the required information in the form here. You need
to register first.
- If you provided a valid file and sufficient information, you should see your method on the leaderboard. Since it only has three detections, it will
be at the bottom of the leaderboard. Now,
you can reiterate. You may upload
at most once per day per challenge. Note that the upload can take a few minutes.
Terms and Conditions
- Submissions must be made before the deadline as listed on the
- You may submit at most once per day per challenge
- The winners are determined by the HOTA (tiebreaker MOTA) metric
- You are allowed to use any publicly available data for training but you must list them at the time of upload.
This also applies to pretraining.
- Note that we (as organizers) may upload models for this challenge BUT we do not compete for a winning position
(i.e. our models do not count on the leaderboard and merely serve as references). Thus, if your method is worse
(in any metric)
than one of the organizer's, you are still encouraged to submit your method as you might win.
In case of any questions regarding the challenge modalities or uploads, please direct them to Benjamin Kiefer.