Find the data, choose a task and check its evaluation protocol. The catalogue brings together 17 dataset families and 23 evaluation tracks across MaCVi editions.
Sample: VisLab / Seagull. Dataset access is by request; no MaCVi leaderboard.
From data to a benchmark result
Get the data. Follow the dataset links and its access terms. Some resources require an access request.
Check the protocol. Use the correct split, label IDs, calibration and preprocessing. Validate locally with the track's starter kit where available.
Submit to the right track. A MaCVi account is needed for hosted submissions. Check the upload page and challenge rules for current limits and entry periods.
OpenReview profiles are used for workshop-paper and summary-paper authorship; they are separate from MaCVi dataset and benchmark accounts. Dataset access does not imply an open competition or automatic paper inclusion.
Object detection: COCO prediction JSON
SeaDronesSee detection v1/v2 and the thermal detection track use COCO-style detection results: one object per array entry. Match the image and category IDs to the selected dataset.
Boxes use pixel coordinates [x, y, width, height]. Check the task's categories and sample file; this example is not a submission for every detection benchmark.
SeaDronesSee single-object tracking: per-sequence text files
The test set has 80 clips. Put 1.txt through 80.txt at the root of a ZIP archive. Each file has one row per frame: x,y,width,height in pixels. Follow the sequence/frame mapping supplied with the dataset.
For standard SeaDronesSee MOT, submit a ZIP with one [video_id].txt file per sequence at its root. Each row is frame_id,object_id,x,y,width,height. Frame IDs follow the dataset annotations; object IDs are assigned by your tracker.
BoaTrack and re-identification have their own protocol. In particular, the re-identification track accepts JSON; do not reuse another track's ZIP format without checking its instructions.
LaRS semantic and panoptic segmentation: different mask encodings
Use one PNG prediction per evaluation image, with matching filenames, at the root of a ZIP archive.
Semantic: water (41,167,224), obstacles (247,195,37), sky (90,75,164).
Panoptic: red encodes the class ID; green and blue encode instance_id = G × 256 + B. Use the published class IDs: static obstacles 1, water 3, sky 5, boat/ship 11, row boat 12, paddle board 13, buoy 14, swimmer 15, animal 16, float 17, other 19; 0 is void.
Boat-MNIST, BoaTetection, distance estimation and the embedded tracks evaluate uploaded models. Use their required input/output shapes, normalisation, export settings and validation steps.
MULTIAQUA, restoration, vision-to-chart, visual odometry and Diver3D each define a separate prediction or model contract. Open the dataset's “metrics & formats” above, then follow its protocol and starter kit.
