3^rd Workshop on Maritime Computer Vision (MaCVi)

Challenges / Marine Vision Restoration Challenge

Marine Vision Restoration Challenge (MVRC)

Quick links: Dataset download Submit Leaderboard Ask for help

Quick Start

1. Download the RSUIGM (extended version) dataset on the dataset page.
2. Train your model on the RSUIGM training set.
3. Upload a .zip file with predictions on the upload page.

Overview

The MarineVision Restoration Challenge (MVRC), the first of its kind as a part of MaCVi Workshop @ WACV 2025, aims to restore underwater images for marine species detection and addresses key issues of marine exploration and conservation efforts. The underwater environment presents unique difficulties for image capture, including poor visibility, color distortion, and light scattering, which significantly hinder accurate marine species detection. This challenge aims to bring researchers from around the world to develop innovative solutions for improving the clarity of underwater images. By restoring the image quality and enabling precise marine species detection, participants will contribute to the broader goal of supporting marine biodiversity studies and ecosystem monitoring. The challenge consists of two phases. In the first phase, we focus on restoring degraded underwater images, and in the second phase, we culminate with the detection and classification of marine species as shown in overview figure.

We provide the dataset comprising of multiple degradation patterns resembling tints and properties of water from various geographic locations like seas, oceans, ponds, lakes etc. The dataset provided comprises of all required annotations and the metadata for training restoration and detection models.

Task

Given degraded underwater images/videos/frames, the participant needs to develop a method for restoring these images/videos/frames, and subsequently use an object detection model to detect and classify the marine species in the images/videos/frames.

Dataset

We provide Realistic Synthetically Generated Underwater Paired Dataset (RSUIGM). The dataset comprises of 6000 images which has characteristics of both oceanic beds, and coastal bed region along with varying depths. This data simulates water properties from various geographic locations. Each pair of data also has corresponding annotation for training object detection models. Each scene in the dataset comprises of 1 ground-truth image and 200 input images (1:200 ratio between ground-truth and input images). The dataset hierarchy is as follows:
|——Root Dir
|              | - Train
|                       | - Ground-truth Images
|                       | - Input Images
|                       | - Labels for OD
|              | - Validation
|                       | - Input Images
|              | - Test
|                       | - Input Images

Evaluation Metrics

For evaluating the submissions, we use a combination of metrics tailored to both image restoration and marine species detection/classification tasks. For underwater image restoration, the results will be primarily assessed using UIQM, UCIQE, and CCF metrics, which focus on underwater image quality. In the detection and classification phase, performance will be evaluated using Mean IoU (mIoU), F1 Score, and mean Average Precision (mAP), which provides a comprehensive assessment for detection and classification models.

Submission Zipfile Preparation Guidelines:

For uninterrupted evaluation of object-detection, please follow the modified version of Ultralytics yolo v5 model to generate predictions.csv file.

Once the "predictions.csv" file is generated, omit the last floating value (confidence score) from predicted bounding box data column, and retain only the values corresponding to "class_lbl x_center y_center height width" values as our submission system will only accept the 4 values. You are likely to face errors evaluating your predictions incase of improper submission formats.

The participants also also requested to exclude the 'labels' folder generated from the yolo v5 model given above for generating zip file.

Participate

To participate in the challenge follow these steps:

1. Download the RSUIGM (extended version) dataset on the dataset page.
2. Train an underwater restoration model on the RSUIGM training set. You can also use additional publicly available training data, but must disclose it during submission.
   • Note: the MVRC - Starter Kit may be a good starting point for the development of your model. The repository contains the scripts for inferring on RSUIGM dataset.
3. Generate restored images and detection predictions on the RSUIGM test images.
   • The restored images should be stored as .jpg files same name as input image.
   • The object detection predictions should be stored above mentioned format
4. For Object Detection and Classification prediction formats and examples, please refer the MVRC - Starter Kit
5. Create a submission .zip archive with your restored image files and object detection predictions(we use YOLO format for evaluation).
   • The prediction files should be placed directly in the root of the .zip file (no extra directories).
   • Refer to the example submission file for additional information ( mvrc_example_submission.zip).

Terms and Conditions

• Submissions must be made before the deadline as listed on the dates page. Submissions made after the deadline will not count towards the final results of the challenge.
• Submissions are limited to three per day per challenge. Failed submissions do not count towards this limit.
• The winner is determined by the as per the evaluation scheme mentioned in Evaluation Metrics Section of this webpage.
• You are allowed to use additional publicly available data for training but you must disclose them at the time of upload. This also applies to pre-training.
• In order for your method to be considered for the winning positions and included in the results paper, you will be required to submit a short report describing your method. More information in regards to this will be released towards the end of the challenge.
• Note that we (as organizers) may upload models for this challenge, BUT we do not compete for a winning position (i.e. our models do not count on the leaderboard and merely serve as references). Thus, if your method is worse (in any metric) than one of the organizer's, you are still encouraged to submit your method. Methods that were submitted as part of the MaCVi 2024 challenge will be marked on the leaderboards.

In case of any questions regarding the challenge datasets or submission, please join the MaCVi Support forum.