3rd Workshop on Maritime Computer Vision (MaCVi)

Challenges / Marine Vision Restoration Challenge

Marine Vision Restoration Challenge (MVRC)

Quick links: Dataset download Submit Leaderboard Ask for help

Quick Start

  1. Download the RSUIGM (extended version) dataset on the dataset page.
  2. Train your model on the RSUIGM training set.
  3. Upload a .zip file with predictions on the upload page.

Overview

The MarineVision Restoration Challenge (MVRC), the first of its kind as a part of MaCVi Workshop @ WACV 2025, aims to restore underwater images for marine species detection and addresses key issues of marine exploration and conservation efforts. The underwater environment presents unique difficulties for image capture, including poor visibility, color distortion, and light scattering, which significantly hinder accurate marine species detection. This challenge aims to bring researchers from around the world to develop innovative solutions for improving the clarity of underwater images. By restoring the image quality and enabling precise marine species detection, participants will contribute to the broader goal of supporting marine biodiversity studies and ecosystem monitoring. The challenge consists of two phases. In the first phase, we focus on restoring degraded underwater images, and in the second phase, we culminate with the detection and classification of marine species as shown in overview figure.

We provide the dataset comprising of multiple degradation patterns resembling tints and properties of water from various geographic locations like seas, oceans, ponds, lakes etc. The dataset provided comprises of all required annotations and the metadata for training restoration and detection models.

Task

Given degraded underwater images/videos/frames, the participant needs to develop a method for restoring these images/videos/frames, and subsequently use an object detection model to detect and classify the marine species in the images/videos/frames.

Dataset

We provide Realistic Synthetically Generated Underwater Paired Dataset (RSUIGM). The dataset comprises of 6000 images which has characteristics of both oceanic beds, and coastal bed region along with varying depths. This data simulates water properties from various geographic locations. Each pair of data also has corresponding annotation for training object detection models. Each scene in the dataset comprises of 1 ground-truth image and 200 input images (1:200 ratio between ground-truth and input images). The dataset hierarchy is as follows:
|——Root Dir
|              | - Train
|                       | - Ground-truth Images
|                       | - Input Images
|                       | - Labels for OD
|              | - Validation
|                       | - Input Images
|              | - Test
|                       | - Input Images

Evaluation Metrics

For evaluating the submissions, we use a combination of metrics tailored to both image restoration and marine species detection/classification tasks. For underwater image restoration, the results will be primarily assessed using UIQM, UCIQE, and CCF metrics, which focus on underwater image quality. In the detection and classification phase, performance will be evaluated using Mean IoU (mIoU), F1 Score, and mean Average Precision (mAP), which provides a comprehensive assessment for detection and classification models.

Submission Zipfile Preparation Guidelines:

  • For uninterrupted evaluation of object-detection, please follow the modified version of Ultralytics yolo v5 model to generate predictions.csv file.
  • Once the "predictions.csv" file is generated, omit the last floating value (confidence score) from predicted bounding box data column, and retain only the values corresponding to "class_lbl x_center y_center height width" values as our submission system will only accept the 4 values. You are likely to face errors evaluating your predictions incase of improper submission formats.
  • The participants also also requested to exclude the 'labels' folder generated from the yolo v5 model given above for generating zip file.
  • Participate

    To participate in the challenge follow these steps:

    1. Download the RSUIGM (extended version) dataset on the dataset page.
    2. Train an underwater restoration model on the RSUIGM training set. You can also use additional publicly available training data, but must disclose it during submission.
      • Note: the MVRC - Starter Kit may be a good starting point for the development of your model. The repository contains the scripts for inferring on RSUIGM dataset.
    3. Generate restored images and detection predictions on the RSUIGM test images.
      • The restored images should be stored as .jpg files same name as input image.
      • The object detection predictions should be stored above mentioned format
    4. For Object Detection and Classification prediction formats and examples, please refer the MVRC - Starter Kit
    5. Create a submission .zip archive with your restored image files and object detection predictions(we use YOLO format for evaluation).
      • The prediction files should be placed directly in the root of the .zip file (no extra directories).
      • Refer to the example submission file for additional information ( mvrc_example_submission.zip).

    Terms and Conditions

    In case of any questions regarding the challenge datasets or submission, please join the MaCVi Support forum.