5^th Workshop on Maritime Computer Vision (MaCVi)

MaCVi @ WACV 2027

Challenges / EURS

EURS: Embedded Underwater Restoration and Segmentation

Underwater imagery is degraded by color distortion, light scattering, and low visibility, which makes downstream perception tasks — like identifying and localizing marine life — substantially harder than on clean images. EURS combines two coupled tasks into a single challenge: restoring a degraded underwater image, and segmenting it into marine-relevant classes.

Track 1: Public Dev-Set Leaderboard

Quick links: Dataset download Submit Leaderboard Ask for help

Overview

Both restoration and segmentation are scored against the same Dev-split ground truth, so a submission needs to get both the pixel-level restoration and the semantic understanding right. This leaderboard runs entirely server-side against the public Dev split, with no device or power constraints and no submission limit beyond a daily cap — use it to iterate while developing. It does not require any special hardware and carries less weight than Track 2 towards final standings.

Task

Given a degraded underwater image, participants must produce two outputs for it:

• A restored image — the same scene with underwater degradation (color cast, haze, low contrast) corrected.
• A segmentation mask — a single-channel image labeling every pixel with one of 5 classes: background, coral, fish, elasmobranch (sharks and rays), other.

Dataset

The dataset consists of real underwater scenes, each provided as one clean ground-truth image/mask pair and multiple synthetically degraded versions of that scene (simulating different water conditions and depths). The Dev split — the split scored by this leaderboard — is available as a direct download:

Download the EURS Dev-split dataset

Evaluation Metrics

This leaderboard's primary ranking metric is the same composite score used by Track 2:

composite = 0.6 × Qseg + 0.3 × (0.45 × PSNRnorm + 0.45 × SSIM + 0.10 × sharpness_ratio) + 0.1 × NRquality

where Qseg = F1 × mIoU × coverage_penalty rewards submissions that get both segmentation coverage (F1) and per-class accuracy (mIoU) right, and NRquality is the mean of UIQM/UCIQE/CCF, each normalized over an empirically-observed range (see Track 2's Evaluation Metrics for the full rationale, including why that no-reference group is capped at a small combined weight). The full set of metrics reported:

• Q_seg, mIoU, F1 — segmentation accuracy across the 5 classes.
• PSNR, SSIM, sharpness_ratio — restoration fidelity against the ground-truth clean image and an anti-blur signal.
• UIQM, UCIQE, CCF — underwater-specific, no-reference image quality metrics, contributing to the composite above at a small combined weight.

Submission Zipfile Preparation Guidelines

• Submit a single .zip archive containing two folders at its root: restored/ and seg_masks/.
• For every Dev-split degraded image, include one output in each folder, named after the degraded image (matching filename stem, any common image extension for restored/).
• Segmentation masks must be single-channel PNGs with integer pixel values 0–4 (background/coral/fish/elasmobranch/other, in that order). Masks with out-of-range values or the wrong number of channels will be rejected.
• Every output must be resized to exactly the resolution the evaluator scores at (longest side capped at 1024px, aspect-preserved from the original scene) — the evaluator does not resize submissions itself, so a mismatched size is rejected outright, not silently corrected. Submissions larger than 2048px on the longest side are rejected before scoring even starts.
• Run the starter kit's resize_for_submission.py as the mandatory last step of your pipeline to get this exactly right — it computes each scene's exact target size from your downloaded degraded images and resizes your raw outputs to match, so you never have to compute this yourself.
• A submission must include both outputs for every Dev-split image. Incomplete submissions are rejected outright rather than silently scored on a subset.

Participate

To participate in the challenge follow these steps:

1. Download the EURS Dev-split dataset.
2. Train a model (or pipeline) that restores the degraded input and predicts a per-pixel class mask. You may use additional publicly available training data, but must disclose it at submission time.
3. Generate restored/ and seg_masks/ outputs for every Dev-split image, following the format above.
4. Run resize_for_submission.py on your outputs — this is mandatory, not optional; the evaluator rejects any output whose size doesn't exactly match what it expects, and this script is the supported way to get that right without computing it yourself.
5. Package both resized folders into a single .zip with restored/ and seg_masks/ at the archive root (no extra nesting).
6. Upload your .zip file on the upload page. You'll need to register to submit.
   • Evaluation runs on the server and typically takes 10–20 minutes for a full Dev-split submission. Refresh the dashboard page to see results; it will also show details on any failed submission (hover over the error icon).
   • You may submit at most 10 times per day to public leaderboard. Failed submissions still count towards this limit, so double-check your archive format before uploading.

Terms and Conditions

• This public leaderboard is for self-evaluation during development and counts towards challenge standings, but Track 2 (embedded, Jetson AGX Orin) carries more weight towards prizes and the results paper.
• You are allowed to use additional publicly available data for training but must disclose it at the time of submission. This also applies to pre-training.
• Note that we (as organizers) may upload reference models for this challenge, but they do not compete for a winning position — they serve only as a reference point on the leaderboard.

Track 2: Embedded (Jetson AGX Orin)

Quick links: Submit Leaderboard Self-check your model's GFLOPs/params Ask for help

Overview

Track 2 runs your submitted ONNX model itself — not pre-computed prediction images — directly on our judging hardware: a Jetson AGX Orin, locked to JetPack/L4T 7.2.1 (r39.2.1), stock nvpmodel MODE_30W, with a 16GB memory cap. The same environment and thresholds apply to every submission, published here so everyone targets the same bar. The exact model you submit produces the restored/ and seg_masks/ outputs used for accuracy scoring and is the thing timed and power-measured for the efficiency metrics — this ties both scores to the same artifact, so a fast dummy model paired with offline-computed predictions can't game the efficiency numbers. Accuracy is scored on a private, held-out Test split (never used by Track 1's Dev-split leaderboard), so repeated Track 1 iteration can't leak information about Track 2's scoring data.

Because evaluation involves compiling and running your model on real physical hardware, results are not immediate — a submission typically takes 15–20 minutes to come back (engine compile, a sustained-load measurement window, then real inference over the full Test split). Refresh the dashboard to see results once ready.

Submission Format

• Submit a single self-contained .onnx file — not a zip, not prediction images. Embed all weights in the same file (save_as_external_data=False if exporting with torch.onnx.export or via onnx.save_model); a companion .onnx.data file is fragile to package and transfer correctly, so it is not supported for submission.
• The graph must use a static input shape (no dynamic axes). This is required for exact compute counting below, and for the TensorRT engine build on the judging hardware.
• You are free to choose your own input resolution, as long as the model clears the compute cap below at that resolution. The organizers' own reference baseline runs at 384×768, given here as a concrete data point, not a requirement.

Design-Space Cap (checked before your model ever touches the hardware)

Your submitted graph's total compute and parameter count are checked statically, before anything runs on the Jetson:

• ≤ 12 GFLOPs (Conv/Gemm/MatMul, computed via static ONNX shape inference)
• ≤ 1.0M parameters

Run the same check yourself locally, on your own machine — wherever you already exported the ONNX file — before submitting, so you don't spend a submission slot on a model that would fail this stage: download check_submission_flops.py, install its one dependency (pip install onnx), then run python check_submission_flops.py --onnx your_model.onnx (defaults already match the locked caps above). It's a plain static analysis of the ONNX graph — no GPU, no Jetson, and no server round-trip needed to use it. A local PASSED only means your model clears this one design-space gate — it doesn't run your model, so it can't tell you anything about accuracy or real-hardware speed/energy. Those only come back from an actual Track 2 submission.

Evaluation Pipeline

Every submission goes through the same stages, in order, stopping at the first failure:

1. Static compute/parameter check — the GFLOPs/params cap above. Fails fast, before your model ever reaches the Jetson.
2. TensorRT engine compile on the judging hardware — fails if your ONNX graph doesn't compile for the target TensorRT version.
3. Quick throughput check — a short sanity pass.
4. Sustained-load measurement — throughput and energy-per-frame under continuous real load, not a single burst.
5. Real inference over the private Test split's degraded images, run directly on the Jetson.
6. Accuracy scoring of those on-device outputs against ground truth.
7. A combined result is returned: either full scores, or a rejection naming the exact stage and reason.

Evaluation Metrics

Track 2's primary ranking metric is the composite score:

composite = 0.6 × Qseg + 0.3 × (0.45 × PSNRnorm + 0.45 × SSIM + 0.10 × sharpness_ratio) + 0.1 × NRquality

where Qseg = F1 × mIoU × coverage_penalty (the same per-class IoU floor as Track 1 — drives Q_seg to exactly 0 if any present class is essentially ignored), PSNRnorm normalizes PSNR over a 15–35dB window, and NRquality is the mean of UIQM/UCIQE/CCF, each normalized over an empirically-observed range. This no-reference group is deliberately capped at a small combined weight: no-reference image quality metrics are known to be gameable (e.g. via contrast/saturation stretching) without reflecting genuine restoration quality — confirmed on our own data, where CCF actually scores higher on degraded inputs than on an honestly restored image. The full set of metrics reported:

• Q_seg, mIoU, F1 — segmentation, same definitions as Track 1.
• PSNR, SSIM, sharpness_ratio — restoration fidelity and an anti-blur signal, same definitions as Track 1.
• UIQM, UCIQE, CCF — no-reference underwater image quality metrics, same definitions as Track 1, here contributing to the composite at low combined weight as described above.
• GFLOPs and Params — the design-space numbers above, reported alongside your score.
• Throughput (QPS) and Energy (mJ/frame) — real, sustained-load measurements on the judging hardware. These are reported for transparency and do not currently gate submissions (no pass/fail threshold is enforced on them yet) — only the GFLOPs/params cap above is actually enforced.

Participate

1. Train your model as for Track 1, then export it to a single self-contained .onnx file with a static input shape.
2. Run check_submission_flops.py locally and confirm it prints PASSED.
3. Upload the .onnx file directly on the upload page. You'll need to register to submit.
   • You may submit at most once per day per team to this track — it runs on a single physical judging device, so slots are scarce by design. Failed submissions still count towards this limit, so self-check locally first.
   • Results are asynchronous and typically take 15–20 minutes. Refresh the dashboard page to see them; it will also show the failing stage and reason for any rejected submission.

Terms and Conditions

• Track 2 carries more weight than the Track 1 public leaderboard towards final challenge standings, prizes, and the results paper.
• Same data-disclosure rule as Track 1: additional publicly available training/pre-training data is allowed but must be disclosed at submission time.
• Organizer reference models may appear on this leaderboard for comparison; they do not compete for a winning position.