submit question

229 views
Skip to first unread message

yp lin

unread,
Nov 18, 2024, 8:48:59 PM11/18/24
to MaCVi Support
where can i submit result for the track " Marine Vision Restoration Challenge "

Nikhil Akalwadi

unread,
Nov 19, 2024, 5:19:23 PM11/19/24
to yp lin, MaCVi Support
We are working on the submission page, should be up very soon. 
Inconvenience regretted. 

MVRC Team

On 19 Nov 2024, at 07:19, yp lin <ilyf...@gmail.com> wrote:

where can i submit result for the track " Marine Vision Restoration Challenge "

--
Sie erhalten diese Nachricht, weil Sie in Google Groups E-Mails von der Gruppe "MaCVi Support" abonniert haben.
Wenn Sie sich von dieser Gruppe abmelden und keine E-Mails mehr von dieser Gruppe erhalten möchten, senden Sie eine E-Mail an macvi-suppor...@googlegroups.com.
Diese Diskussion finden Sie unter https://groups.google.com/d/msgid/macvi-support/04167105-6112-44ce-9ba4-7dabdb4f8fefn%40googlegroups.com.
Weitere Optionen finden Sie unter https://groups.google.com/d/optout.

yp lin

unread,
Dec 1, 2024, 7:14:30 AM12/1/24
to MaCVi Support
I have a new problem, the original image format that needs to be repaired is jpg, but the instructions indicate that we need to submit png? Are both image formats allowed?

Nikhil Akalwadi

unread,
Dec 1, 2024, 7:16:55 AM12/1/24
to yp lin, MaCVi Support
Hi, 
Submission requires jpg files. We will update the instructions. 


Weitere Optionen finden Sie unter https://groups.google.com/d/optout.

Regards,
--
Nikhil Neelkanth Akalwadi
Researcher
(M) +91 87921 88443 
https://nikhilakalwadi.github.io






yp lin

unread,
Dec 2, 2024, 6:14:45 AM12/2/24
to MaCVi Support
Regarding the CSV file to be submitted for the object detection task,  your explanation and the provided reference file are inconsistent. We are confused about the format of the CSV file to  be submitted.
{23A62DC9-EB13-4830-8842-0EDD03CB0F99}.png

Nikhil Akalwadi

unread,
Dec 2, 2024, 6:32:52 AM12/2/24
to yp lin, MaCVi Support
Hi, 

Use the YOLO object detection code and instructions given in the challenge webpage. 

Upon running detect.py in the detection code, 
You will be left with a predictions.csv file. 
In the csv file: 
1. The ‘c’ column in the screenshot you have shared, represents the prediction confidence. 
If you observe the prediction bounding box data column there are 5 values. 
First four values represent class label, x center, y center, height, width (yolo format data) 
The fifth value represents the same confidence score as in column C which is redundant and not required. 
Hence we require you to omit the fifth value and retain all other data as it is during submission. 

Hope this helps. 
Feel free to ask if you have any more questions. 

On Dec 2, 2024, at 16:44, yp lin <ilyf...@gmail.com> wrote:

Regarding the CSV file to be submitted for the object detection task,  your explanation and the provided reference file are inconsistent. We are confused about the format of the CSV file to  be submitted.

Weitere Optionen finden Sie unter https://groups.google.com/d/optout.
<{23A62DC9-EB13-4830-8842-0EDD03CB0F99}.png>

yp lin

unread,
Dec 2, 2024, 8:36:11 PM12/2/24
to MaCVi Support
Hello, the test dataset you provided contains 3200 images. Due to the large number of images and their high resolution, the file we need to submit after processing is approximately 1.6GB. However, the upload limit seems to be 500MB. How can we resolve this issue?  
{333B213C-D4BC-4955-AC6A-79B7B1B8CE1B}.png

Nikhil Akalwadi

unread,
Dec 2, 2024, 9:14:12 PM12/2/24
to yp lin, MaCVi Support
A newer test dataset has been released containing fewer images. You can use the same for testing purposes
Sent from my iPhone

On Dec 3, 2024, at 10:36, yp lin <ilyf...@gmail.com> wrote:


Hello, the test dataset you provided contains 3200 images. Due to the large number of images and their high resolution, the file we need to submit after processing is approximately 1.6GB. However, the upload limit seems to be 500MB. How can we resolve this issue?  

Weitere Optionen finden Sie unter https://groups.google.com/d/optout.
<{333B213C-D4BC-4955-AC6A-79B7B1B8CE1B}.png>

yp lin

unread,
Dec 2, 2024, 9:22:11 PM12/2/24
to MaCVi Support
  Hello, may I know where I can find this dataset?  

Nikhil Akalwadi

unread,
Dec 2, 2024, 9:25:12 PM12/2/24
to yp lin, MaCVi Support
Hi, 
You can find access to the dataset in the dataset drive provided in the challenge page.


On Dec 3, 2024, at 11:22, yp lin <ilyf...@gmail.com> wrote:

  Hello, may I know where I can find this dataset?  

yp lin

unread,
Dec 2, 2024, 9:31:04 PM12/2/24
to MaCVi Support
the dataset drive only contain two previous datasets
{5DB6F825-61C4-4333-BE3B-B87AE009B866}.png

Nikhil Akalwadi

unread,
Dec 3, 2024, 2:52:12 AM12/3/24
to yp lin, MaCVi Support
Dear Participant, 
Please check the drive now. The drive is been updated with the new dataset. 



On 3 Dec 2024, at 11:31, yp lin <ilyf...@gmail.com> wrote:

the dataset drive only contain two previous datasets

Weitere Optionen finden Sie unter https://groups.google.com/d/optout.
<{5DB6F825-61C4-4333-BE3B-B87AE009B866}.png>

Regards,


--
Nikhil Neelkanth Akalwadi
Researcher,
MVRC Team,

yp lin

unread,
Dec 3, 2024, 6:04:50 AM12/3/24
to MaCVi Support
  After submitting, I found that this submission is incorrect. Could you please tell me specifically where it differs from the requirements, and how I should modify it?  
{A7C4E00F-3D2E-4B3C-AD6C-0926308D9E74}.png

Nikhil Akalwadi

unread,
Dec 3, 2024, 6:08:35 AM12/3/24
to yp lin, MaCVi Support
In the D column, the class label is missing. 
Please refer YOLO format for column D in your csv. That should solve your issue. 



On Dec 3, 2024, at 20:04, yp lin <ilyf...@gmail.com> wrote:


  After submitting, I found that this submission is incorrect. Could you please tell me specifically where it differs from the requirements, and how I should modify it?  

Weitere Optionen finden Sie unter https://groups.google.com/d/optout.
<{A7C4E00F-3D2E-4B3C-AD6C-0926308D9E74}.png>

yp lin

unread,
Dec 3, 2024, 9:01:52 PM12/3/24
to MaCVi Support
  Hello, I submitted the sample submission you provided, and it was successfully submitted. However, the image names in the CSV file from the sample do not match the actual images, and the result format is "label x_center y_center height width score." When I replaced my submission CSV with the sample CSV (keeping the restored images I processed, as the image names and size are the same as in the sample), the submission failed. I am not sure if this is due to the fact that I resized the images before restoring and restore them to original size after process. If you can access the submission details, could you please help me check the error reason for submission #13945?  

Nikhil Akalwadi

unread,
Dec 4, 2024, 10:56:27 AM12/4/24
to yp lin, MaCVi Support
Hi, 

There was a potential error in the earlier provided submission. However it has been updated. Please refer to the updated zip file. Also, please note. The predictions.csv may not contain OD for all the 71 images provided in the test set as some images may be failure cases of restoration models and as such the OD models may not be able to detect the object present in the image. For such images the predictions will be considered zero during evaluation. 

Thankyou.


Weitere Optionen finden Sie unter https://groups.google.com/d/optout.
Regards,
--
Nikhil Neelkanth Akalwadi

yp lin

unread,
Dec 5, 2024, 1:32:05 AM12/5/24
to MaCVi Support
Hello, I sincerely apologize for bothering you multiple times.  My submission still has issues.  Submission #13969 used our detection results and your images, and it was successful.  However, submission #13970 used your detection results and our repaired images, but it failed.  I suspect the reason might be an error in the metric calculation for the images restoration (possibly due to a division by zero?).  I can guarantee that our repaired images are the same size as the original ones.  Since the images are too large : we resized them to 3000x3000, obtained the output results, and then resized them back to the original size.  Could you please help me check the error message for submission #13970?
453a8f29-2505-415b-a86e-7043a28f9408.png

Nikhil Akalwadi

unread,
Dec 5, 2024, 2:57:59 AM12/5/24
to yp lin, MaCVi Support
Hi, 
I will surely check your issue with the above said submission IDs and get back to you. 
Please feel free to ask questions. 


On Dec 5, 2024, at 15:32, yp lin <ilyf...@gmail.com> wrote:


Hello, I sincerely apologize for bothering you multiple times.  My submission still has issues.  Submission #13969 used our detection results and your images, and it was successful.  However, submission #13970 used your detection results and our repaired images, but it failed.  I suspect the reason might be an error in the metric calculation for the images restoration (possibly due to a division by zero?).  I can guarantee that our repaired images are the same size as the original ones.  Since the images are too large : we resized them to 3000x3000, obtained the output results, and then resized them back to the original size.  Could you please help me check the error message for submission #13970?

Weitere Optionen finden Sie unter https://groups.google.com/d/optout.
<453a8f29-2505-415b-a86e-7043a28f9408.png>

yp lin

unread,
Dec 5, 2024, 9:01:05 AM12/5/24
to MaCVi Support
Thank you for your patient response. I realized that I forgot to include the CSV file in #13970, but this shouldn't be the main cause of the issue. Because in #13977, a complete file was also submitted failed.

Nikhil Akalwadi

unread,
Dec 5, 2024, 10:52:01 AM12/5/24
to yp lin, MaCVi Support
Hi, 
I just verified your recent submissions, and submission ID#13977 seems to have executed just fine and metrics were logged. Not sure why you were facing error. But I also tried submitting the same CSV file and the images and seems to have worked just fine. Please let me know if you are facing any other issues. 




Weitere Optionen finden Sie unter https://groups.google.com/d/optout.

yp lin

unread,
Dec 5, 2024, 11:07:02 PM12/5/24
to MaCVi Support
Hi, my submission still has issues. I'm not sure why the same files you submitted are successful. If possible, could you please try submitting #13982 again? Additionally, I have found some potential issues. When testing the UIQM evaluation metrics locally, an error occurs. For some images, division by zero happens during the calculation, resulting in NaN values. However, these images look fine in terms of visual restoration. By the way, the original input images don't cause this issue when calculating UIQM. My UIQM code is from the internet and may have some problems. Could you provide your metrics calculation code, or check for any issues on your end?
image-20241206100056062.png

Nikhil Akalwadi

unread,
Dec 6, 2024, 2:51:03 PM12/6/24
to yp lin, MaCVi Support
Hello, 
There seems to be some issue with the UIQM metric. 
However it will be resolved in a few hours. Until then it will be displayed zero in-case of any errors, or the values otherwise. 

Thankyou for bringing this to our notice. 


On 6 Dec 2024, at 13:07, yp lin <ilyf...@gmail.com> wrote:

Hi, my submission still has issues. I'm not sure why the same files you submitted are successful. If possible, could you please try submitting #13982 again? Additionally, I have found some potential issues. When testing the UIQM evaluation metrics locally, an error occurs. For some images, division by zero happens during the calculation, resulting in NaN values. However, these images look fine in terms of visual restoration. By the way, the original input images don't cause this issue when calculating UIQM. My UIQM code is from the internet and may have some problems. Could you provide your metrics calculation code, or check for any issues on your end?

Weitere Optionen finden Sie unter https://groups.google.com/d/optout.
<image-20241206100056062.png>

林子嵎

unread,
Dec 11, 2024, 7:15:10 AM12/11/24
to MaCVi Support
What about the column 'B' in example `predictions.csv`? There seems no provided reference between the class name and the class label in column 'D'.
Screenshot 2024-12-11 201400.png

Nikhil Akalwadi

unread,
Dec 11, 2024, 8:13:16 AM12/11/24
to 林子嵎, MaCVi Support
The column B is for the evaluation on the server end. You may retain it as from the OD predictions. 



On Dec 11, 2024, at 21:15, 林子嵎 <lspss...@gmail.com> wrote:

What about the column 'B' in example `predictions.csv`? There seems no provided reference between the class name and the class label in column 'D'.

Weitere Optionen finden Sie unter https://groups.google.com/d/optout.
<Screenshot 2024-12-11 201400.png>

林子嵎

unread,
Dec 11, 2024, 11:50:59 PM12/11/24
to MaCVi Support
Is there a correct answer for that? According to the example `prediction.csv`:
{class 0: Fish}
{class 4: Ray}
What about class 1, 2, 3, 5, 6?

Nikhil Akalwadi

unread,
Dec 12, 2024, 2:34:23 AM12/12/24
to 林子嵎, MaCVi Support
Since this is for testing, and our primary goal is on restoration, we have limited the object detection classes, and have given images wit complex lighting conditions for restoration and then the further tasks. 



On Dec 12, 2024, at 13:51, 林子嵎 <lspss...@gmail.com> wrote:

Is there a correct answer for that? According to the example `prediction.csv`:

林子嵎

unread,
Dec 12, 2024, 9:43:13 AM12/12/24
to MaCVi Support
So basically, we can just print out whatever we predict? it just doesn't matter if they have the correct class index.
We have: 0=fish, 1=shark, 2=octopus, 3=starfish, 4=stingray, 5=turtle, 6=jellyfish.

Nikhil Akalwadi

unread,
Dec 12, 2024, 3:49:19 PM12/12/24
to 林子嵎, MaCVi Support
No, as I said, please keep them as it is from the model predictions as they will be used during the evaluation. You may not modify it.

On Dec 12, 2024, at 23:43, 林子嵎 <lspss...@gmail.com> wrote:

So basically, we can just print out whatever we predict? it just doesn't matter if they have the correct class index.

yp lin

unread,
Dec 12, 2024, 9:46:06 PM12/12/24
to MaCVi Support
May I ask how to handle the UIQM metric in the end? The value I submitted still shows as 0.  

yp lin

unread,
Dec 18, 2024, 3:26:55 AM12/18/24
to MaCVi Support
I found the issue with UIQM and the potential solution. In the UIQM, when calculating uiconm = logamee(gray), the function bottom = plipsum(blockmax, blockmin) encounters a problem when blockmax = blockmin = 0, meaning the entire block is black. In this case, bottom = i + j - i * j / gamma becomes 0, leading to a division by zero error. I believe a special case can be added here: since the numerator plipsub = k * (i - j) / (k - j) should be 0, we can set the value of this entire block to 0.  
在2024年12月13日星期五 UTC+8 04:49:19<Nikhil Akalwadi> 写道：

Nikhil Akalwadi

unread,
Dec 18, 2024, 3:37:51 AM12/18/24
to yp lin, MaCVi Support
Hi, 

We have already identified and resolved the issue you mentioned with the patch you suggested.


However, we would greatly appreciate it if you could provide us with your version of the UIQM evaluation code. We can verify its functionality and perform additional necessary fixes to the server’s scripts. 




Weitere Optionen finden Sie unter https://groups.google.com/d/optout.

yp lin

unread,
Dec 18, 2024, 7:41:57 AM12/18/24
to MaCVi Support
I only changed this part. You can get it from gdrive. Hope it helps.【https://drive.google.com/file/d/16zkYYhfYrKc_55UPTl5L5NwyQB3KzMZB/view?usp=drive_link】
453a8f29-2505-415b-a86e-7043a28f9408.png

Nikhil Akalwadi

unread,
Dec 18, 2024, 9:56:45 AM12/18/24
to yp lin, MaCVi Support
Hi, 

Thank you for sharing.


It appears that we have encountered similar situations in the past. Nevertheless, we have implemented an additional update to our evaluation script. 


Should you require further assistance, please do not hesitate to contact us. 



On 18 Dec 2024, at 18:11, yp lin <ilyf...@gmail.com> wrote:

I only changed this part. You can get it from gdrive. Hope it helps.【https://drive.google.com/file/d/16zkYYhfYrKc_55UPTl5L5NwyQB3KzMZB/view?usp=drive_link】

Weitere Optionen finden Sie unter https://groups.google.com/d/optout.
<453a8f29-2505-415b-a86e-7043a28f9408.png>
Reply all
Reply to author
Forward
0 new messages