Judging criteria for SCC Overall Winner

[Brown, W. Michael]

The overall winner for the student cluster competition will be determined based on the highest aggregate score for HPCC, Application, and Interview components. In addition to the overall Interview conducted by experts in HPC that are not part of the SCC committee, there is an additional interview score included within the Application component as noted below that is decided by the applications judges. Points for visualization contribute to the interview scores.



HPCC


The HPCC results will be judged with the following priority ranking:
(1) MPI-RandomAccess
(2) MPI-FFT
(3) HPL
(4) PTRANS
(5) EP-RandomAccess (aka "Star RandomAccess")
(6) EP-FFT (aka "Star-FFT")
(7) EP-DGEMM (aka "Star DGEMM")
(8) EP-STREAM (aka "Star STREAM")



HPCC counts for 10% of your aggregate score.



Applications


Your application throughput score accounts for 80% of your aggregate score.



Each application is worth 25 points. 20 points will go towards their throughput and correctness score as determined by each application judge and 5 points towards your application interview score.



Each application has a README or instructions to define the problem and data sets. The point distribution for the application will be described in the data set instructions. The application judges will come Tuesday morning to determine how much your team understands about the application.


Interview


The interview counts for 10% of your aggregate score. You will be judged on your teams general application knowledge, architecture knowledge, team organization/division of tasks, display, and overall presentation.

[kimoo huang]

Can I use my own HPL(maybe in cuda) instead of the HPL in HPCC?

2012/10/16 Brown, W. Michael <bro...@ornl.gov>

--
You received this message because you are subscribed to the Google Groups "SCC12" group.
To post to this group, send email to sc...@googlegroups.com.
To unsubscribe from this group, send email to scc12+un...@googlegroups.com.
Visit this group at http://groups.google.com/group/scc12?hl=en.

--
Jianmeng,

Best Regards

[dust...@gmail.com]

You may use a accelerated version of HPL (Cuda in your case), but the result of that run will not be counted in the score for the HPCC portion of the challenge. The number from the custom run will be used to rank your system to see which has the highest FLOPS, which gets announced when the competition ends. A full unmodified run of HPCC is what will be used in scoring.

[Andrew Wiley]

Isn't this a significant departure from previous years?
Restricting us to unmodified HPCC for scoring will make the score fail to represent the actual performance of our cluster for most if not all of the teams in the standard track.

> Date: Wed, 17 Oct 2012 12:26:53 -0700
> From: dust...@gmail.com
> To: sc...@googlegroups.com
> CC: bro...@ornl.gov
> Subject: SCC12 Re: Judging criteria for SCC Overall Winner

>
> You may use a accelerated version of HPL (Cuda in your case), but the result of that run will not be counted in the score for the HPCC portion of the challenge. The number from the custom run will be used to rank your system to see which has the highest FLOPS, which gets announced when the competition ends. A full unmodified run of HPCC is what will be used in scoring.
>

[Doug Smith]

Allow me to clarify,

A full complete HPCC run (with its included HPL) will be scored as one component of scoring for the overall winner. Any isolated runs of HPL will be used in scoring the highest Linpack award but will not be used in scoring for overall winner.

All scored runs must be done under the power budget and of course what ever configuration you successfully run for HPCC is what your stuck with for the remainder of the competition.

Doug Smith
Co-Chair SCC12

[Doug Smith]

This is not a deviation from previous years.

Doug

On 10/17/2012 3:17 PM, Andrew Wiley wrote:

[Andrew Wiley]

Ah, my apologies, I wasn't remembering that highest Linpack is a separate award.
 
My bad.
 

---

Date: Wed, 17 Oct 2012 18:42:18 -0600
From: Doug....@lasp.colorado.edu
To: sc...@googlegroups.com
Subject: Re: SCC12 Re: Judging criteria for SCC Overall Winner

[chenzh...@gmail.com]

Hi Doug,
I have no idea whether we could modify the source code of HPCC.I rememberd that it was be allowed last year.For examples, I have modified PTRANS or HPL, and could I replace the original one with mine?
By the way, could you tell me the detailed timetable of SCC12? I only know that the SC12 will be held from Nov.10 to Nov.16.
Thanks for your help!

Chen

在 2012年10月18日星期四UTC+8上午8时42分33秒，Doug Smith写道：

> This is not a deviation from previous years.
>
>
>
> Doug

>of

[jenny...@gmail.com]

Hi,

Could we just run the part of HPCC that is judged?
For example, the "Communication bandwidth and latency" doesn't include in the judgement. Could we annotate that part?
Thanks

Wang

chenzh...@gmail.com於 2012年10月24日星期三UTC+8下午7時26分56秒寫道：