About unofficial submissions

[Guohao Li]

Hi team,

Regarding the unofffical submissions on OGB, I find that a couple of submissions using classic GNN models with simple tricks outperform other newly proposed methods significiantly. That is amazing! I really appreciate the effort of community on reimplmenting those models. 

Some of these tricks are non-trivial and make the models very different from the orignal papers. To better understand how the variants evolve, I suggest the contributors could optionally submit an additional teachnical report to write up these findings.

Best,

Guohao Li

[Weihua Hu]

Hi Guohao,

Thank you for your great suggestion. I agree that non-trivial tricks are often very important and can make a big difference in performance. At the same time, it seems a bit too much to ask people to write a dedicated paper just to describe those tricks. 

To strike the good balance, I'd encourage unofficial submissions to explicitly mention any non-obvious tricks in both the Github README and method name (for instance, one can use the name "GCN w/ batchnorm" to emphasize that batch norm is added in the original GCN).

Best,

Weihua

2020年11月9日月曜日 5:35:57 UTC-8 Guohao Li:

[Weihua Hu]

That being said, please feel free to let us know those submissions that you have noticed. We are happy to help to figure out solutions together with the submitters.

Thanks,

Weihua

2020年11月9日月曜日 20:31:54 UTC-8 Weihua Hu:

[Guohao Li]

Hi Weihua,

Thanks for you reply. I agree with you that it may be too much to ask everyone to write a paper about the tricks. So I suggest it can be an option for people would like to share the details. The idea is that it would be great if the OGB team can encourage contributors to share. For instance, there are many submissions from the DGL team. I personally learn a lot from their codebase and README. They may have a lot of interesting findings to share. So if a teachnical report is allowed and encouraged for unofficial submissions, people may be more willing or active to share their experience.

Best,

Guohao

[Weihua Hu]

Hi Guohao,

Thank you for your great suggestion. That makes sense to us. I have added a few sentences to the leaderboard page to encourage what you have suggested.

Best,

Weihua

2020年11月10日火曜日 13:22:14 UTC-8 Guohao Li: