[NeurIPS2023 Workshops] Call for papers: Workshop on "Heavy Tails in ML: Structure, Stability, Dynamics" @NeurIPS

[Umut Simsekli]

Dear colleagues,

We are delighted to announce the first workshop on "Heavy Tails in Machine Learning: Structure, Stability, Dynamics”, which will be a part of NeurIPS 2023 in New Orleans! 

We are now accepting submissions, and the details are as follows.

Submission deadline: September 29th, 2023 (Anywhere on Earth)

Notification of acceptance:  October 20th, 2023 (Anywhere on Earth)

Workshop date: December 15th, 2023 

Workshop website: https://sites.google.com/view/heavy-tails-ml-2023/home

Description:

Heavy-tailed distributions likely produce observations that can be very large in magnitude and far from the mean; hence, they are often used for modeling phenomena that exhibit outliers. As a consequence, the machine learning and statistics communities usually associate heavy-tailed behaviors with rather negative consequences, such as creating outliers or numerical instability. 

Despite their ‘daunting’ connotation, heavy tails are ubiquitous in virtually any domain: many natural systems have been indeed identified as heavy-tailed, and it has been shown that their heavy-tailed behavior is the main feature that determines their characteristics. 

In the context of machine learning, recent studies have shown that appearance of heavy tails is closely related to stability, geometry, topology and dynamics of the learning process, and heavy tails naturally emerge in ML training in various ways. Furthermore, contrary to their perceived image, heavy tails can, in fact, be beneficial for the performance of an ML algorithm.  

The ultimate goal of this workshop is to foster research and exchange of ideas at the intersection of applied probability, theory of dynamical systems, optimization, and theoretical machine learning to make progress on practical problems where heavy tails, stability, or topological properties of optimization algorithms play an important role, e.g., in understanding learning dynamics. 

In our community, the emergence of heavy tails (and the edge of stability) is often perceived as a ‘phenomenon,’ which essentially implies that they are rather ‘surprising’ or even ‘counterintuitive.’ We aim to break this perception and establish that such behaviors are indeed expected, and the theory and methodology should be re-positioned accordingly. 

Topics:

We will accept submissions on ongoing research on (including but not limited to) the following subjects: 

• Heavy tails in stochastic optimization
• Heavy tails and generalization
• Edge of stability
• Empirical scaling laws in large models
• Heavy-tailed auto-correlation
• Iterated function systems 
• Heavy-tailed continuous dynamical systems
• Power-laws in ML

Submission instructions:

We will only accept unpublished submissions on ongoing research. We welcome unfinished work as long as it is relevant and contains original ideas. 

 

Papers should be submitted anonymously to the OpenReview submission system. All the papers will undergo a double-blind peer review process. Papers will be presented as posters at the workshop, with some papers being selected as talks. The workshop will not have official proceedings. The submission should be self-contained, as the referees will not be required to read the supplementary material. Dual submissions to other NeurIPS workshops are allowed. Extended abstracts of work under review at another journal/conference can be submitted if the journal/conference in question permits this.

Submission format:  NeurIPS style, no more than six pages of main content (not including references and supplementary material). Single pdf. 

Submission website: https://openreview.net/group?id=NeurIPS.cc/2023/Workshop/HeavyTails

Contact email:  heavytail...@googlegroups.com

Invited speakers:

Adam Wierman  -  Caltech

Nisha Chandramoorthy  -  Georgia Tech

Charles H. Martin  -  Calculation Consulting

Liam Hodgkinson  -  University of Melbourne

Organizers:

Mert Gürbüzbalaban -    Rutgers

Stefanie Jegelka -   MIT

Michael Mahoney -   Berkeley

Umut Şimşekli -   Inria Paris / ENS Paris