Fwd: Tuesday, October 11, 2016: Vivek Srikumar, GDC 6.302

[Karl Pichotta]

Hey all,

In case you're not on the FAI list---the following just-scheduled talk this coming Tuesday will likely be of interest to NLLers and FLARErs!

If you want to meet with him and his signup sheet is filled, we can probably add some slots in the morning before the talk, FYI.

_k

---------- Forwarded message ---------
From: Karl Pichotta <pich...@cs.utexas.edu>
Date: Thu, Oct 6, 2016 at 11:49 AM
Subject: Tuesday, October 11, 2016: Vivek Srikumar, GDC 6.302
To: f...@utlists.utexas.edu <f...@utlists.utexas.edu>

Forum for AI invites you to the following talk by Vivek Srikumar this coming Tuesday, October 11, at 11:00 AM.

There is a signup sheet for individual meetings with the speaker at:

https://apps.cs.utexas.edu/calendar/node/266798/edit

[If you have a UTCS username/password, you can use that; if you don't, you can sign in as a guess with username "UTCS" and password "Welcome!"]

Title: A Tale of Two Activations

Speaker: Vivek Srikumar (University of Utah)

Location: GDC 6.302 (Faculty Lounge)

Date: 10/11/2016

Time: 11:00 am

Abstract:

Various factors contribute to the empirical successes of neural networks in recent years: for example, the availability of more data, better understanding of the algorithmic aspects of learning and optimization, and novel network architectures and activation functions. The choice of the activation function is an important design consideration because it changes the expressive capacity of the network. But what functions do the various activation functions represent?

In this talk, I will focus on the representative power of two activation functions -- rectified linear units (ReLUs) and cosine neurons. While ReLUs were originally introduced as a means of easing optimization concerns, they have also led to empirical improvements in predictive accuracy across different tasks. As an explanation for this improvement, I will show that ReLU networks can compactly represent decision surfaces that would require exponentially larger threshold networks. Then, we will move to the less popular cosine neuron which is intimately connected to shift-invariant kernels. I will present a new analysis of the cosine neuron that not only quantifies its expressive capacity, but also naturally leads to regularization techniques that are formally justified. I will end this talk with a set of open research questions about connecting these formal results to empirical
observations.

Speaker Bio:

Vivek Srikumar is an assistant professor in the School of Computing at the University of Utah. Previously, he obtained his Ph.D. from the University of Illinois at Urbana-Champaign and was a post-doc at Stanford University . His research lies in the intersection of machine learning and natural language processing and is largely motivated by problems involving text understanding. In particular, he is interested in research questions related to developing semantic representations of text, learning discrete and real valued representations of textual inputs using little or incidental supervision, and efficiently predicting these representations. His work has been published at various NLP and machine learning venues and recently received the best paper award at EMNLP.