algorithm visualization example using D3: Newton-Raphson optimization (2D)

[Daniel Korenblum]

Hi all,

Per request, I created a folllow-up visualization for Newton's method in two-dimensions using D3.js. This time, the visualization shows how a single Newton step is determined by a 2-D quadratic approximation to an objective function. The sliders control the eigenvalues of the 2-by-2 Hessian matrix and shows the corresponding quadratic function using contour plots. Green arrows show the Newton step and gray arrows indicate the steepest descent step.

bl.ocks.org/dannyko/0956c361a6ce22362867/

For a more complete description, please see this post on LinkedIn:

https://www.linkedin.com/pulse/article/20140906044926-58744538-visualizing-newton-s-method-for-optimization-ii

[Sergiy Nesterko]

Good work!

In case you are interested in stochastic optimization, I made two visualizations in that domain:

1. Hybrid Monte Carlo: http://databits.io/bits/hamiltonian-hybrid-monte-carlo

2. Parallel Tempering + Hybrid Monte Carlo: http://databits.io/bits/parallel-tempering-algorithm

Cheers,

Sergiy

[Daniel Korenblum]

Thanks for checking it out! 

Yes, I have used stochastic optimization algorithms before e.g. Metropolis-Hastings for sampling, and "packet annealing", a much more robust version of simulated annealing for global optimization of multiscale potential energy functions describing protein structures for folding problems. But, I haven't used hybrid Monte Carlo yet; maybe I had heard of it before, but if so, I had forgotten :)

Appreciate the link; very interesting. I like the idea of using Langevin dynamics for generating moves to generate samples from interesting probability distributions, to reduce the correlation between consecutive samples. The marginal seem to converge reasonably quickly, at least for this "toy problem" in 2D.

The parallel tempering is also cool. I'm familiar with replica exchange in the context of molecular dynamics (REMD) for molecular simulations, but I haven't in presented in a purely statistical setting before. Nicely done! It's good to see ideas from statistical physics being used in more general statistical problems. One of the recurring themes in my research! Perhaps a toast to Jaynes is in order :)

I'm also interested in stochastic dynamics. It might be kool to make a visualization showing how Fokker-Planck and Smoluchowski equations are derived from distributions of samples from solutions to stochastic differential equations. I haven't heard of jStat or Sylvester before; but I will probably use them next time and they should expand the scope of what I can create. Again, much thanks.

Are you still at Harvard? I'm in the SF bay area, if you're ever in town give me a shout and we can brainstorm some more algorithms and visualizations :)

--
You received this message because you are subscribed to a topic in the Google Groups "d3-js" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/d3-js/0S8exkwpEtA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to d3-js+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Daniel Korenblum]

It looks like there's another newer option for doing linear algebra and statistics in JS too, called science.js -

https://github.com/jasondavies/science.js

[Sergiy Nesterko]

Hey Daniel,

All good stuff. It seems that algorithm visualization has become fashionable these days with Mike Bostock joining the movement, bringing some of the computer science algorithms to life. I wonder if one could create an algorithm visualization playbook (e.g. algorithms from CS, statistics, systems biology etc)...

No longer with Harvard, but still based out of Boston. If you are ever on the East coast, drop me a line -- happy to brainstorm ideas.