<- One of the TensorBoard developers
At the moment, there's no first-class support for custom plots, embedding them as image summaries is really clever! Although, as you noted, very indirect and sounds painful to maintain.
I'd like to add support for custom plots to TensorBoard, this is something that people request pretty often. However, getting all the details right is non-trivial; would we expose a way to write custom data into the summaries and then choose from a standard selection of pre-created charts on the frontend to display the new data sources, or would it be better to have a plugin system on the frontend where people can mix in their own visualization components (e.g. built with D3)? And what would the APIs look like on both sides?
At the moment, this isn't very high on my priorities; I'm looking more at performance, increasing maintainability of TensorBoard, adding in some new visualizations like true histograms. That said, if anyone wants to write a proposal for how this system should work, I'd be glad to discuss it and it would move us closer to having custom plots.