Summary Kick-off LiberTEM 2018-01-15

[Dieter Weber]

Dear all,

thank you for making this event a success! I was very happy about all
your contributions and help, and about the lively discussion.

Appended you find the chat protocols from Skype and Gitter. I had a
feeling that our combination of chat and voice call worked very well to
handle the scale of interaction. What do you think?

As main points, this is what I took away:

* A good file format is most important for our community --> We are
already putting more focus on that. More discussion on Github issues
that Alex will create soon about the existing implementations!

* HDF5 is by far the strongest contender and will be "the one" as a
basis unless we hit big obstacles. Colin's contact to the HDF5 main
developer will be very helpful :-) I'm asking myself why CERN started
ROOT, by the way.

* Separate "dev" mailing list to keep the more administrative list
low-frequency and free of technical details: done -->
https://groups.google.com/forum/#!forum/libertem-dev -- you are invited
to join! :-) Gitter and Github will develop organically. Feel free to
take charge and make (or suggest) changes!

* Free GPU at https://developer.nvidia.com/academic_gpu_seeding -- thank
you Colin! :-)

* Separation between live and post processing. Many algorithms will be
too slow for live. Proposal: All live alogrithms are automatically also
available offline by using the same "math kernel" with different
interfaces to move data in and out, but not vice-versa. PCA and drift
correction will need the whole data set, for example.

* A GUI is important. Suggestions: QT --> Noted! :-)

* Programming: C/C++ essential for some parts, not only Python.

* We need to work on recognition for developer work. --> Yes, that is on
the agenda. Creating DOIs, writing technical papers, giving credit to
code contributions, encouraging citing appropriate sources etc. is on
our management TODO list for the project. More to follow and I welcome
input and suggestions!

* Details about acquisition: For now, we focus on an architecture to
absorb massive data rates without bottlenecks (file format, filters
etc). Special provisions necessary because the K2 is already in the
order of DRAM bandwidth. I'd expect that the rates will go up, up, up in
the next years. Fortunately, the application is very parallel for basic
processing. Implementing acquisition for specific detectors follows
later as soon as we have the basics settled.

* Compression important. Start with built-in HDF5 compression, then we
see about custom filters and such. Definitely worthwile at those data
rates and sizes!

Let's keep the discussion going in the mailing lists, Gitter chat and
issue tracker! :-) Many, many thanks again for your help.

With best regards,
Alex and Dieter

--
Dr. Dieter WEBER

Peter Grünberg Institute, Microstructure Research (PGI-5)
Ernst Ruska-Centre for Microscopy and Spectroscopy with Electrons (ER-C)
Forschungszentrum Jülich
52425 Jülich, Germany

Email: d.w...@fz-juelich.de
Phone: +49 2461 61 9264

Email: d.w...@fz-juelich.de
Phone: +49 2461 61 9264


------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------