Planning future developments

[Dieter Weber]

Dear all,

For future development we have a number of ideas, see list below. Please
let us know which ones sound appealing to you, additional ideas, your
comments and if/how you'd like to contribute to the development. We'd
like to organize a plenary call about this. If you like to participate,
please indicate your availability in this Doodle poll:
https://doodle.com/poll/fav73k2rmcmkafgz

1. Make the computational back-end agnostic to the shape of the data,
i.e. support n-dimensional scans and m-dimensional detector data.
Examples: spectral data with m=1, line scans with n=1, vector scans,
non-rectangular partitioning like in MIB data sets, etc.

2. Build an application-level data transformation and caching layer.
This allows using local SSDs in a cluster for data locality from any
data source, not only Hadoop FS, and helps to efficiently serve data in
the right layout to more advanced algorithms.

3. High-performance free-form region of interest support in the scan
dimension with vectored IO, including random sampling,

4. Generalized high-performance map-reduce functionality. Currently, we
"map" multiplication and "reduce" with sum. In the future this can be
possible with any function that is free of side effects and associative.

5. User-defined functions that work on larger sub-sets of the data, for
example partitions, including functions that need whole frames (Fourier
transform) or local windowing with neighboring scan pixels. Enabled by
item 2.

6. Support for physical units rather than pixels. This includes picking
up the work on NeXus for pixelated STEM.

7. Include more methods from various sources. Which algorithm would you
like to have included in LiberTEM? Enabled by items 1 and 2 so that data
can be fed to your algorithm in many different ways.

8. Integration of simulation packages like Prismatic or Dr. Probe --
file format support, and possibly parallel execution on a cluster
managed by LiberTEM.

9. LiberTEM user and developer workshop

10. What other ideas do you have?

I'm looking forward to your feedback and to a lively discussion!

With best regards,
Dieter

--
Dr. Dieter WEBER

Peter Grünberg Institute, Microstructure Research (PGI-5)
Ernst Ruska-Centre for Microscopy and Spectroscopy with Electrons (ER-C)
Forschungszentrum Jülich
52425 Jülich, Germany

Email: d.w...@fz-juelich.de
Phone: +49 2461 61 85118


------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------