persistent objects/collections on the roadmap?

[Michael Landis]

Is persistence for collections and user defined types on the Julia roadmap?  If not, should it be?

[Stefan Karpinski]

Can you clarify what you mean by that?

[Scott Jones]

That seems like something better handling in packages, but maybe with some standard "protocol" (if Jeff's protocol ideas get in).

It's actually pretty easy to add persistence, with simple syntax to Julia, even without resorting to metaprogramming

(the flexibility of Julia never ceases to amaze me - kudos again

to all of the creators of Julia)

For my work, I added support for serializing/deserializing data and storing/fetching from a key/value store (we use Aerospike, so that it is distributed), i.e.:

myVar = DBSet(mydb, "Scott")

myVar["keyval"] = somejuliaobject

...

juliaobject = myVar["keyval"]

[Mike Innes]

If you mean persistent data structures, there's FunctionalCollections.jl.

[Scott Jones]

On Friday, January 29, 2016 at 7:17:46 PM UTC-5, Mike Innes wrote:

If you mean persistent data structures, there's FunctionalCollections.jl.

OK - that's not what I was talking about (although that sort of persistent data structures for functional programming are very interesting!).

I was talking about persistence in the sense of databases, files, something like HDF5, etc.

The think I miss the most from my former programming environment was the ease of using multidimensional associative arrays, which could be

private or shared, local or distributed, in-memory optimized (possibly using virtual memory) or shared cache w disk/network storage.

After attending the North East Database Day event at CSAIL@MIT yesterday, I'm convinced that Julia would be a great fit for a lot of the work going on

in the database world recently, a lot seems to be focused on linear algebra, statistics, data science use (for example, there was hybrid linear algebra / relational database).

I'm not sure which Michael Landis meant though.

[Michael Landis]

In the sense of serializing and deserializing in memory objects and data structures, so you can checkpoint/restart a (hours or days long) complex computation without restarting at the beginning.

[Stefan Karpinski]

https://github.com/JuliaLang/JLD.jl

[Scott Jones]

Yes, I actually wrote a simple wrapper over what I described above, so that one could register a set of top-level variables in a Dict as your "working set", and then save all of those variables with a simple call, and then restore them later (on the same, or another process).

I had to do so, because the guy I'm consulting for, when playing around with large word corpuses he was analyzing, would accidentally display one to the screen, hit ^C, and Julia usually ends up exiting the the shell (really need a fix for that!).  Now he can checkpoint frequently after long computations, get back working much quicker if julia dies.

[Jameson Nash]

> hit ^C, and Julia usually ends up exiting the the shell (really need a fix for that!)

is this on windows / mintty?

[Scott Jones]

On Sunday, January 31, 2016 at 12:55:06 AM UTC-5, Jameson wrote:

> hit ^C, and Julia usually ends up exiting the the shell (really need a fix for that!) 

is this on windows / mintty?

No, on Mac OS X and Linux (64-bit, running on a Ubuntu 14.04 VM).  We don't use Windows! 😀

[Jameson Nash]

In that case, it should be covered by the `jl_exit_on_sigint` ccall. I haven't seen that fail, so I don't think anyone is tracking any bugs in that.