On persistence

[Bernat Romagosa]

Hi list,

I've been eaten out by my job for a loooong while, but now I got a reduction of my work hours, so I'm back and I've got some new side projects.

One of them is redoing a game I did a long time ago in Squeak to teach children how to read music scores. I wrote it for a school I used to work in and now they asked me to make it beautiful and add some new features.

I'll use Amber for the frontend, and since I'm not aware of the new stuff that went on during all this time in the Amber world, I wanted to ask if there is something new regarding persistence.

On that regard, I did a search on github, amber-lang.net, world.st and google, and only came across some threads talking about possibilities and tests people are trying out. I think it'd be swell if we could write a chapter on the documentation site introducing what persistence options are there, and maybe add some simple examples too.

So, what's the persistence trend in Amber these days? Websockets + Pharo? Any bindings to some NoSQL database? Some nodeJs solution?

Cheers,

--
Bernat Romagosa.

[Apostolis Xekoukoulotakis]

In my project, I will use nodejs and zeromq to talk to any database I want. You might have to write some code in another language though. 

But nodejs will probably have bindings to common databases, I havent checked. Others might know more about these.

2012/11/20 Bernat Romagosa <tibabenfor...@gmail.com>

--

Sincerely yours, 

     Apostolis Xekoukoulotakis

[sebastian]

I know what you feel. I've wrote this:

http://sebastianconcept.com/brandIt/how-to-deliver-realtime-web-and-mobile-applications

because I'm interested in Trello-kind-of-apps/stacks.

WebSockets is okay but it's better to think of it as a transportation layer and outsource all that to Socket.IO (that will magically fallback to other mechanisms when websockets aren't available)

Yes, in the server side having amber apps running on node js would rock. Why? because you can use whatever nodejs stuff you want. for example MongoDb for persistance and Redis for massive broadcasting of events.

But we are not there yet (meaning we can't debug properly at this point). We first need to finish developing Helios 1.0 including a full smalltalk debugger.

Helios will be capable to connect to remote ambers including remotely debugging amber apps (imagine how could would be to do that for mobile apps).

It's all work in progress though

And.... 

please write to Nicolas offering help if you want to help us pushing things in that direction

sebastian

o/

PD: if you need start prototyping your app this week you'll be better by having pharo + zinc + mongoDb client on the server side

[Bernat Romagosa]

+1 for your PD, I'll give it a go :)

What's helios? As I said, I've been away for quite a while...

2012/11/20 sebastian <sebastia...@gmail.com>

--
Bernat Romagosa.

[sebastian]

Helios is Amber's next IDE

You can take a look at it.

It's a branch in github and/or is in the latest master commit

1. go to your git/amber dir and clone or pull your local master from nicolas

2. execute:  ./bin/server

3. open a chrome in localhost:4000/helios.html

have fun

sebastian

o/

[Nicolas Petton]

Bernat, if you want to help with Helios we could do a skype session so I
can give you starting points and infos on how we plan to do stuff.

Everyone else interesting in that or (or any other part of Amber,
there are no many cool things to be done) is welcome!

Cheers,
Nico

--

--
Nicolas Petton
http://nicolas-petton.fr

[Bernat Romagosa]

Sure thing, next week I'll be free pretty much all afternoons, I'll let you know.

About MongoTalk, how do you guys work? Whenever I have to persist stuff I find myself in the middle of the same dilema... should I write a conversor from my objects into whateverDB understands and back? Or should I just give up and store/retrieve data from de DB directly?

I feel the first option is an obvious pain, but the second option means you're not working with real objects anymore...

Cheers,

Bernat.

2012/11/20 Nicolas Petton <petton....@gmail.com>

--
Bernat Romagosa.

[Nicolas Petton]

I'll write a more detailed email on MongoTalk and Voyage on the Pharo
mailing-list. Colleagues at the lab asked me to write slides, etc.

If you don't want to wait, you can ask me (or Esteban, in cc) privately.

Cheers,
Nico

[Brad Midgley]

Sebastian,

I know what you feel. I've wrote this:

http://sebastianconcept.com/brandIt/how-to-deliver-realtime-web-and-mobile-applications

Realtime mobile implies that poor network connection/latency/reliability is not felt by the user. That's one essential aspect of mobile computing, that you account for a terrible connection.

To do this in a js framework will require local storage persistence that can synchronize with remote objects. It's a tough problem... one interesting chunk is disconnected persistence in local storage.

There is a long topic on storing js objects at http://stackoverflow.com/questions/2010892/storing-objects-in-html5-localstorage but one thing that is skipped over is how to work with references between objects. Does Amber provide anything that could help with persisting objects as well as references between them?

For me the immediate problem is storing a tree of objects, so the whole tree *could* be turned into one big string that I save in one slot in local storage. That's not very satisfying though and wouldn't be generally useful.

Thoughts?

Brad

[Nicolas Petton]

Hi,

Amber doesn't provide anything like that (yet). We could look into
existing Smalltalk solutions though, or implement a #storeOn: protocol.

Nico

-

[sebastian]

Yes, we should store in local first (and try to stay in sync every time internet is online)

I've made something simple and mapless to deal with that in a very friendly way

I should publish :/

problem is that I'm currently saturated with work on the seaside pharo type of work (not Amber type of work)

sebastian

o/

[Brad Midgley]

I accidentally replied off-list.

So, #storeOn would have worked for my data structure without cycles (my design was a tree) but I've changed the design and now it does have cycles.

It sounds like #storeBinaryOn would be a better fit. There isn't a standard here for the implementation, but it needs to observe object identity so it can stop its deep traverse when it detects an object that it's already traversed. It sounds like one local storage value can use up the whole limit for the store, so that gives us megabytes to work with.

--
Brad Midgley

[Davorin Ruševljan]

On Tuesday, November 20, 2012 8:17:17 PM UTC+1, Bernat Romagosa wrote:

Hi list,

So, what's the persistence trend in Amber these days? Websockets + Pharo? Any bindings to some NoSQL database? Some nodeJs solution?

If I get enough time, I would liek to take a stab to try to write CouchApp with Amber (The app that is http delivered from CouchDB, and makes javascript db queries against the same CouchDB database)

Davorin Rusevljan
http://www.cloud208.com/

[Brad Midgley]

Nico, 

the examples of #storeOn take a FileStream. It might be nice to use the same protocol even though it's going to local storage.

Where do you think an implementation of local storage fits best? Should it be related to the Stream/PositionableStream/WriteStream/FileStream hierarchy? Would it actually be the implementation of FileStream since we know we'll never have a filesystem?

Brad

[Nicolas Petton]

Hi,

I'd serialize objects into a standard stream and store them using the local
storage API.

Cheers,
Nico

[Manfred Kröhnert]

Hi,

On Fri, Dec 28, 2012 at 6:14 PM, Nicolas Petton <petton....@gmail.com> wrote:

Hi,

I'd serialize objects into a standard stream and store them using the local
storage API.

I just saw the method Package>>commitToLocalStorage: .

Not sure if it works as expected, but might be a good starting point.

Best,

Manfred 

[Nicolas Petton]

Indeed, I didn't remember about this one :)

[Brad Midgley]

I sent a pull request for a LocalStorageStream that reads from local storage at init and saves out at flush/close. I hope it looks ok.

I see storeOn implemented here and there.

|s|

s := StringStream new setCollection: '' ; setStreamSize: 0; yourself. 

(HashedCollection new at: 'key' put: 'value'; yourself) storeOn: s. 

s contents

generates  '#{'key'->'value'}' but how am I supposed to reanimate this? If I implement Object#readFrom: should it call into the compiler?

--
Brad Midgley

[Nicolas Petton]

To read, you would need a #readFrom: method, or something similar. Just
like #storeOn:, only for the opposite action. Also, you would probably
need to store the class name in the JSON data.

Cheers,
Nico

Brad Midgley <bmid...@gmail.com> writes:

> I sent a pull request for a LocalStorageStream that reads from local
> storage at init and saves out at flush/close. I hope it looks ok.
>
> I see storeOn implemented here and there.
>
> |s|
> s := StringStream new setCollection: '' ; setStreamSize: 0; yourself.
> (HashedCollection new at: 'key' put: 'value'; yourself) storeOn: s.
> s contents
>
> generates '#{'key'->'value'}' but how am I supposed to reanimate this? If
> I implement Object#readFrom: should it call into the compiler?
>
> On Friday, December 28, 2012, Nicolas Petton wrote:
>
>>
>> Indeed, I didn't remember about this one :)
>>

>> Manfred Kröhnert <mkroeh...@googlemail.com <javascript:;>> writes:
>>
>> > Hi,
>> >
>> > On Fri, Dec 28, 2012 at 6:14 PM, Nicolas Petton <

>> petton....@gmail.com <javascript:;>>wrote: