[erlang-questions] UBF(A) vs ETF / UBF(C) vs gen_fsm

[Edmond Begumisa]

Hello all,

I've been looking at Dr. Joe Armstrong's UBF with interest, especially for
IPC between two Erlang nodes over TCP (non-epmd) and possibly between an
Erlang node and a XULRunner client (JavaScript/XPCOM). I have two
questions for those with UBF experience...

1) Concerning UBF(A): Joe mentioned in his original paper that UBF(A)
encoding was more upto 40% more compact than the VM External Term Format.
But that was 8 years ago. I understand the external term format has
improved since then. Is UBF(A) still more compact?

2) Concerning UBF(C): If one is using gen_fsm server-side together with
UBF(A) over the wire, would I be correct in saying that the OTP behaviour
provides the contract checking that UBF(C) would provide? That is, would
using gen_fsm with UBF encoding for messages render the UBF(C) contract
checker redundant?

- Edmond -

--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-questio...@erlang.org

[Joe Armstrong]

On Wed, Sep 22, 2010 at 11:28 AM, Edmond Begumisa
<ebeg...@hysteria-tech.com> wrote:
> Hello all,
>
> I've been looking at Dr. Joe Armstrong's UBF with interest, especially for
> IPC between two Erlang nodes over TCP (non-epmd) and possibly between an
> Erlang node and a XULRunner client (JavaScript/XPCOM). I have two questions
> for those with UBF experience...

That sounds like a very good idea - I'd really like to see clear separation of
interest - Erlang on one side of a communication channel, XULRunner on
the other side
with a clear specified interface between and no feature leakage between the two.

You could try UBF for this - if it turned out that it wasn't good I
have a few other
tricks up my sleeve - how is the javascript/XULrunner side of things?
- the erlang bit
is easy - can you easily setup the underlying socket communication
with XULrunner?

>
> 1) Concerning UBF(A): Joe mentioned in his original paper that UBF(A)
> encoding was more upto 40% more compact than the VM External Term Format.
> But that was 8 years ago. I understand the external term format has improved
> since then. Is UBF(A) still more compact?

I haven't measured but I suspect UBF will be more compact - the UBF
caching optimization
allows you to substitute any repeated term by a single character - so
if you discover that
some term is repeated, you can just replace it by a single character -
this depends upon the
encoder, which is up to you.

>
> 2) Concerning UBF(C): If one is using gen_fsm server-side together with
> UBF(A) over the wire, would I be correct in saying that the OTP behaviour
> provides the contract checking that UBF(C) would provide? That is, would
> using gen_fsm with UBF encoding for messages render the UBF(C) contract
> checker redundant?

No - to do this you would have to compile a UBF(C) state machine to a
gen_fsm state machine
I see no advantage in this - it's essentially the same thing, but the
UBF state checker is
is more specialized than gen_fsm. gen_fsm is a generic machine. The
UBF(C) checker is
specialized for UBF contracts

Another thought - the UBF type system would *easily* map to a JSON
transport format
I blogged this in 2009

http://armstrongonsoftware.blogspot.com/2009/02/json-protocols-part-1.html

It's the same idea as UBF using JSON instead of UBF(A) - the encoding
isn't as pretty
but it makes interoperability easier - I guess *every* language has a
JSON encoder/decoder
so you don't have to write a UBF encoder/decoder.

Having parsed the JSON (or UBF(A) ) the parse trees would be
essentially the same thing
so the contract checker would be easy.

Cheers

/Joe

[Edmond Begumisa]

Thanks very much for your quick response...

On Wed, 22 Sep 2010 20:29:30 +1000, Joe Armstrong <erl...@gmail.com> wrote:

> That sounds like a very good idea - I'd really like to see clear
> separation of
> interest - Erlang on one side of a communication channel, XULRunner on
> the other side
> with a clear specified interface between and no feature leakage between
> the two.

I've had some unpleasant experiences with informal home-grown/private
protocols so the benefits of UBF for my purposes immediately jumped out at
me. It's one of those things you can't appreciate unless you've been there.

> You could try UBF for this - if it turned out that it wasn't good I
> have a few other
> tricks up my sleeve - how is the javascript/XULrunner side of things?
> - the erlang bit
> is easy - can you easily setup the underlying socket communication
> with XULrunner?

Promising but it's still early stages. One problem so far is that
XULRunner is best suited for HTTP communication (raw TCP sockets are
doable but a real pain). I'm currently trying to see if rather than going
against the grain by wresting with Mozilla's lower level socket API's, it
might be easier to have the Erlang side serve UBF over HTTP using say
mochiweb (long-poll/keep-alive issues notwithstanding.) I'll be sure to
post some code when I've got something working.

I'm intrigued by those tricks you mention?!?!

> I haven't measured but I suspect UBF will be more compact - the UBF
> caching optimization
> allows you to substitute any repeated term by a single character - so
> if you discover that
> some term is repeated, you can just replace it by a single character -
> this depends upon the
> encoder, which is up to you.

Perhaps it's time for a UBF EEP! IMO, it would also make writing &
debugging port drivers easier.

> No - to do this you would have to compile a UBF(C) state machine to a
> gen_fsm state machine
> I see no advantage in this - it's essentially the same thing, but the
> UBF state checker is
> is more specialized than gen_fsm. gen_fsm is a generic machine. The
> UBF(C) checker is
> specialized for UBF contracts

I see... I understand now.

> Another thought - the UBF type system would *easily* map to a JSON
> transport format
> I blogged this in 2009
>
> http://armstrongonsoftware.blogspot.com/2009/02/json-protocols-part-1.html
>
> It's the same idea as UBF using JSON instead of UBF(A) - the encoding
> isn't as pretty
> but it makes interoperability easier - I guess *every* language has a
> JSON encoder/decoder
> so you don't have to write a UBF encoder/decoder.
>
> Having parsed the JSON (or UBF(A) ) the parse trees would be
> essentially the same thing
> so the contract checker would be easy.

Ditto! It's actually that blog entry that got me to look at UBF (what ever
happened to part II of that BTW!?!) I've been contemplating using Gemini's
UBF-JSON to make life easier on the XULRunner side. I don't know if you've
had a look at/recommend their implementation...

http://github.com/norton/ubf-jsonrpc

Erlang and XULRunner could be a lethal combination. I'm sure I'm not the
only one who has seen this.

Thanks.

[Scott Lystig Fritchie]

Edmond Begumisa <ebeg...@hysteria-tech.com> wrote:

>> Having parsed the JSON (or UBF(A) ) the parse trees would be
>> essentially the same thing so the contract checker would be easy.

eb> Ditto! It's actually that blog entry that got me to look at UBF
eb> (what ever happened to part II of that BTW!?!) I've been
eb> contemplating using Gemini's UBF-JSON to make life easier on the
eb> XULRunner side. I don't know if you've had a look at/recommend their
eb> implementation...

eb> http://github.com/norton/ubf-jsonrpc

Yes, that's what the ubf-jsonrpc package does. There's also the option
of using the http://github.com/norton/ubf stuff as-is to implement a
"JSF" server, which is straight JSON across a TCP socket (i.e., without
any JSON-RPC HTTP stuff).

IIRC, it's just a server-side configuration option {proto, ubf} or
{proto, jsf} or {proto, ebf} to have an Erlang server speak UBF(A),
"JSF", or UBF-terms-encoded-with-term_to_binary(), respectively.

-Scott

[Edmond Begumisa]

Thanks Scott, I was trying to figure out how to do that... documentation
links I was referred to are a bit scattered at the moment. I'm sure when
they get round to working on them the Gemini UBF tools will spread.

- Edmond -

--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/

________________________________________________________________

[Joe Armstrong]

I've been thinking ...

It would be really great if I could send messages from a browser
to erlang.

I want to add an extra button to firefox that when pressed
analyzes something about the current page and sends a message
to an erlang server.

I was reading about solvent

http://simile.mit.edu/wiki/Solvent

some quotes

" Interactively highlight parts of the page you wish to scrape,
directly in your browser, and obtain the right XPaths for them

Edit and execute the scraper code directly in the browser, making
the development cycle fast and incremental

Save and publish the scraper with the required metadata, so that
others can discover it

..."

This is *exacty* what I want to do - highlight some text in the
brower - as I release the mouse the highlighted text is sent
to and erlang server to be analysed and stored.

IMHO bookmarking a site is not interesting - it's some small fragment
on a page that interest me - I want to:

a) highlight it
b) edit the highlighted bit (ie throw me into an editor)
c) store the edited result

(later I might even be able to "pre-read" a new page finding
what might be interesting for me in the page :-)

This is very generic - if I could isolate "analyze something about the
current page" into a convenient js module then I could do all sort of
fun things.

This seems to require a firefox plugin - any idea how to write this?

If I want to start a collaborative project where do I advertise for
smart javascript programmers who know the firefox internals.

The erlang stuff is easy :-) but the js is tricky - (and I guess this group is
not the best lace to ask js questions?)

(( where is the best group for this (ie js questions)))

[erlang-q...@erlang.org]

Dr Armstrong,

Some comments...

This sounds very interesting. If I've understood correctly, you want the
results delivered to you on the Erlang side as HTML snippets or JSON with
embedded HTML strings?

Also, where you say "throw me into an editor" -- do you mean a editor in
the word-processing sense or an editor in the html authoring sense (like
in Solvent?) The former would be easy (Mozilla have that infrastructure
already), the latter substantially more involved.

> This is very generic - if I could isolate "analyze something about the
> current page" into a convenient js module then I could do all sort of
> fun things.
>

> This seems to require a Firefox plugin - any idea how to write this?

Yes and No. No because I've never actually written a Firefox
extension/add-on before. Yes because writing one is basically a
subset/specialised form of a XULRunner/Mozilla Framework application which
I do have knowledge of.

If you want to have a crack at it yourself (you know the saying "if you
want something done properly..."), you probably want to start with the XUL
tutorial, then the Extension tutorial, then as a reference the official
free e-book "Creating Applications with Mozilla" which goes through the
Mozilla framework...

https://developer.mozilla.org/en/XUL_Tutorial
http://kb.mozillazine.org/Getting_started_with_extension_development
https://developer.mozilla.org/en/Setting_up_extension_development_environment
https://developer.mozilla.org/en/Building_an_Extension
http://books.mozdev.org/html/index.html

You might need the js debugger too...

http://www.mozilla.org/projects/venkman/venkman-walkthrough.html

> If I want to start a collaborative project where do I advertise for
> smart javascript programmers who know the firefox internals.
>

I'm not sure where you'd advertise (Stackoverflow maybe), but I can advise
on what to advertise for...

You might want to specifically advertise for a programmer familiar with
the Mozilla Application Framework -- of which JavaScript is just an
element (XUL/XBL/XPCOM/RDF-XML being the major other bits.) This will weed
out the hobbyists. Another thing to look out for is the fact that the
Mozilla flavor of JavaScript is not quite the same as the typical
JavaScript. That is, the js used in web-pages is "narrower" than the js
used in Firefox extensions/XULRunner apps -- those familiar with the
former (prototype/jquery types) are unlikely to be useful with the latter.
For example, extension writers frequently access Mozilla C++ interfaces
via js and navigate a different DOM (the internals you refer to.) Their
code looks alien to web js programmers. So, familiarity with ns (netscape)
interfaces and the Firefox/XUL DOM is something else you should
specifically ask for.

IMO, rather than advertise, your best bet is to look for a Firefox
extension that has some characteristics of what you're looking for, then
contact the author. You can browse/search for extensions here...

https://addons.mozilla.org/en-US/firefox/

> The erlang stuff is easy :-) but the js is tricky - (and I guess this
> group is
> not the best lace to ask js questions?)
>
> (( where is the best group for this (ie js questions)))
>

The Mozilla add-on forum is where you'll find Firefox extension types who
speak "Mozilla" JavaScript...

https://developer.mozilla.org/forums/ucp.php?mode=login

That said, I'm going to put a day aside and try to whip something up
myself (assuming I've understood what you want.) I'm very interested in
being part of this project -- especially to learn more about Erlang. It's
also a good excuse to make the Mozilla Framework -> Firefox Extension
migration I've been meaning to do.

[Edmond Begumisa]

Dr Armstrong,

Some comments...

On Sat, 25 Sep 2010 20:39:43 +1000, Joe Armstrong <erl...@gmail.com> wrote:

This sounds very interesting. If I've understood correctly, you want the
results delivered to you on the Erlang side as HTML snippets/JSON with
embedded HTML strings? The clever parts will be on the Erlang side?

Also, where you say "throw me into an editor" -- do you mean a editor in
the word-processing sense or an editor in the html authoring sense (like
in Solvent?) The former would be easy (Mozilla have that infrastructure
already), the latter substantially more involved.

> This is very generic - if I could isolate "analyze something about the

> current page" into a convenient js module then I could do all sort of
> fun things.
>
> This seems to require a firefox plugin

Yes, you'd need to write a Firefox extension to do this.

> - any idea how to write this?
>

Yes and No. No because I've never actually written a Firefox

extension/add-on before. Yes because writing one is basically a

subset/specialised form of a Mozilla Framework/XULRunner application which
I do have experience with.

If you want to have a crack at it yourself (you know the saying "if you
want something done properly..."), you probably want to start with the XUL

tutorial, then the extension tutorial, then as a reference the official

free e-book "Creating Applications with Mozilla" which goes through the

Mozilla Application Framework...

https://developer.mozilla.org/en/XUL_Tutorial
http://kb.mozillazine.org/Getting_started_with_extension_development
https://developer.mozilla.org/en/Setting_up_extension_development_environment
https://developer.mozilla.org/en/Building_an_Extension
http://books.mozdev.org/html/index.html

You might need the js debugger too...

http://www.mozilla.org/projects/venkman/venkman-walkthrough.html

> If I want to start a collaborative project where do I advertise for

> smart javascript programmers who know the firefox internals.
>

I'm not sure where you'd advertise (Stackoverflow maybe), but I can advise
on what skills to advertise for...

You might want to specifically advertise for a programmer familiar with
the Mozilla Application Framework -- of which JavaScript is just an
element (XUL/XBL/XPCOM/RDF-XML being the major other bits.) This will weed
out the hobbyists. Another thing to look out for is the fact that the
Mozilla flavor of JavaScript is not quite the same as the typical
JavaScript. That is, the js used in web-pages is "narrower" than the js
used in Firefox extensions/XULRunner apps -- those familiar with the
former (prototype/jquery types) are unlikely to be useful with the latter.
For example, extension writers frequently access Mozilla C++ interfaces
via js and navigate a different DOM (the internals you refer to.) Their
code looks alien to web js programmers. So, familiarity with ns (netscape)
interfaces and the Firefox/XUL DOM is something else you should
specifically ask for.

IMO, rather than advertise, your best bet is to look for a Firefox
extension that has some characteristics of what you're looking for, then
contact the author. You can browse/search for extensions here...

https://addons.mozilla.org/en-US/firefox/

> The erlang stuff is easy :-) but the js is tricky - (and I guess this

> group is
> not the best lace to ask js questions?)
>
> (( where is the best group for this (ie js questions)))
>
>

The Mozilla add-on forum is where you'll find Firefox extension types who
speak "Mozilla JavaScript"...

https://developer.mozilla.org/forums/ucp.php?mode=login

That said, with your permission, I'd like to put a day aside and try to

whip something up myself (assuming I've understood what you want.) I'm
very interested in being part of this project -- especially to learn more
about Erlang. It's also a good excuse to make the Mozilla Framework ->

Firefox Extension migration I've been meaning to do. Is this something you
intend on open-sourcing?

[Edmond Begumisa]

My apologies,

Ignore that first reply with a blank 'from' address -- that was an
incomplete draft that my caffeinated fingers decided to send all on their
own.

- Edmond -

On Sat, 25 Sep 2010 20:39:43 +1000, Joe Armstrong <erl...@gmail.com> wrote:

[Ted Karmel]

Well though this is not the approriate forum for js, take a look at jetpack for easy firefox plugin scripting.