[erlang-questions] Two beautiful programs - or web programming made easy
Joe Armstrong
For a long time one of my favorite Erlang program was this:
loop() ->
receive
F -> F(),
loop()
end.
It's nice because it does very little, but what is does is universal. It
enables mobile code.
Well now I can do this in Javascript.
The Javascript equivalent is:
function onMessage(evt) {
eval(evt.data);
}
Where the data comes from a websocket.
Websockets are controlled with a simple API:
websocket = new WebSocket(wsUri);
websocket.onopen = function(evt) { onOpen(evt) };
websocket.onclose = function(evt) { onClose(evt) };
websocket.onmessage = function(evt) { onMessage(evt) };
Linking a websocket and Erlang is pretty easy. So now I can write code like
this in
erlang
-module(demo1).
-export([start/1]).
start(Pid) ->
Pid ! {eval, "document.body.innerHTML='';"},
Pid ! {eval, "document.body.style.backgroundColor='#eeffaa';"},
Pid ! {eval, "document.body.innerHTML+='<h1>Hello World</h1>'"},
event_loop(Pid).
event_loop(Pid) ->
receive
Any ->
io:format("??event loop:~p~n",[Any]),
event_loop(Pid)
end.
This code pushes asynchronous messages containing javascript to a generic
web page, and the web page evals the result.
This technique is amazingly powerful.
So now I only need one generic web page. Think of that.
Only one page is needed - forever.
All the generic pages does is:
loop:
wait for a message containing Javascript
eval the message
Beautiful
This is the easiest way to program a GUI I can conceive of.
You can download and run the examples from:
https://github.com/joearms/SEBG
You'll need something like Google chrome to run them.
Have fun
/Joe
Masklinn
>
> You'll need something like Google chrome to run them.
Also, it's going to be a complete and utter pain to debug: even the more modern JS debuggers of Firefox (via Firebug) and Webkit have insane amounts of trouble finding and debugging dynamically evaluated code.
[0] http://www.ietf.org/mail-archive/web/hybi/current/msg04744.html
[1] http://codereview.chromium.org/5643005/
________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-questio...@erlang.org
Joe Armstrong
The use of higher order functions that generate the code will make this
a lot easier.
The next stop is to generate the JS with HOFs and make sure the generators
are correct.
You don't want to write JS by hand - that's work - let a program do it for
you :-)
/Joe
Ulf Wiger
Well, I read those links not as an indictment of websockets per se, but of the UPGRADE-based handshake. I think Joe is definitely on to something here, and this sort of setup fits Erlang beautifully.
Essentially, Joe may have found a way to dust off his ex11 idea[1], in a context where it is much more likely to succeed. ;-)
(But please, Joe, leave out the !! this time - one revolution at a time.)
BR,
Ulf W
[1] http://ftp.sunet.se/pub/lang/erlang/workshop/2004/ex11.pdf
Ulf Wiger, CTO, Erlang Solutions, Ltd.
http://erlang-solutions.com
Joe Armstrong
<ulf....@erlang-solutions.com>wrote:
>
> Well, I read those links not as an indictment of websockets per se, but of
> the UPGRADE-based handshake. I think Joe is definitely on to something here,
> and this sort of setup fits Erlang beautifully.
>
> Essentially, Joe may have found a way to dust off his ex11 idea[1], in a
> context where it is much more likely to succeed. ;-)
>
> (But please, Joe, leave out the !! this time - one revolution at a time.)
>
!! would be evil in this context. As soon as you get outside erlang !! is
bad.
! is good because it creates no dependencies, the message either gets there
or
it does not. Code that can deal with this uncertainty will be good, other
code is
not.
It's actually pretty awe-inspiring to say
Pid ! button("click me") and see a button just appear in the browser
Great fun.
The great thing is that we can now abstract the interface
button("text") means return a thing that when evaluated makes a button
this code return Javascript, but it could equally well return xul, or html,
or llvm assembler
it doesn't matter - as long as the eval end knows how to eval it. So the top
level
code can be written entirely without knowledge of what's going on under the
covers
/Joe
/Joe
Jerome Martin
enable it by default, the world would be a better place to deploy web UIs
:-)
Because that remains the biggest issue for now, until I see this happening,
all that beauty might be lost :-(
--
Jérôme Martin
Masklinn
> Well, I read those links not as an indictment of websockets per se
I don't believe I wrote anything against Joe's idea, I just forwarded short and mid-term issues with actually applying them outside of well-controlled demonstration environments.
Joe Armstrong
A client (web browser) can do an RPC to a server
We can't (easily)
do an RPC from the server to the client
send an asynchronous message from server to client
send an asynchronous message from the client to the server
Of these four concurrency patterns three are crippled.
Instant messageing, for example,is trivial if you can push messages to a
client.
Turning web sockets off for security reasons is stupid.
The solution is to sandbox the thing that receives the messages
or if unsandboxed strongly authenticate the messages.
/Joe
and
Masklinn
> As I see it the web is predominantly built with one concurrency pattern.
>
> A client (web browser) can do an RPC to a server
>
> We can't (easily)
>
> do an RPC from the server to the client
> send an asynchronous message from server to client
> send an asynchronous message from the client to the server
In fact, synchronous RPC-type calls are definitely frowned upon as they create terrible use experience (due to in-browser javascript being a purely single-threaded event loop).
> Turning web sockets off for security reasons is stupid.
>
> The solution is to sandbox the thing that receives the messages
> or if unsandboxed strongly authenticate the messages.
I don't follow you here. You're saying it's OK to release broken standards (even though they're fixable) and asserting that every implementor should go through the process of creating his own broken sandbox which will fail to be correctly secured?
Marc Worrell
Zotonic switches between a long polling comet connection and websockets depending on the capabilities of the browser.
By the way, websockets is not broken. Some proxies are broken.
- Marc
Edmond Begumisa
> As I see it the web is predominantly built with one concurrency pattern.
>
> A client (web browser) can do an RPC to a server
>
> We can't (easily)
>
> do an RPC from the server to the client
> send an asynchronous message from server to client
These first 2 difficulties have been the biggest reason why I avoided
writing web-apps that run in the browser for so long (I still avoid it).
Just give me a socket and I'm happy.
When you're used to socket programming, where after a connection is made
the line between who is the server and who is the client becomes blurry,
it gets really hard to adapt to the idea that only the connectER can make
requests to the connectEE (pardon my word invention.) Designs that you
took for granted become next to impossible to implement.
> send an asynchronous message from the client to the server
>
> Of these four concurrency patterns three are crippled.
>
> Instant messageing, for example,is trivial if you can push messages to a
> client.
>
> Turning web sockets off for security reasons is stupid.
>
I agree 110%
With modern web-app development targeting browsers, a large amount of time
is being spent finding work-arounds to enable programmers to do things
that have been done in other environments. From bi-directional requests
(websockets/comet/long-poll) to rich-UI (menus, drag and drop, etc.)
I think web-development is out-growing the technologies it's based on --
web developers obviously want more. Attempts to stifle ideas like
websockets that are trying to respond to the demand won't succeed in the
long run.
In the meantime, runtime environments like Mozilla XULRunner and Adobe AIR
that give developers a wider range of patterns to choose from (while
giving them more responsibility for things like security) will gain more
traction while browsers debate on whether to lift their constraints. Me
thinks.
- Edmond -
> The solution is to sandbox the thing that receives the messages
> or if unsandboxed strongly authenticate the messages.
>
> /Joe
>
>
>
> and
>
> On Sat, Feb 12, 2011 at 1:49 PM, Masklinn <mask...@masklinn.net> wrote:
>
>> On 2011-02-12, at 13:15 , Ulf Wiger wrote:
>> > Well, I read those links not as an indictment of websockets per se
>> Why would you? They're not, they're just warnings that websockets are
>> getting disabled in browsers due to being currently broken, and that
>> this
>> kind of approach is therefore nice as a research toy but can not be
>> considered for serious work at this point.
>>
>> I don't believe I wrote anything against Joe's idea, I just forwarded
>> short
>> and mid-term issues with actually applying them outside of
>> well-controlled
>> demonstration environments.
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
Joe Armstrong
> On 2011-02-12, at 14:03 , Joe Armstrong wrote:
> > As I see it the web is predominantly built with one concurrency pattern.
> >
> > A client (web browser) can do an RPC to a server
> >
> > We can't (easily)
> >
> > do an RPC from the server to the client
> > send an asynchronous message from server to client
> > send an asynchronous message from the client to the server
> This is incorrect. HTTP does not support "push" messaging (from a server to
> a client), but the main (and default) communication mechanism in Javascript
> is asynchronous (and event-driven). Unless you have a very different
> definition of "asynchronous message" than the one I'm used to.
>
To what does the word "this" apply?
"this" (ie the first word of your comment) could refer to
any combination of four things - I said we can't easily ... send an
asynchronous message
from the server to the client - I am fully aware the HTTP does not support
this that
why I said "easily" it can be done with horrible work arounds like comet
etc.
>
> In fact, synchronous RPC-type calls are definitely frowned upon as they
> create terrible use experience (due to in-browser javascript being a purely
> single-threaded event loop).
>
>
> > Turning web sockets off for security reasons is stupid.
> >
> > The solution is to sandbox the thing that receives the messages
> > or if unsandboxed strongly authenticate the messages.
> I don't follow you here. You're saying it's OK to release broken standards
> (even though they're fixable) and asserting that every implementor should go
> through the process of creating his own broken sandbox which will fail to be
> correctly secured?
I didn't say it was ok to release broken standards.
I didn't say "broken sandbox" I said "sandbox" - If you want to be secure
you should put
every application in a trusted sandbox. That way you can play with web
sockets even if they
are broken. You need a security policy for your sandbox that disallows
access to local storage, opening new socket etc.
/Joe
Masklinn
>
> By the way, websockets is not broken. Some proxies are broken.
Masklinn
On 2011-02-12, at 15:52 , Joe Armstrong wrote:
> On Sat, Feb 12, 2011 at 3:29 PM, Masklinn <mask...@masklinn.net> wrote:
>
>> On 2011-02-12, at 14:03 , Joe Armstrong wrote:
>>> As I see it the web is predominantly built with one concurrency pattern.
>>>
>>> A client (web browser) can do an RPC to a server
>>>
>>> We can't (easily)
>>>
>>> do an RPC from the server to the client
>>> send an asynchronous message from server to client
>>> send an asynchronous message from the client to the server
>> This is incorrect. HTTP does not support "push" messaging (from a server to
>> a client), but the main (and default) communication mechanism in Javascript
>> is asynchronous (and event-driven). Unless you have a very different
>> definition of "asynchronous message" than the one I'm used to.
>>
>
> To what does the word "this" apply?
>
> "this" (ie the first word of your comment) could refer to
> any combination of four things
This referes to what i quoted. The sum is incorrect since some of its parts (namely your assertion that asynchronous client->server communications are "not easy") are incorrect. Which is what I explain in the next three lines.
> - I said we can't easily ... send an
> asynchronous message
> from the server to the client - I am fully aware the HTTP does not support
> this that
> why I said "easily" it can be done with horrible work arounds like comet
> etc.
You also said, in the same bundle, that "we can't easily send an asynchronous message from the client to the server". Which is completely incorrect.
>>> Turning web sockets off for security reasons is stupid.
>>>
>>> The solution is to sandbox the thing that receives the messages
>>> or if unsandboxed strongly authenticate the messages.
>> I don't follow you here. You're saying it's OK to release broken standards
>> (even though they're fixable) and asserting that every implementor should go
>> through the process of creating his own broken sandbox which will fail to be
>> correctly secured?
> I didn't say it was ok to release broken standards.
You said turning off web sockets due to them being broken was stupid. It follows that you advocate releasing and enabling websockets in a broken state.
Edmond Begumisa
>
> On 2011-02-12, at 15:52 , Joe Armstrong wrote:
>
>> On Sat, Feb 12, 2011 at 3:29 PM, Masklinn <mask...@masklinn.net> wrote:
>>
>>> On 2011-02-12, at 14:03 , Joe Armstrong wrote:
>>>> As I see it the web is predominantly built with one concurrency
>>>> pattern.
>>>>
>>>> A client (web browser) can do an RPC to a server
>>>>
>>>> We can't (easily)
>>>>
>>>> do an RPC from the server to the client
>>>> send an asynchronous message from server to client
>>>> send an asynchronous message from the client to the server
>>> This is incorrect. HTTP does not support "push" messaging (from a
>>> server to
>>> a client), but the main (and default) communication mechanism in
>>> Javascript
>>> is asynchronous (and event-driven). Unless you have a very different
>>> definition of "asynchronous message" than the one I'm used to.
>>>
Possibly you do have different definitions of "asynchronous message" :) I
don't think client-side JavaScript actually push asynchronous messages
over the wire, it just looks like this from the web-programmer's
perspective. See below...
>>
>> To what does the word "this" apply?
>>
>> "this" (ie the first word of your comment) could refer to
>> any combination of four things
> This referes to what i quoted. The sum is incorrect since some of its
> parts (namely your assertion that asynchronous client->server
> communications are "not easy") are incorrect. Which is what I explain in
> the next three lines.
>
>> - I said we can't easily ... send an
>> asynchronous message
>> from the server to the client - I am fully aware the HTTP does not
>> support
>> this that
>> why I said "easily" it can be done with horrible work arounds like comet
>> etc.
> You also said, in the same bundle, that "we can't easily send an
> asynchronous message from the client to the server". Which is completely
> incorrect.
>
I have to disagree, I think the statement is perfectly correct. I believe
what Joe meant here is:
* From the client, HTTP requests are not asynchronous because HTTP
requests (GET/PUT/POST) by definition await a response from the server
(200 OK, 404, etc.) This is not send and forget. It's send and wait. It's
synchronous communication built on asynchronous communication.
* The browser hides the above fact from you (the web-developer) by
enabling the single-threaded JavaScript environment to do other work
in-between the request and the response even executing other requests (aka
AJAX.) So your JavaScript is async but the actual messages (HTTP requests)
are not.
* One workaround to send true async messages from the client to the server
(which *I* know of and have used) is to multipart POST with a very large
content-length header value and keep adding to the body, say lines of
JSON, and allow the server to "eat" them as they come while not concluding
the request for as long as possible. A kind long-poll in reverse. This is
a pain for both client code and server code which is why I think Joe is
correct when he says it's not "easy".
- Edmond -
>>>> Turning web sockets off for security reasons is stupid.
>>>>
>>>> The solution is to sandbox the thing that receives the messages
>>>> or if unsandboxed strongly authenticate the messages.
>>> I don't follow you here. You're saying it's OK to release broken
>>> standards
>>> (even though they're fixable) and asserting that every implementor
>>> should go
>>> through the process of creating his own broken sandbox which will fail
>>> to be
>>> correctly secured?
>> I didn't say it was ok to release broken standards.
> You said turning off web sockets due to them being broken was stupid. It
> follows that you advocate releasing and enabling websockets in a broken
> state.
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questio...@erlang.org
>
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
________________________________________________________________
Joe Armstrong
How the heck can you easily send an an asynchronous message from a client to
a server?
HTTP get and post wait for a reply and thus are synchronous.
By asynchronous I mean "fire and forget" like UDP.
If it easy easy them I'd love to know how ..
>
> >>> Turning web sockets off for security reasons is stupid.
> >>>
> >>> The solution is to sandbox the thing that receives the messages
> >>> or if unsandboxed strongly authenticate the messages.
> >> I don't follow you here. You're saying it's OK to release broken
> standards
> >> (even though they're fixable) and asserting that every implementor
> should go
> >> through the process of creating his own broken sandbox which will fail
> to be
> >> correctly secured?
> > I didn't say it was ok to release broken standards.
> You said turning off web sockets due to them being broken was stupid. It
> follows that you advocate releasing and enabling websockets in a broken
> state.
I never mentioned "releasing" or "enabling" in my post so you can't say
I advocated either of these. I said turning them off was stupid - nothing
else.
Actually i wasn't bothered about whether they were broken or not. I was
advocating using them because they are useful. And turning off something
that is useful
is stupid. Windows, for example, allows viruses (by stupid design) but it is
still useful.
Virtually all software I've ever seen is broken in one way or another. It
can still be
useful even if it is broken. It's not a question of absolutes here.
Even if they are broken they are still useful and i can easily put a layer
of strong crypto
on top if I were worried - which I'm not.
/Joe
Jerome Martin
> On 2011-02-12, at 15:34 , Marc Worrell wrote:
> >
> > By the way, websockets is not broken. Some proxies are broken.
> If these proxies being broken create a *security issue* with using
> websockets, then websockets themselves are broken.
>
This must be one of the most nonsensical sentence I have read since a long
time from someone supposedly mastering basic logic. Nice one.
--
Jérôme Martin
Edmond Begumisa
Sounded like circular logic, but I _kind of_ see what he was trying to say
;)
Browsers, much like OSs, are in the unfortunate position of having to be
careful what they introduce coz of the impact on other-people's faulty
code. Too careful in the websockets case. Web developers have been
suffering from the uni-directional issue for too long. It's not a minor
limitation, it's major. Broken proxies just need to be fixed and not pass
the blame.
I've waited in vain for the Mozilla Framework (and hence Firefox) to
support websockets. The debate reminds me of Linus rather rudely
complaining about the undue status given to fixing "security" bugs over
"normal" bugs...
http://thread.gmane.org/gmane.linux.kernel/701694/focus=706950
The solution might be for web-developers to move away from browsers and
handle their own security.
- Edmond -
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
________________________________________________________________
Alain O'Dea
> On 2011-02-12, at 15:34 , Marc Worrell wrote:
>>
>> By the way, websockets is not broken. Some proxies are broken.
> If these proxies being broken create a *security issue* with using websockets, then websockets themselves are broken.
To clarify for the unwary:
No exploit lab or otherwise exists that functions in the context of the whole websockets stack from Chrome or Firefox. The vulnerability is identified on one component in isolation where it never would be in any real system.
Florian Weimer
> How the heck can you easily send an an asynchronous message from a client to
> a server?
>
> HTTP get and post wait for a reply and thus are synchronous.
>
> By asynchronous I mean "fire and forget" like UDP.
>
> If it easy easy them I'd love to know how ..
XMLHttpRequest has an asynchronous option.
And you can get emulate a single full-duplex connection using two
half-duplex ones, so Websockets is just an optimization, and likely
not even a very much needed one. The barrier right now seems to be
the BSD sockets API and kernel state management, and you still have
that with Websockets.
Alain O'Dea
> * Joe Armstrong:
>
>> How the heck can you easily send an an asynchronous message from a client to
>> a server?
>>
>> HTTP get and post wait for a reply and thus are synchronous.
>>
>> By asynchronous I mean "fire and forget" like UDP.
>>
>> If it easy easy them I'd love to know how ..
>
> XMLHttpRequest has an asynchronous option.
>
> And you can get emulate a single full-duplex connection using two
> half-duplex ones, so Websockets is just an optimization, and likely
> not even a very much needed one. The barrier right now seems to be
> the BSD sockets API and kernel state management, and you still have
> that with Websockets.
HTTP is a synchronous protocol. The async option is XmlHttpRequest is only to background the synchronous request processing so that the browser remains responsive.
Websockets is not merely an optimization. It removes the packaging around conversational events in a web application. This has far-reaching scalability impact since it reduces the cost per client substantially in a chatty application.
For an application using SEBG tge reduced cost per request will reduce the delay between a server event and its realization on the client.
Again, XmlHttpRequest's async option does not make the HTTP request itself asynchronous (it wouldn't be HTTP). It merely runs the synchronous requedt in the background on the client.
Edmond Begumisa
wrote:
> * Joe Armstrong:
>
>> How the heck can you easily send an an asynchronous message from a
>> client to
>> a server?
>>
>> HTTP get and post wait for a reply and thus are synchronous.
>>
>> By asynchronous I mean "fire and forget" like UDP.
>>
>> If it easy easy them I'd love to know how ..
>
> XMLHttpRequest has an asynchronous option.
>
There seems to be some confusion. Forgive me, but aren't asynchronous HTTP
requests very different from asynchronous messages? Isn't the
"asynchronous" in XMLHttpRequest talking about the scripting environment
in relation to thread-blocking and not the messages themselves?
Asynchronous XMLHttpRequest just means that you can do something else
while you wait for the reply, but a failed reply is still a fault so the
message depends on the reply (or the reply can be viewed as part of the
message). XMLHttpRequest is just a workaround to allow single-threaded
scripting environments not to block so they can handle multiple requests
without introducing multi-threading, but you *always* have to deal with
the reply. My understanding is XMLHttpRequest in whatever mode does not
magically turn HTTP requests into asynchronous *messages*.
What is an "asynchronous message" anyway?
AFAIK, it means send and forget. For Erlang's case this is...
Recipient ! Message
From here on, you are free to go about your business and not have to deal
with any response. The recipient need not send you back any
acknowledgement to make the message "final". The message is thus
asynchronous because you send and forget.
Compare this with HTTP...
PUT /foo?message=Message HTTP/1.1
From here, the message is not complete until the server responds...
HTTP/1.1 201 Created [or 504 or whatever]
For XMLHttpRequest in synchronous mode, the scripting environment's thread
blocks until the message is concluded. For XMLHttpRequest in asynchronous
mode, the scripting environment's thread churns on but the message is
still not concluded until the reply arrives. The message is thus
synchronous either way because you don't send and forget, you have to
write code to deal with the reply.
Correct me if I'm wrong, but if you send and have to deal with a response
from the recipient, regardless of whether you can do something else as you
await the response, the message itself is not asynchronous. An HTTP
message consists of 1) send and 2) get reply, so is synchronous by
definition. XMLHttpRequest can't erase this, but it can allow you to
workaround it.
The workaround is an ugly hack that involves packing many "sub" messages
into the request and waiting forever for a reply while taking advantage of
non-thread-blocking in the scripting environment (i.e. comet/long-poll)...
So, to send truly asynchronous *messages* from server to client, the
client would...
GET /foo/comet_interface HTTP/1.1
Server responds, streaming back a reply while not concluding the reply for
as long as it can hold the connection open. It first sends headers, then
keeps appending the asynchronous messages...
HTTP/1.1 200 OK
..Headers..
InnerMessage1
InnerMessage2
..
InnerMessageN
Client then uses it's non-thread-blocking XMLHttpRequest to read the inner
messages as they arrive and tries not to close the connection. The client
has to know how to chop them up and the server might need to keep sending
a regular special heartbeat sequence when idle to keep the connection
open. So the "outer" HTTP GET message itself is synchronous (send and wait
for reply), but the "inner" messages packed inside are asynchronous (send
and forget). This is ugly, but web-developers have made it work out of
desperation.
To send truly asynchronous messages from client to server is even tricker
and uglier. It might looks something like...
POST /foo/multipart_interface HTTP/1.1
Content-type: multipart/form-data, boundary=AaB03x
--AaB03x
content-disposition: form-data; name="message1"
InnerMessage1
--AaB03x
content-disposition: form-data; name="message2"
InnerMessage2
--AaB03x
...
content-disposition: form-data; name="messageN"
InnerMessageN
--AaB03x--
Again, the "outer" HTTP POST message is synchronous (send and wait for
reply), but the "inner" messages are asynchronous (send and forget). This
hackery is harder than it needs to be.
> And you can get emulate a single full-duplex connection using two
> half-duplex ones,
Synchronous full-duplex messages are still synchronous messages. Both
sides still expect replies from the other, they can just do so at the same
time. What you can do, as illustrated above, is simulate asynchronous
messages with synchronous HTTP. And it's complete pain in the arse.
> so Websockets is just an optimization,
I disagree. WebSockets allow you to send asynchronous full-duplex messages
without the hackery of comet/long-polling. Just look at how clean Joe's
code is. A websocket-less version would be possible, but uglier.
> and likely not even a very much needed one.
IMO, the very fact that web-developers have frequently used various
workarounds proves that it's needed. There is a disconnect between what
web developers want and what the infrastructure currently provides.
Websockets tries to address some of that.
Personally, I think it's *very* much needed because it opens up the
possibilities of what you can do with the web-browser as a client. The web
browser starts to look more like a ordinary TCP client. You can do more of
the things that are easy with plain sockets which have always frustrated
me with browsers. Just look at Joe's idea. Those sorts of ideas don't
immediately spring to mind when you're stuck in the mindset of
uni-directional synchronous messages.
- Edmond -
> The barrier right now seems to be
> the BSD sockets API and kernel state management, and you still have
> that with Websockets.
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questio...@erlang.org
>
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
________________________________________________________________
Jerome Martin
(warning, if you are prompt to be upset by dramatically different opinion
than your, please do not read the following)
On a totally (not so ?) subjective mode:
- (my) Taste tells me Joe's code is beautiful.
- If such beautiful code is conceptually possible, it seems the right
thing to do (remember, we are in subjective mode)
- Preventing that beauty from happening (not supporting features enabling
it), or claiming it is not necessary (presenting ugly code that does not
feel right and claiming it is the same) seems like the worst thing to do,
even if alternatives can be motivated (security reasons, etc.).
Now, on a purely pragmatic mode:
- My experience (dealing with computer code since 20 years) tells me that
my taste buds are not that subjective, and are more than often right:
usually, I find (or someone else does) later a technical demonstration that
intuition and taste got it right.
- It is rare enough to trigger taste to a level that we can call beautiful,
so definitely Joe must be on to something there :)
To everyone with rotten taste buds (serious mode but I won't argue or defend
my views):
- No, taste is not that subjective. I do think some people have bad taste
and some other good taste, in an absolute sense. Matters of taste are a tale
imagined not to upset anyone.
- Can't you see that there is something beyond technical arguments thee
that yells "this is the right way to do it" ? I mean, really ?
--
Jérôme Martin
Edmond Begumisa
The thread went off topic. Joe e-mailed me with an idea like this many
months ago off-list. I liked it then and I like it now. I even borrowed
little bits of it in my current XULRunner projects (XUL has some
limitations preventing the exact model).
In fact, I've been thinking of some ways adding some templating to his
code. So you can send "groups of widgets" down to the browser to display.
It also got me thinking about some sort of flow language that can be used
for people that don't know Erlang so they can easily string screens and
widgets together in a flow chart manner.
- Edmond -
Edmond Begumisa
<tramjo...@gmail.com> wrote:
> - Preventing that beauty from happening (not supporting features
> enabling it), or claiming it is not necessary (presenting ugly code that
> does not
> feel right and claiming it is the same) seems like the worst thing to do,
> even if alternatives can be motivated (security reasons, etc.).
And BTW: I was presenting the ugly code precisely to show that although
similar, it doesn't not feel right, and hence, NOT supporting the features
enabling the beautiful code is holding everyone back from coming up with
great ideas like Joe's.
- Edmond -
Edmond Begumisa
<ebeg...@hysteria-tech.com> wrote:
> On Mon, 14 Feb 2011 04:08:05 +1100, Jerome Martin
> <tramjo...@gmail.com> wrote:
>
>> - Preventing that beauty from happening (not supporting features
>> enabling it), or claiming it is not necessary (presenting ugly code
>> that does not
>> feel right and claiming it is the same) seems like the worst thing to
>> do,
>> even if alternatives can be motivated (security reasons, etc.).
>
> And BTW: I was presenting the ugly code precisely to show that although
> similar, it doesn't not feel right, and hence, NOT supporting the
OOPS: ^^^^^^^^^^^^^^^^^^^^^^^^^
TYPO: it doesn't feel right
Jerome Martin
ebeg...@hysteria-tech.com> wrote:
> On Mon, 14 Feb 2011 04:08:05 +1100, Jerome Martin <tramjo...@gmail.com>
> wrote:
>
> - Preventing that beauty from happening (not supporting features enabling
>> it), or claiming it is not necessary (presenting ugly code that does not
>> feel right and claiming it is the same) seems like the worst thing to do,
>> even if alternatives can be motivated (security reasons, etc.).
>>
>
> And BTW: I was presenting the ugly code precisely to show that although
> similar, it doesn't not feel right,
Yup, that was clear, don't worry :-) Sorry for phrasing my point in a way
that could make one think you were defending that ugly code or presenting it
as a viable alternative.
and hence, NOT supporting the features enabling the beautiful code is
> holding everyone back from coming up with great ideas like Joe's.
>
<nod>
--
Jérôme Martin
Jerome Martin
post<http://armstrongonsoftware.blogspot.com/2009/12/comet-is-dead-long-live-websockets.html>
I
think many people have had that idea in the back of their minds, floating
around, waiting for some signal that WebSocket was out to the general public
for good and could be considered as a usable in real life. Unfortunately, as
we all realize, it is not the case yet, and most frustrating, probably for
the wrong reasons :-(
On Sun, Feb 13, 2011 at 6:25 PM, Edmond Begumisa <
ebeg...@hysteria-tech.com> wrote:
--
Jérôme Martin
Joe Armstrong
> Yes, since Joe's WebSocket blog post<http://armstrongonsoftware.blogspot.com/2009/12/comet-is-dead-long-live-websockets.html> I
> think many people have had that idea in the back of their minds, floating
> around, waiting for some signal that WebSocket was out to the general public
> for good and could be considered as a usable in real life. Unfortunately, as
> we all realize, it is not the case yet, and most frustrating, probably for
> the wrong reasons :-(
>
>
Well it might not be universally available, but it's here and now on my
machine.
All the general public have to do is change browsers, and how difficult is
that?
It's one click away.
While the giants fight for market share by making their browsers mutually
incompatible
I just download the one that best suits my purposes.
I just want to make things as simple as possible - life is to short to play
browser wars.
Hopefully a kind of software Darwinism will prevail - that's my naive
belief.
Life is too short to fix broken software.
/Joe
Frédéric Trottier-Hébert
On 2011-02-12, at 06:33 AM, Joe Armstrong wrote:
>
> The Javascript equivalent is:
>
> function onMessage(evt) {
> eval(evt.data);
> }
>
> Where the data comes from a websocket.
>
This is rather risky. Eval will take any code whatsoever and run it for you. If you have dynamic content, without proper escaping and being very careful, users could run arbitrary code in your page, including stuff to steal session data and send it over to either some other site, or perform actions for the user which they do not necessarily approve on (making their profile public, closing their account, worms, etc.)
In fact, this is a reason why people like Douglas Crockford prefered to write JSON parsers rather than just evaluating them. It's just not safe enough.
Plus you have to call the javascript parser and whatnot, which is usually rather slow. The whole idea is pretty bad on the web, where you have to assume that people will actively try to break your stuff and steal data from other users (or you).
>
> This technique is amazingly powerful.
>
> So now I only need one generic web page. Think of that.
>
> Only one page is needed - forever.
>
This is a problem when it comes to bookmarks, sharing the link with a friend, searchability, browser history, etc. The web wasn't exactly intended to be a stateful thing and you'll have to resort to hacks such as hash-bangs to get around it. I suggest reading Tim Bray's Broken Links to see why that isn't a good solution anyway.
Plus I'd argue that javascript and Erlang should be kept separate and you shouldn't try to generate one with the other, but at this point, I figure it's more of a matter of who wants to give himself the trouble than anything.
--
Fred Hébert
http://www.erlang-solutions.com
Alain O'Dea
>
> On 2011-02-12, at 06:33 AM, Joe Armstrong wrote:
>
>>
>> The Javascript equivalent is:
>>
>> function onMessage(evt) {
>> eval(evt.data);
>> }
>>
>> Where the data comes from a websocket.
>>
> This is rather risky. Eval will take any code whatsoever and run it for you. If you have dynamic content, without proper escaping and being very careful, users could run arbitrary code in your page, including stuff to steal session data and send it over to either some other site, or perform actions for the user which they do not necessarily approve on (making their profile public, closing their account, worms, etc.)
>
> In fact, this is a reason why people like Douglas Crockford prefered to write JSON parsers rather than just evaluating them. It's just not safe enough.
>
> Plus you have to call the javascript parser and whatnot, which is usually rather slow. The whole idea is pretty bad on the web, where you have to assume that people will actively try to break your stuff and steal data from other users (or you).
The source of of the eval'd script is the websocket which is domain locked. The scripts can only act in the context of the page. This is much safer than an iPhone app for example which has far more liberal and troubling data access. The trouble with eval is when you are using it for scripts whose source is unverified. The security risks are not serious in this case so lobg as you take care on the server side.
>
>>
>> This technique is amazingly powerful.
>>
>> So now I only need one generic web page. Think of that.
>>
>> Only one page is needed - forever.
>>
> This is a problem when it comes to bookmarks, sharing the link with a friend, searchability, browser history, etc. The web wasn't exactly intended to be a stateful thing and you'll have to resort to hacks such as hash-bangs to get around it. I suggest reading Tim Bray's Broken Links to see why that isn't a good solution anyway.
Bookmarking is definitely hard to achieve in this scenario, but hashes (like you say) could be used to restore previous states. Alternatively, you could take the view that the current state of the resource is its persistent state so long as the initial content the user gets upon visiting the page is the aggregate of all previous incremental websocket changes. In general this could be achieved by appending all the JS of those together and storing it or by having some form of caching user agent that tracks the outcomes.
>
> Plus I'd argue that javascript and Erlang should be kept separate and you shouldn't try to generate one with the other, but at this point, I figure it's more of a matter of who wants to give himself the trouble than anything.
>
>
> --
> Fred Hébert
> http://www.erlang-solutions.com
>
>
________________________________________________________________
Roberto Ostinelli
> Yes, since Joe's WebSocket blog
> post<
> http://armstrongonsoftware.blogspot.com/2009/12/comet-is-dead-long-live-websockets.html
> >
> I
> think many people have had that idea in the back of their minds, floating
> around, waiting for some signal that WebSocket was out to the general
> public
> for good and could be considered as a usable in real life. Unfortunately,
> as
> we all realize, it is not the case yet, and most frustrating, probably for
> the wrong reasons :-(
>
i have implemented websockets in misultin, which is built to support
multiple websockets versions at the same time.
i have to say that i've currently stopped developing, waiting for a
finalized version to come out. tired of losing time on this, and eager to
settle to something finally usable out there.
r.
Edmond Begumisa
<fred....@erlang-solutions.com> wrote:
>
> On 2011-02-12, at 06:33 AM, Joe Armstrong wrote:
>
>>
>> The Javascript equivalent is:
>>
>> function onMessage(evt) {
>> eval(evt.data);
>> }
>>
>> Where the data comes from a websocket.
>>
> This is rather risky. Eval will take any code whatsoever and run it for
> you.
Likewise the browser will take any static js (<script> tags) whatsoever
from your server and run it for you.
> If you have dynamic content, without proper escaping and being very
> careful, users could run arbitrary code in your page, including stuff to
> steal session data and send it over to either some other site, or
> perform actions for the user which they do not necessarily approve on
> (making their profile public, closing their account, worms, etc.)
>
Likewise if you have any dynamic content in js code on your server without
proper escaping and not being very careful, users could...
... Don't the "same source" XXS rule for non-evaled code apply to evaled
code? Doesn't the same duty of care to end-users for protecting privacy,
properly escaping data, etc, apply in both cases? Don't you have to be
careful either way?
> In fact, this is a reason why people like Douglas Crockford prefered to
> write JSON parsers rather than just evaluating them. It's just not safe
> enough.
>
Indeed you are correct, but...
From http://www.json.org/js.html ...
"...The use of eval is indicated when the source is *trusted* and
*competent*..."
"...In web applications over XMLHttpRequest, communication is permitted
only to the same origin that provide that page, so it is *trusted*. But it
*might not be competent*. If the server is not rigorous in its JSON
encoding, or if it does not scrupulously validate all of its inputs, then
it could deliver invalid JSON text that could be carrying dangerous
script..."
So it boils down to the competence of the code on the server. You have to
be careful how you construct your pages and javascript. But then, this
should *always* be the case.
> Plus you have to call the javascript parser and whatnot, which is
> usually rather slow.
One could send core of the app logic in a static js file then have the
eval only making simple calls like "appui.getInvoinces()". That will
perform fairly well.
> The whole idea is pretty bad on the web, where you have to assume that
> people will actively try to break your stuff and steal data from other
> users (or you).
>
That assumption is a bit dramatic. Questions on security cannot be viewed
in isolation of application. One of my favorite quotes from Bruce Schneier
is applicable here. He was once asked about the possibility of chaos
ensuing due to internet security breaches...
"No. Chaos is hard to create, even on the Internet. Here's an example. Go
to Amazon.com. Buy a book without using SSL. Watch the total lack of
chaos."
I don't see how you can canvas the "whole idea" as being bad. It may
require adjustments here and there. e.g For particular pages where
paranoid security is needed, nothing stops you from doing it differently
there. You could crypto what's sent. You could even serve those pages the
standard way with static files and SSL if it makes you feel safer.
>>
>> This technique is amazingly powerful.
>>
>> So now I only need one generic web page. Think of that.
>>
>> Only one page is needed - forever.
>>
> This is a problem when it comes to bookmarks, sharing the link with a
> friend, searchability, browser history, etc. The web wasn't exactly
> intended to be a stateful thing and you'll have to resort to hacks such
> as hash-bangs to get around it. I suggest reading Tim Bray's Broken
> Links to see why that isn't a good solution anyway.
>
True. But this problem is an age-old general AJAX/dynamic-markup problem.
I agree it might be very visible in this case.
However, I've written XULRunner apps with no back buttons -- no need for
them with easy-to-navigate UIs. Most Adobe AIR apps I've seen have no
browser history. It's made me question: How badly do end-users really need
those things? If they do, couldn't we give them better
application-specific versions inside our web-app UI?
> Plus I'd argue that javascript and Erlang should be kept separate and
> you shouldn't try to generate one with the other,
Good point. I thought about sending the js in static files and reducing
the calls from Erlang to simple one-liners. But also note that the more
powerful aspect of this (IMO) is not just sending js, but sending UI
elements. Sending blocks of UI to an empty page! How can anyone not like
that?
- Edmond -
> but at this point, I figure it's more of a matter of who wants to give
> himself the trouble than anything.
>
>
> --
> Fred Hébert
> http://www.erlang-solutions.com
>
>
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
________________________________________________________________
Todd
https://github.com/ToddG/experimental/tree/master/erlang/wilderness
If anyone has comments about what I'm doing, better tools, better ways
of approaching and/or explaining... I'd love to get feedback.
-Todd
Jerome Martin
2011/2/13 Edmond Begumisa <ebeg...@hysteria-tech.com>
> [...]
>
> However, I've written XULRunner apps with no back buttons -- no need for
> them with easy-to-navigate UIs. Most Adobe AIR apps I've seen have no
> browser history. It's made me question: How badly do end-users really need
> those things?
I think users are _hooked_ to back-button, the same way most of us are
hooked to undo. I find those two to be essentially the same, in terms of
behavioral semantics. However, maybe the question here is whether or not the
history + back button functionalities being generic and implemented at the
browser level is a good thing, or if it is too restrictive. Clearly, "the
web" has evolved, and still haven't swallowed the "web application" pill
completely. Maybe a standard way to delegate 'undo/back' to the web
application, could be useful, like a meta HTTP tag containing the URL to
point at in case of the user using back/undo. But then I am not sure anymore
that HTTP/HTML is the proper foundation for this... An then suddenly I
realize that this is too obvious an idea not to be thought by people smarter
than me, so I just looked up HTML5 on w3c site and voila:
window.history.back() and window.history.forward() provide programmatic
interfaces to browser back and forward functions in HTML5. See this
page<http://www.w3.org/TR/html5/history.html#the-history-interface>,
that one <http://html5demos.com/history> and maybe even this
one<http://www.adequatelygood.com/2010/7/Saner-HTML5-History-Management>
.
> If they do, couldn't we give them better application-specific versions
> inside our web-app UI?
Asked myself that very question 2 minutes ago, and it seems HTML5 has the
answer for us :-)
So I would say that Joe's code could very much be well integrated in all
HTML5-supporting browser, granted that the appropriate hooks are set on the
javascript side, and some standard logic hanging on those hooks on the
erlang side (gen_html5app anyone ?).
Thoughts ?
--
Jérôme Martin
Jerome Martin
github repo, but unti I dig more, just wanted to say I like the approach,
thanks for sharing!
--
Jérôme Martin
Todd
Thanks! As an erl-newbie, I'm really curious about a couple of things:
1. The differences between rebar and sinan, what people use and why
I want to spend as little time as possible wiring up boilerplate and
figuring out how to package up project dependencies.
2. How to *effectively* peruse the documentation. I wound up extracting
the erlang man pages into a text file so that I can grep around and find
things.
I'll stop there, as the reality is that I'm pretty curious about 100
other things... It's really a full-time job getting up-to-speed on Erlang.
-Todd
On 2/13/11 1:57 PM, Jerome Martin wrote:
> I wouldn't comment much because I barely scratched the surface of your
> github repo, but unti I dig more, just wanted to say I like the
> approach, thanks for sharing!
>
> On Sun, Feb 13, 2011 at 10:29 PM, Todd <t.green...@gmail.com
> <mailto:t.green...@gmail.com>> wrote:
>
> I'm just starting to document my experiences learning erlang here:
>
> https://github.com/ToddG/experimental/tree/master/erlang/wilderness
>
> If anyone has comments about what I'm doing, better tools, better
> ways of approaching and/or explaining... I'd love to get feedback.
>
> -Todd
>
> ________________________________________________________________
> erlang-questions (at) erlang.org <http://erlang.org> mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questio...@erlang.org
> <mailto:erlang-questio...@erlang.org>
>
>
>
>
> --
> J�r�me Martin
Kenny Stone
Andrew Matthews
As an even newbier n00b than you, I found it very useful, and I'll be following your notes closely.
I'd be particularly interested in anything you produce on deploying and using distributions produced from 'sinan dist'. Thanks!
Regards,
Andrew
Andrew Matthews
On a side note - I have a .NET web-based softphone system that (prior to COMET becoming widely available in the .NET tech-stack) used polling to establish duplex comms between the browser and the server. Often at rates of up to 4 Hz and for hundreds/thousands of users simultaneously. The challenge was to boil down the payload down to the absolute minimum. The end result was that we pushed all of the presentation-level intelligence off to the client, and sent state-machine vectors (i.e. a vector of the current states of each of the controlling state models, represented as integers) instead, since they were the most dense way of saying how the UI ought to look.
My point is that for fear of blowing out your bandwidth usage, you might eventually end up making the payload more and more declarative, till eventually the payload could barely be called code at all. Would you really control a UI from a server in practice, when you can push off that job to the browser itself?
Regards,
Andrew
-----Original Message-----
> --snip --
Well it might not be universally available, but it's here and now on my machine.
All the general public have to do is change browsers, and how difficult is that?
It's one click away.
While the giants fight for market share by making their browsers mutually incompatible I just download the one that best suits my purposes.
I just want to make things as simple as possible - life is to short to play browser wars.
Hopefully a kind of software Darwinism will prevail - that's my naive belief.
Life is too short to fix broken software.
/Joe
________________________________________________________________
Alain O'Dea
2. http://erldocs.com/ is excellent for rapidly searching the docs. It doesn't have the user guides, but once you find the thing you need you can cross-reference http://erlang.org/doc/ for those.
Take a look at http://learnyousomeerlang.com/ if you get a chance. It is a great tutorial with a good sense of humour.
In reality it's a full time job to get up to speed on any language or framework to the point that you can work professionally with it. Our "Visually Learn", "in 24 Hours" culture just leads to unreasonable expectations and disappointment. I started learning Erlang in 2007 and I loved it from the beginning. I was able to build some useful and fun things with it quickly, but with a very odd code style and chock full of bad practices. That's a good thing really. You need to enjoy the language first by making it your own or — in my case — by writing Java-ish code in Erlang. The suit and tie of idiomatic Erlang comes later. Have fun, share your experience, give yourself a lot of time to become an expert and give yourself lots of credit for taking on the challenge of doing it right :)
Keep up the good work, I love the repo tutorial you have going :)
>> Jérôme Martin
Frédéric Trottier-Hébert
On 2011-02-13, at 15:43 PM, Edmond Begumisa wrote:
> On Mon, 14 Feb 2011 05:59:19 +1100, Frédéric Trottier-Hébert <fred....@erlang-solutions.com> wrote:
>
>>
>> On 2011-02-12, at 06:33 AM, Joe Armstrong wrote:
>>
>>>
>>> The Javascript equivalent is:
>>>
>>> function onMessage(evt) {
>>> eval(evt.data);
>>> }
>>>
>>> Where the data comes from a websocket.
>>>
>> This is rather risky. Eval will take any code whatsoever and run it for you.
>
> Likewise the browser will take any static js (<script> tags) whatsoever from your server and run it for you.
Right. This is why ideally you want to pass in very precise function and do something RPC-like (despite Joe not liking it) or have your own parser (as it is the case with JSON). It's not that it's impossible to make the other ways safe, but I wouldn't trust most people (including myself) to get it right most of the time.
>
>> If you have dynamic content, without proper escaping and being very careful, users could run arbitrary code in your page, including stuff to steal session data and send it over to either some other site, or perform actions for the user which they do not necessarily approve on (making their profile public, closing their account, worms, etc.)
>>
>
> Likewise if you have any dynamic content in js code on your server without proper escaping and not being very careful, users could...
Exactly. You're always as safe as your weakest link. Some frameworks (like JQuery on some methods) handle it for you, but usually there is no such thing for 'eval'.
> ... Don't the "same source" XXS rule for non-evaled code apply to evaled code? Doesn't the same duty of care to end-users for protecting privacy, properly escaping data, etc, apply in both cases? Don't you have to be careful either way?
>
Yes. But some ways to do things are safer than others by default. The problem with 'doing things right' is how much trust you put in yourself and your team of developers. I'm of the opinion that most people who feel good enough to handle security actually overlook a lot of it. Have you always checked everything for XSS in all encodings? CSRF? Ever used something like MD5 or SHA to hash passwords? Sent such passwords over e-mail, etc? Those are very basic options and I can tell you that most developers to have worked on the web had a problem with at least one of these at some point or another one. Hell, even gmail had sever CSRF holes at some point that let people randomly inject themselves into your forwarded email adresses.
Security is hard, and stepping clear of the risky line is often a good option if you're not 100% sure of what you're doing. A cook skilled enough can likely prepare a meal while juggling with knives safely, but it's often not necessary to do so, and often not appropriate for everyone to follow that line either.
>> In fact, this is a reason why people like Douglas Crockford prefered to write JSON parsers rather than just evaluating them. It's just not safe enough.
>>
>
> Indeed you are correct, but...
>
> From http://www.json.org/js.html ...
>
> "...The use of eval is indicated when the source is *trusted* and *competent*..."
The *competent* part is the one that worries me. I think most developers (myself included) tend to overestimate their competence when it comes to security.
>
> "...In web applications over XMLHttpRequest, communication is permitted only to the same origin that provide that page, so it is *trusted*. But it *might not be competent*. If the server is not rigorous in its JSON encoding, or if it does not scrupulously validate all of its inputs, then it could deliver invalid JSON text that could be carrying dangerous script..."
>
> So it boils down to the competence of the code on the server. You have to be careful how you construct your pages and javascript. But then, this should *always* be the case.
Yes, agreed. Again, I'm supporting the position of 'why risk it?' not the line of 'it's impossible to be safe!'
>> Plus you have to call the javascript parser and whatnot, which is usually rather slow.
>
> One could send core of the app logic in a static js file then have the eval only making simple calls like "appui.getInvoinces()". That will perform fairly well.
Yes, if the invoices do contain fairly limited and well-defined data that you know can *never* cause a problem.
>
>> The whole idea is pretty bad on the web, where you have to assume that people will actively try to break your stuff and steal data from other users (or you).
>>
>
> That assumption is a bit dramatic. Questions on security cannot be viewed in isolation of application. One of my favorite quotes from Bruce Schneier is applicable here. He was once asked about the possibility of chaos ensuing due to internet security breaches...
>
> "No. Chaos is hard to create, even on the Internet. Here's an example. Go to Amazon.com. Buy a book without using SSL. Watch the total lack of chaos."
The idea is fairly dramatic, but the concept is basically that once someone's got an axe to grind against you or your applications, then someone actively trying to break your stuff is *actually* going to happen. A lax attitude is what made one of our products (at some previous job) vulnerable to Russian hackers who ended up emailing customers with addresses stolen straight out of our databases. When it happens, it's already too late to react.
>
> I don't see how you can canvas the "whole idea" as being bad. It may require adjustments here and there. e.g For particular pages where paranoid security is needed, nothing stops you from doing it differently there. You could crypto what's sent. You could even serve those pages the standard way with static files and SSL if it makes you feel safer.
SSL is protecting you against things like man-in-the-middle attacks. Encryption helps you on other points. There is nothing there regarding problems with application-level security. The whole idea is not bad, but I would certainly want a serious specialist to look over my application if I were to use that trick in many places.
>
>>>
>>> This technique is amazingly powerful.
>>>
>>> So now I only need one generic web page. Think of that.
>>>
>>> Only one page is needed - forever.
>>>
>> This is a problem when it comes to bookmarks, sharing the link with a friend, searchability, browser history, etc. The web wasn't exactly intended to be a stateful thing and you'll have to resort to hacks such as hash-bangs to get around it. I suggest reading Tim Bray's Broken Links to see why that isn't a good solution anyway.
>>
>
> True. But this problem is an age-old general AJAX/dynamic-markup problem. I agree it might be very visible in this case.
>
> However, I've written XULRunner apps with no back buttons -- no need for them with easy-to-navigate UIs. Most Adobe AIR apps I've seen have no browser history. It's made me question: How badly do end-users really need those things? If they do, couldn't we give them better application-specific versions inside our web-app UI?
If I'm using a browser, I'd enjoy being able to use the web. What constitutes a 'very easy to use' application to you might not be the same for everyone. I do remember many flash pages falling pray to the same problem. I think this is mostly a deeply rooted problem in the web where you're piggy-back riding sessions on a protocol that was absolutely not made for that. It sometimes works well enough (I'm thinking of chat applications or even grooveshark here), so it's certainly not black and white, but I figure you know what I mean.
>
>> Plus I'd argue that javascript and Erlang should be kept separate and you shouldn't try to generate one with the other,
>
> Good point. I thought about sending the js in static files and reducing the calls from Erlang to simple one-liners. But also note that the more powerful aspect of this (IMO) is not just sending js, but sending UI elements. Sending blocks of UI to an empty page! How can anyone not like that?
>
Separation of concerns. JS is about behaviours on the page, dynamic content. UI is both HTML (structure) and CSS (presentation). One very simple question I like to ask to sort this out is "would I be able to hire a designer to work on my site without guiding them around too much?" "Could I hire someone to just work on my javascript and HTML without them needing to know anything else?"
If you say no to these, you might have some overlapping domains in what you're doing.
Then again, I'm a fan of really well-separated components in my applications, which is why I like Erlang's processes and OTP applications in the first place :)
Another advantage of keeping things separate is caching -- this is however pretty application and audience specific in terms of needs and requirements.
Joe Armstrong
Ok so "separation of concerns" is good but having different notations for
expressing the concerns
is crazy- to make a web thing that interacts with a server you need to learn
something like
HTML
Javascript
CSS
PHP
MySQL
And to be able to configure Apache and MySQL - other combinations are
possible.
Then you have to split the flow of control to many places.
All of this is crazy madness. There should be *one* notation that is
powerful enough to express all
these things. In the browser is seems sensible to forget about css and html
only use Javascript
The only communication with the browser should be by sending it javascript.
How you generate the javascript is irrelevant - by hand or by program - who
cares. If you make it by
program the chances are that it's right.
Security is orthogonal to this - send encrypted js over the wire and make
sure your key-rings are secure
this is a completely different problem.
/Joe
Vlad Dumitrescu
I've been following the discussion with mixed feelings. I may be simply
confused, but isn't this something that one has been able to do with Flash
since quite a while? So yes, it's cool that it can now be done without
proprietary plugins, but is it new?
2011/2/14 Joe Armstrong <erl...@gmail.com>
> Ok so "separation of concerns" is good but having different notations for
> expressing the concerns
> is crazy- to make a web thing that interacts with a server you need to
> learn
> something like
>
> HTML
> Javascript
> CSS
> PHP
> MySQL
>
> And to be able to configure Apache and MySQL - other combinations are
> possible.
>
> Then you have to split the flow of control to many places.
>
What do you mean by "web thing", there are many species out there? Some are
content-based (regular sites), some are behaviour-based (games), some are
meant for machine consumption (web services), some are somewhere in-between.
Isn't it a matter of using the best tool for the job? If I need searchable
text, I use html. If I need user interaction, I use javascript or flash or
java.
> All of this is crazy madness. There should be *one* notation that is
> powerful enough to express all
> these things. In the browser is seems sensible to forget about css and html
> only use Javascript
> The only communication with the browser should be by sending it javascript.
>
This is reasonable [*] for a behaviour-based "web thing". If one wants/needs
to be able to interact with the "web thing" from lynx, emacs, via a web-bot
or other non-javascript-enabled alternative, then this won't work.
[*] As reasonable as using flash or java applets. No flame wars here.
Then half of the things you enumerated are for the server-side - I think
that's orthogonal to the issue of what kind of data goes to the browser.
PHP/MySql/apache or (for example) erlang/mnesia/yaws are still not
javascript, nor is one or the other easier to configure and setup (unless
one has been working extensively with one or the other).
BTW, how do you mean that the javascript will draw things on the screen?
Wouldn't it be by manipulating the browser document model, building a html
document on the fly? So maybe I don't have to know the html syntax, but I
still need to understand how a html document is structured (which imho is
the more difficult part of the two). An alternative would be to actually
draw pixels, but then it would be easier to skip the browser altogether.
Re-reading my answer, it sounds a negative, but what I mean is to try to
clarify the intent of Joe's idea, maybe I'm missing the point. Personally, I
would go way out to the wild side and suggest replacing the javascript
engine in the browser with an erlang one :-)
best regards,
Vlad
Joe Armstrong
> Hi!
>
> I've been following the discussion with mixed feelings. I may be simply
> confused, but isn't this something that one has been able to do with Flash
> since quite a while? So yes, it's cool that it can now be done without
> proprietary plugins, but is it new?
>
You're right I could have done it with flash, or actionscript, but I didn't
do it.
Javascript provides a abstract interface to the browser. Of course js
actually manipulates the DOM
but the user should not be aware of this. JS routines should manipulate the
DOM set css properties
etc. to present the user with a uniform model of what's underneath.
>
> Re-reading my answer, it sounds a negative, but what I mean is to try to
> clarify the intent of Joe's idea, maybe I'm missing the point. Personally, I
> would go way out to the wild side and suggest replacing the javascript
> engine in the browser with an erlang one :-)
>
That's a wee bit more tricky.
/Joe
>
> best regards,
> Vlad
>
>
Vlad Dumitrescu
> On Mon, Feb 14, 2011 at 10:50 AM, Vlad Dumitrescu <vlad...@gmail.com>wrote:
>
>> I've been following the discussion with mixed feelings. I may be simply
>> confused, but isn't this something that one has been able to do with Flash
>> since quite a while? So yes, it's cool that it can now be done without
>> proprietary plugins, but is it new?
>>
>
> You're right I could have done it with flash, or actionscript, but I didn't
> do it.
>
Ok, I think I understand your position. Thanks for the clarification.
BTW, how do you mean that the javascript will draw things on the screen?
>> Wouldn't it be by manipulating the browser document model, building a html
>> document on the fly? So maybe I don't have to know the html syntax, but I
>> still need to understand how a html document is structured (which imho is
>> the more difficult part of the two). An alternative would be to actually
>> draw pixels, but then it would be easier to skip the browser altogether.
>>
>
> Javascript provides a abstract interface to the browser. Of course js
> actually manipulates the DOM
> but the user should not be aware of this. JS routines should manipulate the
> DOM set css properties
> etc. to present the user with a uniform model of what's underneath.
>
I would say that this is what jQuery, mootools, yui and the other frameworks
try to do - it doesn't seem easy to agree on which way is right :-)
> Re-reading my answer, it sounds a negative, but what I mean is to try to
>> clarify the intent of Joe's idea, maybe I'm missing the point. Personally, I
>> would go way out to the wild side and suggest replacing the javascript
>> engine in the browser with an erlang one :-)
>>
>
> That's a wee bit more tricky.
>
Of course, but it's a wee bit more fun, imo!
regards,
Vlad
Frédéric Trottier-Hébert
>
> Ok so "separation of concerns" is good but having different notations for expressing the concerns
> is crazy- to make a web thing that interacts with a server you need to learn something like
>
> HTML
> Javascript
> CSS
> PHP
> MySQL
>
> And to be able to configure Apache and MySQL - other combinations are possible.
I can agree with that. To have a functional website, you do need to know a lot of different technologies. The web evolved organically and each part of the problem space had its own solution developed over time.
>
> Then you have to split the flow of control to many places.
>
> All of this is crazy madness. There should be *one* notation that is powerful enough to express all
> these things. In the browser is seems sensible to forget about css and html only use Javascript
> The only communication with the browser should be by sending it javascript.
There should, but there isn't. The truth here is that most programmers are awful at design. In any somewhat large setup, your backend programmers, your designers and your integrators (the guys just handling HTML, and CSS, maybe some Javascript) are not necessarily the same person.
Right now the ring of web technologies is divided in a way that makes it somewhat simple to have different people from different background and knowledges to work on different part of your software. It makes sense to have the designer or integrator to be able to change the look and feel of a website without having to play in your code and maybe mess up database queries. Modern template engines in fact try to forbid all kinds of seriously side-effecting code (like DB queries) from happening in the templates.
There should be no worry for your guy working in Javascript that he'd not need to suddenly learn Erlang to be able to debug your application.
Then again, this separation of concerns allows specialists to work on their speciality with more ease. It makes things somewhat simpler in larger organisations, but quite painful for one-man operations. I'll tell you that it makes a lot of sense when you know all of the tools in the toolkit though :)
> How you generate the javascript is irrelevant - by hand or by program - who cares. If you make it by
> program the chances are that it's right.
>
Yes and no. Generated javascript is nearly as old as the language -- many, many .NET apps had that kind of things. Some editors like Dreamweaver could generate JS for you. One of the problem with this is that it was often pure garbage, or it wouldn't work in all browsers uniformly. If you can manage to generate and capture complex behaviours in a compliant manner, all the better. I have myself lost much hope with regards to that though.
> Security is orthogonal to this - send encrypted js over the wire and make sure your key-rings are secure
> this is a completely different problem.
This is only transmission security. Encryption has nothing to do with Cross-Site Scripting (XSS, where some user is able to run arbitrary JS in your page for you and ends up stealing information), Cross Site Request Forgery (CSRF, where the attacker uses the fact your application is forgetting about things like the origin of the queries to hijack the client's session in their place. This is related to Same Origin Policy issues and not easy to handle), SQL injection, overwriting some parameters because you don't fetch them in the right order server-side (see problems with the $_REQUEST variable in PHP), etc.
1. XSS
XSS is, as mentioned above, the ability to run abritrary JS on a page. This is the risky thing with your eval. http://en.wikipedia.org/wiki/Cross-site_scripting contains many details on understanding the related issues. It's not always a simple matter of escaping. Some more advanced attacks even rely on string encoding to make sure your escaping fails. See http://www.governmentsecurity.org/forum/index.php?showtopic=18105.
2. CSRF
CSRF is a tricky thing. Because HTTP doesn't support sessions, over the years, the guys from Netscape (back then) or Opera (or whoever) ended up using Cookies to share data on every query. What happens there is that on every query the browser sends to a server, it also packages the cookies neatly in the headers -- no matter what page you were on when they were sent. The issue here is that the server might not check from what page the call is coming from.
Basically, if twitter had an URL call such as http://twitter.com/tweet/add?message=SomeMessageHere that would automatically add a tweet from your account and I put that link in an image tag on some site, every time you would load that image, you would automatically make a call to the server, your browser sending in your cookies and making it look like YOU actually made that call, even if you didn't know. In this case, the request is especially easy to do because twitter would be using GET parameters to have side-effects on the server. By forcing people into using POST, you can make things harder, but not impossible.
One way to work again POST is using a fake website -- let's say I use learnyousomeerlang.com. On my own site, I'll be putting a fake javascript form inside an iframe (so that the page doesn't refresh when submitted) and have the script automatically send in the POST form. Now I send the link to my trick page over twitter and everyone who clicks on it from there will be guaranteed to have their session open and sending in data. I've in fact used this trick to have the site owner at my old job to close his own admin account on his own website so he could realise the importance of the threat.
How can you solve this one? Well there are a few ways -- for one you could check the HTTP referrer, but that won't work everywhere -- if you expect calls from flash, it doesn't always send these elements of the HTTP header. In the case of HTTPS, depending on how you handle things, the header might not always be sent either so you can't know for sure. Better than that, if I'm using the <img> trick on your own website (on twitter, for twitter users), the domain will be the same, without you being able to check for anything.
The only foolproof way to do this is to use what they call 'tokens': each call you make to the server has to have a unique piece of data that the server knows about that can prove that the call you just made comes from you, but also from your own forms on your own websites. These tokens should have an expiration time and be hidden from plain view, submitted automatically with any form. If you don't have this, your application might not be safe.
This has *nothing* to do with encryption, and everything to do with not understanding the potential threats of the web correctly. It is an application-level issue, much like XSS is. And it's pretty damn important.
3. SQL injection is a different beast, where you do not properly escape the parameters of a request going to the database, letting your run arbitrary DB calls. Erlang with Mnesia doesn't have to worry about that, but Erlang with any SQL has to, even if you end up using QLC (it depends on the library at the back in this case though, and is generally safe enough). http://en.wikipedia.org/wiki/Sql_injection has sufficient details.
4.You have to consider that sometimes these attacks are combined together to be able to really do damage.
I haven't even covered using weak hashing for passwords, bad security policies on cookies, opening files on dynamic paths without filtering the input, etc.
Web application security is not a joke and it's certainly not easy. It's a very serious thing and most developers get it wrong at one point or another. Wordpress got it wrong, Twitter got it wrong, facebook got it wrong, Google got it wrong, and so on, even though they're supposed to be leaders in the field. Most of them got it wrong more than once too. This is why I kind of support a 'paranoid' line of thought.
Edmond Begumisa
> On Mon, Feb 14, 2011 at 10:50 AM, Vlad Dumitrescu
> <vlad...@gmail.com>wrote:
>
>> Hi!
>>
>> I've been following the discussion with mixed feelings. I may be simply
>> confused, but isn't this something that one has been able to do with
>> Flash
>> since quite a while? So yes, it's cool that it can now be done without
>> proprietary plugins, but is it new?
>>
>
> You're right I could have done it with flash, or actionscript, but I
> didn't
> do it.
>
One difference is that you can send not just flash-looking SVG, but you
can also send plain HTML rendered by the browser with no special rendering
libraries or plugins needed.
With XULRunner, I'm sending chunks of XUL from Erlang.
- Edmond -
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
________________________________________________________________
Edmond Begumisa
http://cdn.freedownloadaday.com/wp-content/uploads/2007/03/Songbird.png
Songbird is a winamp-like media player. It's a XULRunner application so
the UI you see there is just mark-up + css + js
Frédéric: you can thus get a graphic designer to do the cosmetics without
him/her needing to learn to program.
Jerome: Notice the very application specific back/forward/refresh buttons
there - in context it's much more useful than a generic browser
back/forward/refresh. Also, the textbox where it says "Library", if you
put your mouse there you get an application specific URI scheme that you
can copy and paste somewhere else. The URI path takes you an item on tree
on the left. Notice also the application-specific bookmarks.
This is currently my preferred way of delivering heavily interactive
"internet applications" (even business applications) -- via a specialised
runtime on the client end and Erlang on the server end. Joe's code gives
me hope that I can finally start using the browser as a client for this
instead without flash or plugins :)
- Edmond -
--
Jerome Martin
Now if we can do this without XULRunner but in a plain browser, like you
said without proprietary plugins, it would be fantastic!
2011/2/14 Edmond Begumisa <ebeg...@hysteria-tech.com>
--
Jérôme Martin
Richard O'Keefe
the physical presentation of code. I wanted to see what this
beautiful code of Joe's was all about, to learn from it.
On screen, the code is green letters on a black background,
but the physical width of the column is too narrow for the text,
so it's full of weird unsemantic line breaks.
On paper, it's a very pale grey on a white background, so the
half hour I spent at home last night trying to decipher it was
totally wasted.
Whatever anyone says about CSS, it _does_ let
you provide different renderings for screen and paper so that
you _can_ have kewl whizzo hey-ma-look-what-I-can-do looks on
the screen and _also_ have something that works on paper. And
for "here is how to write an important kind of code" pages,
isn't it _worth_ committing to paper for study?
I also spent quarter of an hour last night grousing to my wife
about a fancy automatic public convenience, Exeloo brand, that
I'd had occasion to use. It's very like a Javascript web page,
really. You want water? Put your hands where a sensor can
detect them, and it electronically turns a switch to turn the
water on. You don't want it? Take your hands away. You know
how you are supposed to wash your hands for the duration of a
verse of Twinkle Twinkle little star?
Get soap on hands.
Put hands under sensor.
Twinkle twink... where's the rest of the water?
Describe the designer's ancestry for five generations.
Twinkle twi... why does the water keep stopping?
Recite the designer's ancestry for ten generations,
during which dispenser helpfully deposits more soap
on hands, twice.
Twinkle twink...
Recite War and Peace.
Twinkle twi...
A simple tap would have been a *far* better user interface.
Edmond Begumisa
call from my browser. Note that unlike JSON, this is an actual function
and should therefore contain code (so parsing out the code wouldn't make
any sense)...
a) Static version: Stick foo in a file foo.js on the server. Erlang side
streams the file to the browser, which reads it, interprets it and runs it.
b) Eval version: Stick foo in a message. Erlang side streams the function
to the browser, which using eval, reads it, interprets it and runs it.
Why is b) suddenly crossing some threshold of risk that is not equally
inherent to a)? What unacceptable extra risk am I introducing by doing b)
instead of a)? Why does doing b) suddenly require N times more competence
and security conciseness that you don't trust yourself and would much
prefer a) instead?
From what I've read, it appears that you'd never use js at all in your
websites.
- Edmond -
On Mon, 14 Feb 2011 12:39:06 +1100, Frédéric Trottier-Hébert
Edmond Begumisa
call from my browser. Note that unlike JSON, this is an actual function
and should therefore contain code (so parsing out the code wouldn't make
any sense)...
a) Static version: Stick foo in a file foo.js on the server. Erlang side
streams the file to the browser, which reads it, interprets it and runs it.
b) Eval version: Stick foo in a message. Erlang side streams the function
to the browser, which using eval, reads it, interprets it and runs it.
Why is b) suddenly crossing some threshold of risk that is not equally
inherent to a)? What unacceptable extra risk am I introducing by doing b)
instead of a)? Why does doing b) suddenly require N times more competence
and security conciseness that you don't trust yourself and would much
prefer a) instead?
From what I've read, it appears that you'd never use js at all in your
websites. It would always be "too risky."
- Edmond -
On Mon, 14 Feb 2011 12:39:06 +1100, Frédéric Trottier-Hébert
Edmond Begumisa
web-developer generating dynamic content from the client-side (probalby a
good chunk websites uploaded since 2005.)
What I still don't get is why you find the generation of dynamic content
from static js files acceptable while using eval for the same you find
unacceptable. I don't see how XSS, CSRF, SQL Injection, and all the things
you list are more unmanageable from static js that generates content vs
streamed js that generates content.
- Edmond -
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
________________________________________________________________
Edmond Begumisa
serve from my Erlang webserver and call from my browser. Note that unlike
parsing out the code wouldn't make any sense)...
a) Static version: Stick foo in a file foo.js on the server. Erlang side
streams the file to the browser, which reads it, interprets it and runs it.
b) Eval version: Stick foo in a message. Erlang side streams the function
to the browser, which using eval, reads it, interprets it and runs it.
Why is b) suddenly crossing some threshold of risk that is not *equally*
inherent to a)? What unacceptable extra risk am I introducing by doing b)
instead of a)? Why does doing b) require N times more competence and
security conciseness that you suddenly don't trust yourself and would much
prefer a) instead?
From what I've read, it appears that you'd never use js at all in your
websites, let alone js that generates content. It would always be "too
risky."
- Edmond -
On Mon, 14 Feb 2011 12:39:06 +1100, Frédéric Trottier-Hébert
Frédéric Trottier-Hébert
On 2011-02-14, at 15:17 PM, Edmond Begumisa wrote:
> Here's where you loose me: Say I have a js function foo that I want to serve from my Erlang webserver and call from my browser. Note that unlike JSON, this is an actual function and should therefore contain code (so parsing out the code wouldn't make any sense)...
>
> a) Static version: Stick foo in a file foo.js on the server. Erlang side streams the file to the browser, which reads it, interprets it and runs it.
>
> b) Eval version: Stick foo in a message. Erlang side streams the function to the browser, which using eval, reads it, interprets it and runs it.
>
> Why is b) suddenly crossing some threshold of risk that is not *equally* inherent to a)? What unacceptable extra risk am I introducing by doing b) instead of a)? Why does doing b) require N times more competence and security conciseness that you suddenly don't trust yourself and would much prefer a) instead?
>
Not a whole lot of chances for things to go wrong at first. The only way to really have problem would be to dynamically take the filename -- some user-submitted variable, maybe not properly whitelisted. For the future, though? If at some point you get to customise the script a bit with run-time data. If it's simple integers and again whitelisted data, not much of a problem. The time you start taking in strings with random input, it gets more risky.
> From what I've read, it appears that you'd never use js at all in your websites, let alone js that generates content. It would always be "too risky."
No, there are cases that are good for it, but you have to be very careful. In the context of Joe's mail, it was "hey I could replace my whole site by this kind of pattern" which is not the same as say, "inject user-related data in the JS to generate better advertisement" or "have a dynamic way of showing profile pictures" (although there are cleaner ways than streaming JS to an eval function for this).
As a general (and generic) pattern, the eval() in Joe's code worries me. Individual cases can be tested and proven safe on an individual basis without too much trouble.
Frédéric Trottier-Hébert
On 2011-02-14, at 15:17 PM, Edmond Begumisa wrote:
> You've outlined a nice list of the top security concerns of *every* web-developer generating dynamic content from the client-side (probalby a good chunk websites uploaded since 2005.)
>
> What I still don't get is why you find the generation of dynamic content from static js files acceptable while using eval for the same you find unacceptable. I don't see how XSS, CSRF, SQL Injection, and all the things you list are more unmanageable from static js that generates content vs streamed js that generates content.
>
The point of my security list wasn't that much about eval itself rather than contradicting the precise point that 'all you need to do is encrypt javascript'. That's a reductionist and erroneous view.
Generatic dynamic content from static JS files has a few advantages: caching on the browser side, distributing the code via CDNs rather than through your app server, benefiting from JIT if available (rather than calling the compiler/interpreter each time you update content), potential static analysis of code (even through things like JS-Lint). Also a smaller payload on the network and bandwidth -- if you only send in the functions to run the code once rather than on every call, you'll save a lot. In most cases, rendering the page (CSS included), running the JS and transferring the data counts for 90% of the time a user will wait when querying a page. Streaming JS to then evaluate it is going to be terrible for performances on larger scale applications. There's probably more to add to the list, but that's what I can think of in 15-30 seconds.
And things like what I mentioned are not more or less unmanageable from static files (I agree with you there), except for XSS:
XSS is better treated in many cases by things like JS frameworks. If I'm getting the result from some web query into JS, I will receive a neat string, without a chance of it being wrong. If I then push this string through my framework (say JQuery), it'll take care of doing specific escaping of things like element attributes, element content, etc. If I do it dynamically through my applications, chances are much better I'll get the escaping wrong in Erlang (and you need to escape on more levels) than JS, where it can be made on a per-element basis when building the DOM: "create a tag, add the attribute, add another one, add the tag's content, push it" vs. "mix and matches all these strings into hopefully valid JS". This is even truer when you consider hacks such as Google's UTF-7 encounter back in the day. This follows the idea that JS knows JS better.
That's the same reason why you might want things like Erlang handling Erlang parsing, SQL handling its own escaping, etc. If you generate and send JS as one over the wire, you will have to double-check it server-side. Unless you're running with node.js, that's going to be annoying for no good reason.
For CSRF, There is likely no incidence at all. It's a question of shared data between the server and HTML forms. How that data gets there is not really important at first. I could be wrong on that one though and it might be worse than what I expect. For SQL injection, it's all about the last line of defence before sending the data to your DB engine. If you treat it in JS, God have mercy on your application.
But yeah, this little security roundup was again to comment on the 'encrypting your JS' is what you need comments. There's a safety element to using eval, and also performance, clarity and semantic concerns to be had.
Edmond Begumisa
What I still don't get is why you find the client-side generation of
dynamic content from static js files acceptable while using eval for the
same you find completely unacceptable. I struggling to see how XSS, CSRF,
SQL Injection, and all the things you list are more unmanageable from
static js that generates content vs streamed js that generates content.
- Edmond -
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
________________________________________________________________
Edmond Begumisa
What I still don't get is why you find the client-side generation of
dynamic content from static js files acceptable while using eval for the
same you find completely unacceptable. I don't see how XSS, CSRF, SQL
Injection, and all the things you list are more unmanageable from static
js that generates content vs streamed js that generates content.
- Edmond -
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
________________________________________________________________
Edmond Begumisa
You've outlined a nice list of the top security concerns of *every*
web-developer generating dynamic content on the client-side (probalby a
good chunk websites uploaded since 2005.) Things everyone should be aware
of but IMO some should be more concerned than others.
What I still don't get is why you find the client-side generation of
dynamic content from static js files acceptable while using eval for the
same you find completely unacceptable. I struggling to see how XSS, CSRF,
SQL Injection, and all the things you list are more manageable from static
js that generates content vs streamed js that generates content*
- Edmond -
PS: *Joe is not the only one that's doing this BTW -- it's becoming
increasingly common for AJAXers.
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
________________________________________________________________
Edmond Begumisa
web-developer generating dynamic content on the client-side (probalby a
good chunk websites uploaded since 2005.)
What I still don't get is why you find the client-side generation of
dynamic content from static js files acceptable while using eval for the
same you find completely unacceptable. I struggling to see how XSS, CSRF,
SQL Injection, and all the things you list are more manageable from static
js that generates content vs streamed js that generates content*
- Edmond -
PS: Joe is not the only one that's doing this BTW -- it's becoming
increasingly common for AJAXers.
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
________________________________________________________________
Frédéric Trottier-Hébert
On 2011-02-14, at 15:37 PM, Frédéric Trottier-Hébert wrote:
>
> Generatic dynamic content from static JS files has a few advantages: ...
>
>
This should obviously read "Using purely static JS files has a few advantages". Sorry for the messy wording.
Ryan Zezeski
>
>
> This has *nothing* to do with encryption, and everything to do with not
> understanding the potential threats of the web correctly. It is an
> application-level issue, much like XSS is. And it's pretty damn important.
>
>
You make a very impressive summarization of web security problems, but there
are many apps where these problems go away because they simply, don't,
matter. For example, how about building a new interface to the appmon
application using this with Raphael or Protovis? Do I really need to worry
about all this crap? I'd argue, no. If I'm using appmon it's behind a
firewall and I trust the user to be competent. Just like 'rm -f 'in unix.
Plus, who gives a sh*t (pardon my French) about a hijacked session when
someone else just cracked your gateway because of default passwords. My
point being, there are so many doors, and this focuses on one. Getting it
right is very hard, and thus very costly, and that's why I think the first
thing to ask is "Does this even need to be secure?" Given enough time,
anything can be had.
If nothing else Joe has stumbled upon a way to rapidly produce nice-looking
and feature rich GUIs in Erlang, and yet it seems everyone has somehow
missed this point and focused on security and support and whatnot. I'm glad
there are Joe's in the world, otherwise it would be a pretty boring place.
Don't let perfect get in the way of good enough. As Joe said, life is too
[expletive deleted] short.
-Ryan
Frédéric Trottier-Hébert
On 2011-02-14, at 18:38 PM, Ryan Zezeski wrote:
>
>
> 2011/2/14 Frédéric Trottier-Hébert <fred....@erlang-solutions.com>
>
> This has *nothing* to do with encryption, and everything to do with not understanding the potential threats of the web correctly. It is an application-level issue, much like XSS is. And it's pretty damn important.
>
>
> Except when it isn't.
>
> You make a very impressive summarization of web security problems, but there are many apps where these problems go away because they simply, don't, matter. For example, how about building a new interface to the appmon application using this with Raphael or Protovis? Do I really need to worry about all this crap? I'd argue, no. If I'm using appmon it's behind a firewall and I trust the user to be competent. Just like 'rm -f 'in unix. Plus, who gives a sh*t (pardon my French) about a hijacked session when someone else just cracked your gateway because of default passwords. My point being, there are so many doors, and this focuses on one. Getting it right is very hard, and thus very costly, and that's why I think the first thing to ask is "Does this even need to be secure?" Given enough time, anything can be had.
I won't argue with this. You're right on this. Again I'll repeat myself by saying that as a general (and generic) pattern, the eval() in Joe's code worries me. Individual cases can be tested and proven safe on an individual basis without too much trouble.
>
> If nothing else Joe has stumbled upon a way to rapidly produce nice-looking and feature rich GUIs in Erlang, and yet it seems everyone has somehow missed this point and focused on security and support and whatnot. I'm glad there are Joe's in the world, otherwise it would be a pretty boring place.
>
> Don't let perfect get in the way of good enough. As Joe said, life is too [expletive deleted] short.
>
> -Ryan
There is nothing wrong with what Joe is doing. There is nothing wrong with exploration. However, things already exist in Javascript and on the web in general and we shouldn't reinvent the wheel all the time because we might be coming from a different domain or we don't have the same experience as real web developers have.
You only have to look into what people are doing with node.js. Just look at this example: http://www.screentoaster.com/watch/stUE5XQEVMRFtXQ1xVXFlYVlZX/serrano_session_sharing
These guys are doing session sharing over the browser (including mouse movement and everything).
You can also look at the nodeknockout as a whole: http://nodeknockout.com/teams (some links are now dead). These include multiplayer browser games, chat systems, drawing applications, charts, etc. Dreamers and innovators are at work on the web already, they're just not necessarily doing it with Erlang.
They are doing that kind of explorative nice-looking innovation we all find awesome (even if they likely don't do it 100% safe either).
I'm sorry if I sounded patronising or just bossy around this discussion, it's just that there are fantastic ideas to be inspired from in what is done by these programmers all the time. They usually do it cleaner, safer and smarter. At least on the client side.
If people can learn from Erlang when it comes to heavily concurrent and fault-tolerant applications, Erlang programmers can certainly learn from other domains when the developers working there are more specialised at what they do than we could be as a community in here.
Ryan Zezeski
Thanks for the links, very cool stuff.
I think reinventing the wheel is a good thing. The thing is, it's not
really reinventing, but rather constant refinement. I mean, lets take the
literal wheel, you know the rubber tires found on cars. They've been around
for a long time, and you would think everything that can be said or done
with a wheel has already been done, yet tires improve every year to get
maximal traction in the dry while maintaining grip in the wet. While Joe's
discovery, on the whole, is not revolutionary, it is a twist on common
practice. At least, in the small world of Erlang GUIs.
I think your point about learning from others is _spot_ on, and I honestly
think that's just what has happened here. Joe took a look at websockets, a
look at Erlang's msg passing, and went "What happens when I put the two
together?" Bam! New way to think about interfacing Erlang with the
browser.
At least, that's what I see, and I think it's pretty cool. I look forward
to seeing what comes out of this. I only hope that I may, for once,
contribute something myself, rather than just talk about it :)
-Ryan
Richard O'Keefe
including one hosted at this very University (but not a science department).
In both cases I was seriously searching for information, on behalf of
someone else having a fairly pressing need for it.
In both cases, what *could* have been a simple static page linking to a
bunch of other static pages went through an unholy mix of Javascript
and SQL, and ended up spewing buggy SQL over my screen instead of useful
information.
Heck, one of these was even a cross-site-scripting bug.
(Page here has Javascript that creates 'whatever?at=code, accessed at
<some date>' in a link, user clicks on harmless text of link, server
at other end trusts blindly, crashy time.)
Javascript *can* be used competently to do wonderful things.
But not for the 2-10% of visitors using browsers with Javascript
support missing or switched off.
And WebSockets will be used competently to do wonderful things.
But not for the [27% to 70% depending on who you ask] of visitors
using Internet Explorer.
We shouldn't fall into the trap of "one page sites".
(See http://catless.ncl.ac.uk/Risks/26.34.html#subj8 )
If you are providing information through the Web and it does not
get communicated, you have *failed*, no matter how wonderful the
experience might have been had it worked.
Ryan Zezeski
> I just ran into trouble with a couple of "smart" pages today,
> including one hosted at this very University (but not a science
> department).
> In both cases I was seriously searching for information, on behalf of
> someone else having a fairly pressing need for it.
> In both cases, what *could* have been a simple static page linking to a
> bunch of other static pages went through an unholy mix of Javascript
> and SQL, and ended up spewing buggy SQL over my screen instead of useful
> information.
>
> Heck, one of these was even a cross-site-scripting bug.
> (Page here has Javascript that creates 'whatever?at=code, accessed at
> <some date>' in a link, user clicks on harmless text of link, server
> at other end trusts blindly, crashy time.)
>
> Javascript *can* be used competently to do wonderful things.
> But not for the 2-10% of visitors using browsers with Javascript
> support missing or switched off.
> And WebSockets will be used competently to do wonderful things.
> But not for the [27% to 70% depending on who you ask] of visitors
> using Internet Explorer.
>
> We shouldn't fall into the trap of "one page sites".
> (See http://catless.ncl.ac.uk/Risks/26.34.html#subj8 )
>
> If you are providing information through the Web and it does not
> get communicated, you have *failed*, no matter how wonderful the
> experience might have been had it worked.
>
>
Sure, but not all applications have this requirement. I'll go right back to
my appmon example. appmon is a great little utility in the Erlang world for
visualizing your process graph, but it suffers from a absolutely horrible UI
(at least in OS X). Using Joe's work to make a new UI would be really cool
and useful to those that have the appropriate support, and it wouldn't hurt
anyone that doesn't. Furthermore, this would not be something exposed to
the web, and even if it was it's not something anyone expects to search. Of
course, if you really wanted to, you could use HTTP's content negotiation to
handle different representations.
For the very reasons that everything should _not_ be a "one page site" I
would also argue not everything needs to be a "multi-page site." There is
room for both. Think about video games, they perfectly fit the model of a
"one page site." How about google maps, google docs, etc? If the historic
web was so good why all the fuss over Ajax/DHTML, and more recently HTML5
stuff like canvas, video, webgl, 3D CSS, websockets, and the like? For
certain things, namely "applications," people want an experience that the
current web doesn't give them. Personally, I'm all for it.
-Ryan
Joe Armstrong
It seems to me you can either secure the pipe (ssh) or secure the things in
the
pipe (ie the content).
I'd prefer to secure the content since I can do this with my own code - rsa
in erlang
boils down to A^C mod N with Erlang and bignums this is trivial - I trust my
own code.
Now suppose I've gotten as far as decrypting and authenticating the
encrypted blog
of code, dare I *run* the code - this boils down to trust and is a
non-technical issue.
Do I trust the NSA not to have put a back-door in ssh? - no. Do I trust my
rsa code - yes.
People happly click on "install this code" "run this code" buttons because
they trust
the source.
If I still want to run the code anyway, despite the fact I mistrust the
source I can run it
on a sand boxed machine that is not connected to anything.
But again do I trust the USB memroy (or whatever) that I used to transfer
the program?
What If somebody installs a keyboard logger in the firmware of my keyboard?
It's all about trust
/Joe
2011/2/15 Frédéric Trottier-Hébert <fred....@erlang-solutions.com>
Jesper Louis Andersen
> It seems to me you can either secure the pipe (ssh) or secure the things in
> the
> pipe (ie the content).
The right thing to do is obviously to protect the content. The pipe is
not what you execute, the content is. So there is no security in
protecting the content. Specifically we want integrity of the content
more than a confidentiality: We want a seal-of-approval on the code we
run so we are sure where it originates from. It is a question of trust
- either that the originator is the sole owner of the signature (in a
public/secret key cryptosystem), or that the Least Precondition
verifier locally is correct (in the case of proof-carrying-code).
> I'd prefer to secure the content since I can do this with my own code - rsa
> in erlang
> boils down to A^C mod N with Erlang and bignums this is trivial - I trust my
> own code.
Beware the side-channel attack. Crypto done right, mathematically, is
not secure anymore. You need certain functions to take the same amount
of time always, or you can gleam off bits from information theoretic
attacks.
> Now suppose I've gotten as far as decrypting and authenticating the
> encrypted blog
> of code, dare I *run* the code - this boils down to trust and is a
> non-technical issue.
There is another important issue: you have the seal-of-approval on the
code, but you need to trust that the code does not, by intention or
mistake, pull in other code to execute. Javascript is bad because
there is no security model on it at all and you can't trust all code
by transitive closure. Most modern browser attacks revolve around this
idea in some way: inject javascript into a place and get it executed
by the browser by "mistake". It is unfortunate that
proof-carrying-code (see
http://en.wikipedia.org/wiki/Proof-carrying_code ) did not take off
more.
On the other hand: The situation currently is bad, but it will
probably not get worse with JS+signatures.
As an aside, I note that Alan Kay once said that the web should have
been an executable program from the start, rather than HTML. That way,
you could have built an HTML renderer if you so desired, but you could
also have done different things at the same time. PDF rendering would
have been easy, Flash would have been trivial and so on. I don't think
this idea is so far off the vision.
--
J.
Alceste Scalas
I just wanted to quote a few excerpts of Frédéric's emails,
because I completely agree with them (see below).
I think that if you want to develop good web applications, then
you need to know HTTP, and design the client-server interaction
around its principles and its stateless nature, i.e. following a
RESTful approach [1]. The user interface elements (HTML, CSS,
Javascript...) just come on top of it, and usually require
dedicated knowledge and experience in order to get them right.
I also think that everything which tries to circumvent the
HTTP/web architecture in order to "help" the programmer reach
from the server-side to the client-side is doomed to fail,
especially when scalability becomes an issue, either on the
developer side (more than a few programmers are expected to
work on a project) or the client side (more than a few users
are expected to interact with a web application).
Just to make a few examples, I'm referring to "cool" stuff like
continuation-based web frameworks from the Smalltalk/Lisp/Scheme
world, or automatic server-side Javascript generation (which
reminds me of automatic SQL schema or query generation: it
usually sucks, and thus it is hated and avoided by developers
and DB admins who are actually proficient in SQL).
IMHO, it is an error to focus on toy applications (and/or
single-user applications, like the Appmon) and assume that an
useful solution (at least from the POV of a single developer,
or a small group) could "scale up" to real-world application
development. When projects start this way, and they grow,
they usually need to be thrown away and rewritten ground-up.
Here's a famous example: a few years ago, Reddit was considered
a proof that "Lisp works", because it was a prominent web
application written in Lisp. When the user base grew, Reddit
was completely rewritten in Python [2]. This was considered a
"betrayal" by the Lisp community, and thus several developers
started to write "a Reddit clone in just 20 lines of Lisp".
Those were interesting exercises, but they didn't address any
of the problems of the first Reddit: they didn't scale,
didn't support templating, didn't support l10n or l18n,
dind't offer battleground-tested libraries...
From my experience, I can say that it's much better to
start from the complete and realistic scenario of a "real" web
application, and see if (and how) an approach that works at
this level could "scale down" and become "easy" in single-user,
single-developer scenarios. IMHO it's also the most effective
way to propose Erlang as a sound alternative for web
development.
References:
[1] http://en.wikipedia.org/wiki/Representational_State_Transfer
[2] http://blog.reddit.com/2005/12/on-lisp.html
On Mon, 14 Feb 2011 07:43:57 -0500, Frédéric Trottier-Hébert
<fred....@erlang-solutions.com> wrote:
> The truth here is that most programmers are awful at design. In any
> somewhat large setup, your backend programmers, your designers and
> your integrators (the guys just handling HTML, and CSS, maybe some
> Javascript) are not necessarily the same person.
>
> Right now the ring of web technologies is divided in a way that makes
> it somewhat simple to have different people from different background
> and knowledges to work on different part of your software. It makes
> sense to have the designer or integrator to be able to change the
> look
> and feel of a website without having to play in your code and maybe
> mess up database queries. Modern template engines in fact try to
> forbid all kinds of seriously side-effecting code (like DB queries)
> from happening in the templates.
>
> There should be no worry for your guy working in Javascript that he'd
> not need to suddenly learn Erlang to be able to debug your
> application.
>
> Then again, this separation of concerns allows specialists to work on
> their speciality with more ease. It makes things somewhat simpler in
> larger organisations, but quite painful for one-man operations. I'll
> tell you that it makes a lot of sense when you know all of the tools
> in the toolkit though :)
On Mon, 14 Feb 2011 20:50:11 -0500, Frédéric Trottier-Hébert
<fred....@erlang-solutions.com> wrote:
> There is nothing wrong with what Joe is doing. There is nothing wrong
> with exploration. However, things already exist in Javascript and on
> the web in general and we shouldn't reinvent the wheel all the time
> because we might be coming from a different domain or we don't have
> the same experience as real web developers have.
--
Alceste Scalas <alc...@muvara.org>
Alain O'Dea
page. HTTP does not necessarily apply. SEBG could be used in the context
of a larger RESTful application to provide lighter-weight incremental
updates. A RESTful server would still maintain the up to date
representations for subsequent full fetches, but user agents would be kept
up to date with a lightweight stream. They are complementary, not
conflicting :)
An Erlang/OTP app built with one SEBG process per user should have no
inherent scalability problems. Any bottlenecks that exist should be fixable
on the server-side by applying OTP principles.
Again SEBG is complementary to REST, not in conflict with it just as
Websockets are complementary to HTTP.
Robert Virding
----- "Jesper Louis Andersen" <jesper.lou...@gmail.com> wrote:
> Beware the side-channel attack. Crypto done right, mathematically, is
> not secure anymore. You need certain functions to take the same
> amount
> of time always, or you can gleam off bits from information theoretic
> attacks.
This reminds of something from the annals of history, from the golden age of computing. Apparently on a Dec-10 you could tell how many of the characters in an attempted password were correct by the time it took for the system to return that it was an illegal password. Or so the legends say.
Robert
--
Robert Virding, Erlang Solutions Ltd.
Yurii Rashkovskii
Bengt Kleberg
Google does not find anything about this so from memory:
The password had to be stored on two different virtual memory pages. You
started with the first character on page one, and the rest on page 2.
The library function that checked if this was the right password would
return faster if the character on page one was correct. After trying all
possible first characters you would then know the correct character.
Then put both the correct first character and another (probably wrong)
character one virtual memory page one, the rest on page two. Repeat.
bengt
On Tue, 2011-02-15 at 21:59 +0100, Robert Virding wrote:
> ----- "Jesper Louis Andersen" <jesper.lou...@gmail.com> wrote:
>
> > Beware the side-channel attack. Crypto done right, mathematically, is
> > not secure anymore. You need certain functions to take the same
> > amount
> > of time always, or you can gleam off bits from information theoretic
> > attacks.
>
> This reminds of something from the annals of history, from the golden age of computing. Apparently on a Dec-10 you could tell how many of the characters in an attempted password were correct by the time it took for the system to return that it was an illegal password. Or so the legends say.
>
> Robert
>
Håkan Huss
page was non-existant and checked whether or not it had been created
after the call. See for instance
http://groups.google.com/group/alt.folklore.computers/msg/00d243bb0caa9f69?dmode=source&pli=1
Anyway, no-one stores passwords in plain text these days, right? (I
always check the "forgot password" mechanism of web sites that I sign
up to. If they can send me my password I tend to be wary of their
security. If they offer to reset my password, they at least got one
thing right...).
Regards,
/Håkan
Joe Armstrong
"look-at-this-when-you-get-time"
list - thanks for the tip
/Joe
On Wed, Feb 16, 2011 at 1:00 AM, Yurii Rashkovskii <yra...@gmail.com> wrote:
> Wow, what a thread :) I just had a thought — wouldn't it be more productive
> to do something like, for example, a port of socket.io (http://socket.io/)
> server to Erlang? (with misultin/mochiweb/yaws/.. supported) This way we can
> get WebSockets API pretty much regardless of the browser in question; it can
> also
> help luring some JS developers into Erlang since they'll be able to reuse
> their frontends for their Node.js services that don't scale.
>
> Basically, socket.io gives you that WebSocket API but fallbacks through a
> number of other methods until it finds one that works with a visitor's
> browser.
>
> I started some bits of it at https://github.com/yrashk/socket.io-erlangbut I am ultimately not able to keep up with all projects I am interested.
> So if anybody is interested, please let me know.
>
>
>
Robert Raschke
protect, and who from, no? Would you refrain from using a forum if they sent
you your password?
When talking about security there is no such thing as a general approach.
You need to look at the particular threats and risks to figure out how much
and how good your security must be.
This threat and risk analysis appears to be common practice in other fields,
but tends to be ignored in the IT world, where a whole industry revolves
around some magical maximum security.
Robby
Håkan Huss
However, I will be restrictive on such a site when entering
information in forms and such.
/Håkan
Edmond Begumisa
I think if you take a closer look at _how_ Joe is actually using eval,
you'll find you need not be so alarmed by it. You'll find the *same*
security issues all web-programmers are used to. You'll find you can deal
with them in the *same* ways.
See comments inline...
But it gets more risky in *both* cases, in equal magnitude. Doing a)
instead of b) won't protect if you start "taking in strings with random
input". Let's analyse how it "gets more risky" and then compare it to
Joe's code...
Say programmer Helga stores a value into a Key-Value DB carelessly from
the browser like so...
/* NB: We cannot trust ::textContent here coz we're saving it for later
use */
var f = getElementById('f').textContent; // Say f = 'x + y'
kvSaveToServer("f", f); // Server stores f verbatim
Then later, on another page, she tries a little calc UI...
/* NB: We can reasonably trust ::textContent here coz it's only used on
the same page */
var x = getElementById('x').textContent; // Trusted
var y = getElementById('y').textContent; // Trusted
var f = kvReadFromServer('f'); // Untrusted (f = 'x + y', but our
saving was dubious)
var z = eval(f);
The warning from Mozilla and others concerning eval here is clear and
every web-developer is aware of it: Helga can't really be sure what f
contains because it's code that comes from someone other than her (e.g. a
malicious user when it was saved.) Hence the general advice to avoid
eval(). But if she *is* sure what f contains because coz she damn-well
wrote it like so...
var x = getElementById('x').textContent; // Trusted
var y = getElementById('y').textContent; // Trusted
var z = eval('x + y'); // Trusted (Evaling code we've written)
Then this is no different from...
var x = getElementById('x').textContent; // Trusted
var y = getElementById('y').textContent; // Trusted
var z = x + y; // Trusted (code we've written)
It makes no difference whether the *static* 'x + y' comes streaming in
from the server and ran as eval('x + y') in the former case, or if it
comes locally from a js file that also comes from the server in the latter
case (there might be a scope issue in the former but we'll get to that.)
Now let's take it further and load the arbitrary strings you mention
within the eval'ed code (closer to what Joe's doing but not quite). I
believe this is the code that alarms you...
/* Assume keys x and y were saved earlier using careless
getElementById('x').textContent verbatim */
var x = kvReadFromServer('x'); // Untrusted
var y = kvReadFromServer('y'); // Untrusted
var z = eval('x + y'); // **Looks trusted but is not**
However, and here's half the point I was making, this is the same as...
var x = kvReadFromServer('x'); // Untrusted
var y = kvReadFromServer('y'); // Untrusted
var z = x + y; // **Looks trusted but is not**
So the non-eval'ed code here is just as risky and deceptive as the eval'ed
code. Helga avoiding eval on *code that she wrote* (former) and using a
non-eval'ed version *that she wrote* (latter) will not protect her. Using
the former is not _increasing_ her susceptibility to code injections from
the "random strings" coming from others in the database. She's equally
screwed either way by her carelessness with the DB.
**BUT**, and this is key, that's not even what Joe's doing! Basically, Joe
is taking this supposedly safer latter static version, and eval'ing it
like so...
var c = "var x = kvReadFromServer('x'); // Untrusted" +
"var y = kvReadFromServer('y'); // Untrusted" +
"var z = x + y; // **Looks trusted but is not**";
eval(c);
Joe is taking the code that you'd prefer to see in a static file, and just
eval'ing it. Yet, and this is the second half of my point, any
carelessness in either is equal. I find it hard to be petrified by the
lower version while being more comfortable with the one above because it's
somehow "less risky." The real problem is the untrusted values.
AFIAK (at least with the Mozilla code-base of which I'm fairly familiar
with), using eval like this *introduces* only one *new* security concern:
3rd party js can see the scope at which eval was evoked. And if you're
using 3rd party js that you can't trust, you're probalby screwed anyway :)
All other security concerns are the *same* in both versions.
Now, getting more realistic, he'll obviously want to do the reading server
side, thus replacing kvReadFromServer('x') with <<"'", Val, "'">> cat'ed
into the eval'ed string before it's delivered. So he now gets something
like...
var c = "var x = 'valfromuser'; // Untrusted" +
"var y = 'valfromuser'); // Untrusted" +
"var z = x + y; // **Looks trusted but is not**";
eval(c);
AHA! You raised a valid concern about how doing the proper escaping would
be tricky to get correct, esp from Erlang, and jquery does this sort of
thing better. But nothing stops Joe from making calls to JQuery with the
code he pushes (actually, he does use JQuery), or from using the native
JSON parser...
var c = "var x = JSON.Parse('valfromuser'); // Trusted" +
"var y = JSON.Parse('valfromuser'); // Trusted" +
"var z = x + y; // Trusted";
eval(c);
- Edmond -
Jesper Louis Andersen
> Anyway, no-one stores passwords in plain text these days, right? (I
> always check the "forgot password" mechanism of web sites that I sign
> up to. If they can send me my password I tend to be wary of their
> security. If they offer to reset my password, they at least got one
> thing right...).
It doesn't really matter. If they simply decided to run SHA1 on your
password, they can't give you back your password, but it is not secure
by any means. It can be attacked by brute force and by rainbow tables
easily. Rainbow tables can be defeated by adding a salt, but the real
solution is to use a key derivation function where a check is forced
to take computing time and or memory, see PBKDF2, bcrypt and scrypt.
The only real solution is to derive a random password for each site
that wants one. You can't trust others to get crypto right, as most
often they wont.
--
J.
Frédéric Trottier-Hébert
On 2011-02-16, at 07:48 AM, Edmond Begumisa wrote:
> On Security:
>
> I think if you take a closer look at _how_ Joe is actually using eval, you'll find you need not be so alarmed by it. You'll find the *same* security issues all web-programmers are used to. You'll find you can deal with them in the *same* ways.
>
> See comments inline...
>
Nothing to argue there. Again, specific use cases can be safe. My biggest worry has to do with escaping and how to do it properly. There's an inherent risk in using dynamic data in all cases. The last way (with 'eval(c)') is till safe when you know exactly what data you get in.
To quote myself another time, As a general (and generic) pattern, the eval() in Joe's code worries me. Individual cases can be tested and proven safe on an individual basis without too much trouble.
>
> Now, getting more realistic, he'll obviously want to do the reading server side, thus replacing kvReadFromServer('x') with <<"'", Val, "'">> cat'ed into the eval'ed string before it's delivered. So he now gets something like...
>
> var c = "var x = 'valfromuser'; // Untrusted" +
> "var y = 'valfromuser'); // Untrusted" +
> "var z = x + y; // **Looks trusted but is not**";
> eval(c);
>
> AHA! You raised a valid concern about how doing the proper escaping would be tricky to get correct, esp from Erlang, and jquery does this sort of thing better. But nothing stops Joe from making calls to JQuery with the code he pushes (actually, he does use JQuery), or from using the native JSON parser...
>
> var c = "var x = JSON.Parse('valfromuser'); // Trusted" +
> "var y = JSON.Parse('valfromuser'); // Trusted" +
> "var z = x + y; // Trusted";
> eval(c);
>
> - Edmond -
>
>
> --
> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
Yes, hard-coding all the calls to jQuery could help. But only slightly so. You now need to escape on many levels:
- Is the user submitting data that could break HTML? If so, is the user breaking it with:
- invalid tags?
- invalid tag attributes?
- invalid tag content?
- are URLs valid?
The rules for an HTML element's attributes are not the same that the content of the tag. How deep does the nesting go? Do you handle all cases? That's where you want your framework to act and you could call it from code generated in Erlang, yes.
- Is the user submitting data that could break Javascript itself?
Right now this might be the biggest (if not one of the only) risk of using Joe's method if you do take care about making sure everything goes through a framework. Your 'valfromuser' here could be the culprit and you need to have a pass over your data in Erlang to make sure you won't mess it up when inputting it into the code to eval.
Escaping JS is somewhat conceptually simple, but unless you know all the tricks, nothing guarantees you won't be caught by a thing like Google's UTF-7 hack, or some other weird escaping technique.
The point here is that you might be trying to escape and handle data on the backend, while it will run on the frontend. It is generally just saner to send it over to JS in these cases. Again, JS knows JS better and you'll generally be safer.
You have to know, do all browsers handle JS the same? Can something break things in one browsers but not the others? You might already know that older versions of IE do accept things like backticks (`) as a valid quotation character or that certain HTML entities can do the job fine while your browser of choice might not accept them the same way (I assume your browser of choice is not some old IE). Are you sure you're going to handle these cases correctly all of the time when inputting user data as variables or as strings in your code?
I know I wouldn't be so sure myself. If different browsers handle things differently (and that might also depend on the doctype of your page, so there's no foolproof solution from the backend's point of view without additional user knowledge), can you again give me a guarantee that your code is safe? And if it's safe today, will it still be safe tomorrow? Will it be safe on all of my pages, even if my frontend guy ends up changing things?
I'm pretty sure that it's less effort to just keep your JS framework of choice up to date and focus on making sure the data won't break HTML once it's been used with JS, but that might just be me.
- Are you going to mess up the content your users provided by trying to escape it?
This isn't a security concern, but an applicative one. Erlang's modules are in latin-1 by default. Erlang can handle UTF-8. What's your webpage in? Is the input and the output in the same encoding? Will manipulating and escaping your data in Erlang risk messing it up? This is somewhat easy to solve, but yeah. Just another concern.
Ultimately, it's your application and I won't be trying to break into your office to scold you for using things I don't like. You know the kind of payload your site could represent. You might or might not fully know how safe your page is. You make a judgement call, but I felt like voicing my worries over this thread because a lot of people here didn't seem to worry at all (which worried me more!)
Edmond Begumisa
I'd like to think this is the kind of community where someone can say ..
"Hey, I've got this whacky idea. It's early stages but I think I'm onto
something."
Then in *addition* to the community saying "We'll, there are problems x, y
and z you may have overlooked", the community *also* says "Possibly p, q,
and r might help with these."
I think if you look at what Joe's doing, it can be expanded on to get
something very useful. Let's take some of everyone's concerns and actually
*try* give Joe some advice on how they might be addressed.
I've started inline, hopefully others can add some input...
On Tue, 15 Feb 2011 07:37:44 +1100, Frédéric Trottier-Hébert
<fred....@erlang-solutions.com> wrote:
>
>
> On 2011-02-14, at 15:17 PM, Edmond Begumisa wrote:
>
>> You've outlined a nice list of the top security concerns of *every*
>> web-developer generating dynamic content from the client-side (probalby
>> a good chunk websites uploaded since 2005.)
>>
>> What I still don't get is why you find the generation of dynamic
>> content from static js files acceptable while using eval for the same
>> you find unacceptable. I don't see how XSS, CSRF, SQL Injection, and
>> all the things you list are more unmanageable from static js that
>> generates content vs streamed js that generates content.
>>
>
> The point of my security list wasn't that much about eval itself rather
> than contradicting the precise point that 'all you need to do is encrypt
> javascript'. That's a reductionist and erroneous view.
>
> Generatic dynamic content from static JS files has a few advantages:
> caching on the browser side, distributing the code via CDNs rather than
> through your app server,
How do AJAX sites solve this? One way is to break the "one page app" into
a few pages. Maybe he could introduce a window concept that maps to a
page...
Pid ! {new, window(...)}
.. work .. work...
Pid ! {new, window(...)}
There's a start.
> benefiting from JIT if available (rather than calling the
> compiler/interpreter each time you update content), potential static
> analysis of code (even through things like JS-Lint). Also a smaller
> payload on the network and bandwidth -- if you only send in the
> functions to run the code once rather than on every call, you'll save a
> lot.
I suggested before sending parts of your app in ordinary static js files,
then call the functions as libraries. The benefits above will start to be
felt.
Hmmm.. I wonder what Nitrogen does, they might have some tricks.
> In most cases, rendering the page (CSS included), running the JS and
> transferring the data counts for 90% of the time a user will wait when
> querying a page. Streaming JS to then evaluate it is going to be
> terrible for performances on larger scale applications.
I dunno about this. Might be a bit of a blanket statement. Wasn't the very
reason AJAX came around to INCREASE performance of larger scale
applications *precisely* by streaming markup and JS for evaluation and
rendering on-demand rather than all-at-once because in the latter case you
normally send more than is actually needed?
> There's probably more to add to the list, but that's what I can think
> of in 15-30 seconds.
Likewise, I could probably add more but these are the ideas for improving
Joe's concept that I can come up with in 15-30 seconds ;) Others can pitch
in.
>
> And things like what I mentioned are not more or less unmanageable from
> static files (I agree with you there), except for XSS:
>
> XSS is better treated in many cases by things like JS frameworks. If I'm
> getting the result from some web query into JS, I will receive a neat
> string, without a chance of it being wrong. If I then push this string
> through my framework (say JQuery), it'll take care of doing specific
> escaping of things like element attributes, element content, etc.
What stopping you?
As I illustrated in the previous mail on security, you can call JQuery
from code that's being run in eval too! Joe's calling everything from
JQuery to SVG libraries!
> If I do it dynamically through my applications, chances are much better
> I'll get the escaping wrong in Erlang (and you need to escape on more
> levels) than JS, where it can be made on a per-element basis when
> building the DOM: "create a tag, add the attribute, add another one, add
> the tag's content, push it" vs. "mix and matches all these strings into
> hopefully valid JS". This is even truer when you consider hacks such as
> Google's UTF-7 encounter back in the day. This follows the idea that JS
> knows JS better.
>
Right, let's convert that statement from a critisism into a really good
piece of advice:
Joe: That code where you're manipulating the DOM, where you do
".insertElement" and such, know what? Better do that via JQuery instead.
> That's the same reason why you might want things like Erlang handling
> Erlang parsing, SQL handling its own escaping, etc. If you generate and
> send JS as one over the wire, you will have to double-check it
> server-side.
How about adding templating? (I suggested this to Joe off-list)...
One could possibly use leex/yecc to compile say "std.tpl" file and access
it from Erlang code like so...
Pid ! {insert, std.grid(List)}
which might use the content of std.tpl to produce...
Pid ! {insert, <<"<table>blah blah</table>">>) % Or
<<"createElement(blah)">>
which might then stream to the client...
"document.body.innerHTML("<table>blah blah</table>") /* Or
createElement/or jQuery insert call */
The nice thing with this is, std.tpl could have versions.
Joe likes SVG: so his std.grid(List) might produce some fancy SVG code.
I like XUL: so my std.grid(List) might produce "<grid>blah</grid>"
Templating might be extendable, so you might have an app specific my.tpl
which extends on std.tpl...
so when you: Pid ! {new, my.wnd(..)}
the client gets a new page with stylesheet references, script tags, etc,
specified in the my.tpl
With all the great minds on this list, surely suggestions could be made to
turn this early one-paged code into something more and more useful??
- Edmond -
Joe Armstrong
Why? - the only reason I can think of is cross-browser compatibility. At the
moment I don't really
like libraries since I want to see whats going on as near to the bottom
level as I can get - libraries
obscure what's going on. My goal is understanding, and minimal lines of code
(to aid understanding)
Right now I have several competing ideas - I could do with some informed
advice here.
I'll fire off some questions:
For Graphics:
1) SVG or
2) HTML5 canvas
Canvas is faster but has no support for objects, making onclick and ondrag,
onmouseenter in a canvas
is a pain and probably either eats CPU or memory. Any good libraries - I've
looked at all the well know
ones. I only want object support for a canvas (ie object grouping and adding
click, move events to
object groups) - not lots of other stuff. This is why I'm currently using
SVG.
For the keyboard:
How can I get keystrokes into my program from javascript - virtually every I
try is buggy -
Do I really have to sniff the browser type and fix the bugs of every single
browser ..
Rich text. I want to do pixel exact typography
I can make rich text by adding spans and css and stuff in the dom, but I
want pixel accurate
sizeing of spans - I want to do the following:
define several on screen divs with absolute size and position. Link them in
some order.
for example say
<div id="a" style="absolute:...." next="b">
<div id="b" ... next="c">
Then given rich text <p><span class="c1">...</span> I want to flow this
into div a, so that it overflows into div b - I need to pixel exactly
calculate where to spit the text in order to do this.
If I could do this I could easily port erlguten to run in a browser
All these seem like pretty basic things - but I can't seem to find any code
/Joe
Mathias Picker
> Rich text. I want to do pixel exact typography
>
> I can make rich text by adding spans and css and stuff in the dom, but
> I
> want pixel accurate
> sizeing of spans - I want to do the following:
>
> define several on screen divs with absolute size and position. Link
> them in
> some order.
> for example say
>
> <div id="a" style="absolute:...." next="b">
> <div id="b" ... next="c">
>
> Then given rich text <p><span class="c1">...</span> I want to flow
> this
> into div a, so that it overflows into div b - I need to pixel exactly
> calculate where to spit the text in order to do this.
>
> If I could do this I could easily port erlguten to run in a browser
>
> All these seem like pretty basic things - but I can't seem to find any
> code
>
> /Joe
>
From my understanding there is no such thing as pixel perfect with html.
One important idea behind it (and css) is device independence. A web
site should be readable from my 3.8'' N900 to my 26'' main monitor.
Partly for this reason, with CSS, I'm allowed to put my own user
stylesheet before any site supplied (which, incidentally, I do),
breaking every pixel perfect designs you might try.
HTML is no "rich text" in the sense of RTF, it's simple document markup,
and css gives you some (!) ability to influence the look of it (but
*cascading*, so the user has the last word)
If you need pixel perfect, use a different technology. If you want to do
html & css, accept that the recipient and his device has the last word,
and you can only hint at what you want.
Cheers, Mathias
David Mercer
> Am Mittwoch, den 16.02.2011, 16:34 +0100 schrieb Joe Armstrong:
> > Rich text. I want to do pixel exact typography
. . .
> If you need pixel perfect, use a different technology. If you want to
> do
> html & css, accept that the recipient and his device has the last
> word,
> and you can only hint at what you want.
Agreed, though you if you really *really* wanted to do it, you could place each individual character with SVG. You could even draw each individual character from your own font in SVG. Cufón Fonts — or, rather, Raphaël's implementation of it — is what I have experimented with. You can even look through the font metric information and get kerning values if you want to implement kerning (though, if I recall correctly, Raphaël implemented kerning wrong — I think it was a sign error, but I don't remember exactly). It *can* be done, but such usage may better be implemented by creating a PDF.
Cheers,
DBM
Edmond Begumisa
Apparently, JQuery is super smart how it handles this in relation to
possible XSS attacks ... or so I hear. Personally, I don't really see the
difference. Frédéric seems more informed here and would be in a better
position to explain.
> At the moment I don't really
> like libraries since I want to see whats going on as near to the bottom
> level as I can get - libraries
> obscure what's going on. My goal is understanding, and minimal lines of
> code
> (to aid understanding)
>
I too would only use a client side library for things that I cannot do
myself (like render SVG on non-SVG supported browsers) or things that are
too tedious to do manually (like handling cross-browser quirks). But I'm
informed on this thread that there's more to these tools like escaping
JS/Tags/attributes, which I'm told is really hard to do correctly from
Erlang. Again, I'm not sure about this myself, and planned on
investigating further.
> Right now I have several competing ideas - I could do with some informed
> advice here.
>
> I'll fire off some questions:
>
> For Graphics:
>
> 1) SVG or
> 2) HTML5 canvas
>
> Canvas is faster but has no support for objects, making onclick and
> ondrag,
> onmouseenter in a canvas
> is a pain and probably either eats CPU or memory. Any good libraries -
> I've
> looked at all the well know
> ones. I only want object support for a canvas (ie object grouping and
> adding
> click, move events to
> object groups) - not lots of other stuff. This is why I'm currently using
> SVG.
>
Hmmm... Question: What is your higher priority? Having a canvas to deliver
UI to your users or making the browser easier to program UI to?
If you are not so bothered whether delivery is to an actual browser so
long as it's done over the web using web-standards then I have a
suggestion...
SVG support in the Mozilla Framework is pretty OK (save a few features);
the Gecko layout engine handles SVG natively so it renders pretty quickly
since no JS wrapper libraries are required. What if I whipped up a little
XULRunner client that you could push your UI to?
Your users could download and install this generic client. At startup, it
could popup a dialog asking them which app (i.e. server URI) they want to
connect to, then open a new blank window that you could push all your SVG.
One could also push HTML, XUL, etc.
Sure, it's not a browser that everyone already has, so your end-users will
have bothered to install it. But they'll have to do this only once since
the client is generic. Besides, what you're doing doesn't really fit
precisely into the browser idiom anyway, so maybe a special client is
appropriate. And this XULRunner-based generic client could be compiled for
and installed on every platform that Firefox supports.
> For the keyboard:
>
> How can I get keystrokes into my program from javascript - virtually
> every I
> try is buggy -
> Do I really have to sniff the browser type and fix the bugs of every
> single
> browser ..
>
AFIAK, yes. Keystrokes are one of those cross-browser quirks library
writers don't focus on. So you have to wade through the mess yourself.
But..
Another advantage of a generic XULRunner-based client: You won't have to
deal with cross-browser quirks since you'll only be targeting 1 layout
engine, 1 js engine, 1 DOM implementation, 1 etc, 1 etc.
> Rich text. I want to do pixel exact typography
>
> I can make rich text by adding spans and css and stuff in the dom, but I
> want pixel accurate
> sizeing of spans - I want to do the following:
>
> define several on screen divs with absolute size and position. Link them
> in some order. for example say
>
> <div id="a" style="absolute:...." next="b">
> <div id="b" ... next="c">
>
> Then given rich text <p><span class="c1">...</span> I want to flow this
> into div a, so that it overflows into div b - I need to pixel exactly
> calculate where to spit the text in order to do this.
>
I don't think you can control the size of spans in that way. Spans are
non-replaced inline elements, so they have no width or height properties...
http://www.w3.org/TR/CSS2/visudet.html#the-width-property
http://www.w3.org/TR/CSS2/visudet.html#the-height-property
Though Mathias and David have indicated web-presentation isn't designed
for this, I have seen people pull this kind of thing off with client-side
js trickery (don't remember where exactly, I think it was one of those
AJAX word-processors), but it went something like this...
* Create an invisible DIV with the width property set to DIV "a" but the
height property set to "auto"
* Give invisible DIV the appropriate typography settings with which you
want to measure (font, font-points, etc).
* Populate invisible DIV a with your text. DIV should expand downwards to
fit the text.
* Loop, removing say 100 characters from the invisible DIV until it's
height is smaller than div "a"
* Now loop adding 1 char back til invisible DIV's height is larger than
DIV "a"
* You've now found your sweet spot.
* Put those chars less one into DIV "a"
* Put the rest into DIV "b"
IIRC, the algo was optimised not to start with the entire text, but take a
reasonable guess based on the size of DIV "a". It's ugly but I remember
being surprised how well it worked.
- Edmond -
________________________________________________________________
Edmond Begumisa
It's actually much smarter to keep adding 100 chars of text and checking
if the invisible DIV grows to be bigger than DIV "a" than removing... but
you probably already saw that. It's really hot where I am and I'm not
thinking straight :)
- Edmond -
Joe Armstrong
http://stackoverflow.com/questions/118241/calculate-text-width-with-javascript
Which seems to answer my question.
Now I think I've got enough to write a decent in-browser editor that gets
quotes right.
<aside> why do in-brower editors not do quotes correctly? + the start quote
and end quote
symbols are *different* - if anybody knows of a javascript in-browser text
edit thats gets quote
right please tell me.</aside>
/Joe
Frédéric Trottier-Hébert
On 2011-02-16, at 23:26 PM, Edmond Begumisa wrote:
>
> Apparently, JQuery is super smart how it handles this in relation to possible XSS attacks ... or so I hear. Personally, I don't really see the difference. Frédéric seems more informed here and would be in a better position to explain.
The framework uses the usual DOM functions to manipulate HTML elements. As an example, you have the .text() and .html() functions to handle text for you. using .html(Val) on some element will allow you to change the HTML inside it. Using .text(Val) will however attach the element with something equivalent to 'document.createTextNode(Val)' (with all compatibility taken in account for all browsers on top of it).
jQuery usually just makes it simpler to use a clean interface, have proper attributes and whatnot, but it does delegate most of the escaping to the browser (and rightly so!).
Things where jQuery does more than the basic browser are usually on a higher level. As an example, it provides JSON parsing/handling for $.getJSON calls. Another example could be the '$.getScript()' method, which allows you to dynamically add a .js file to the page, but with safer escaping than just dumping it in (it does just create a text node, but again, provides the proper abstraction to make it sure you'll be safer).
It is possible to have safe input of data by using the right type of DOM manipulation. It is easier to do it right when you have proper abstractions through libraries or frameworks and whatnot.
--
Fred Hébert
http://www.erlang-solutions.com
Toby Thain
> ...
> var c = "var x = JSON.Parse('valfromuser'); // Trusted" +
> "var y = JSON.Parse('valfromuser'); // Trusted" +
> "var z = x + y; // Trusted";
> eval(c);
>
The bug that remains here is rather amusing in the context of the
conversation. Is it intentional?
--Toby
> - Edmond -
Edmond Begumisa
> On Thu, Feb 17, 2011 at 5:26 AM, Edmond Begumisa <
> ebeg...@hysteria-tech.com> wrote:
>
>> I don't think you can control the size of spans in that way. Spans are
>> non-replaced inline elements, so they have no width or height
>> properties...
>>
>> http://www.w3.org/TR/CSS2/visudet.html#the-width-property
>> http://www.w3.org/TR/CSS2/visudet.html#the-height-property
>>
>> Though Mathias and David have indicated web-presentation isn't designed
>> for
>> this, I have seen people pull this kind of thing off with client-side js
>> trickery (don't remember where exactly, I think it was one of those AJAX
>> word-processors), but it went something like this...
>>
TYPO: That sentence should have started "Though *as* Mathias and David
have indicated web-presentation isn't designed for this..."
Missing word made it sound like I was saying they were wrong. Sorry Sirs.
Forgot to mention: Ample sdk is pretty cool. Has a bunch of cross-browser
stuff you might be interested in like SVG and, HTML 5 Canvas would you
believe, XUL (in IE/Opera/Chrome)
- Edmond -
________________________________________________________________
Edmond Begumisa
- Edmond -
On Thu, 17 Feb 2011 05:09:38 +1100, Toby Thain <to...@telegraphics.com.au>
wrote:
--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/
________________________________________________________________
Yurii Rashkovskii
for erlang is now available on GitHub:
https://github.com/yrashk/socket.io-erlang
We'll be adding more documentation and some nice features soon; but it
seems to be quite usable as is :)