[erlang-questions] Programming Erlang: Chap 18, Websockets
7stud
{badmatch,{error,{not_started,ssl}}
I'm on Mac OSX 10.10.5. Here is my output:
===========
../ezwebframe-master$ gmake
...
....
Erlang/OTP 19 [erts-8.2] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
a simple_demo of websockets....
Load the page http://localhost:1456/ in your browser
Starting:{ok,"/Users/7stud/erlang_programs/ezwebframe/ezwebframe-master/demos"}
Eshell V8.2 (abort with ^G)
1> {"init terminating in do_boot",{{badmatch,{error,{not_started,ssl}}},[{ezwebframe,start_link,2,[{file,"src/ezwebframe.erl"},{line,22}]},{init,start_em,1,[]},{init,do_boot,3,[]}]}}
init terminating in do_boot ()
Crash dump is being written to: erl_crash.dump...done
make[1]: *** [all] Error 1
gmake: *** [Makefile:4: all] Error 2
===========
Thanks for any help is solving this problem!
_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions
Jesper Louis Andersen
Joe Armstrong
changed since I wrote the book. I also used make and not rebar.
You've now run into what is one of the biggest problems in the
computer world - code that used to work no longer works despite the
fact the code itself has not been changed.
One way around this is to use zero dependencies - in fact several
programs I wrote 30 years ago in Erlang still work fine with no
changes because there are no external dependencies. (as an aside: this
is why I like code with zero external dependencies - it takes a lot
longer to write - but you don't have to support it into the future
when they things you depend upon change in a manner that is
incompatible manner)
In this example the underlying websockets protocol changed -
websockets its is a crazy mess - websockets should have provided raw
socket transport instead they chose to scramble the data in a weird
manner to avoid problems with badly written proxies.
The idea of "write once run anywhere" is a great goal - but we can't
even do this - "write once run forever with no changes to your code"
would be even better and things like NixOS are a step in the right
direction.
Welcome to the world of broken software.
In my opinion this (making sure software evolution does not break
existing functioning software) is one of the biggest remaining
software problems - my guess is that this problem is getting worse
(due to the explosion in the numbers of programming languages,
libraries and frameworks and build systems) and will be even worse in
the IoT world.
Lots for you guys to work on :-)
Cheers
/Joe
Tristan Sloughter
branch of a git repo. So the dependency is defined as one that will
continually change over time.
The version of ranch used when the code was written probably didn't list
`ssl` as a dependency, now that it does `application:start(ranch)` will
fail. So the quick fix is to add `application:start(asn1),
application:start(public_key), application:start(ssl),` before the start
of ranch. Or just `application:ensure_all_started(ranch)`.
Moral of the story: lock your dependencies (even better if on a package
and not just a git reference) and use a release or
`application:ensure_all_started` for running a project :)
--
Tristan Sloughter
"I am not a crackpot" - Abe Simpson
t...@crashfast.com
Richard A. O'Keefe
> On 26/06/2017, at 9:00 AM, Joe Armstrong <erl...@gmail.com> wrote:
> You've now run into what is one of the biggest problems in the
> computer world - code that used to work no longer works despite the
> fact the code itself has not been changed.
policy for this very reason. Sadly, it does not protect against
operating system or compiler changes. It is notoriously the
case that in pursuit of "optimisation", C compilers are taking
ever more advantage of the strict letter of the C standard, so
that code that used to work may stop working.
For example, there's a compiler from a historic programming language
to C. It used to work on my Mac. After an Xcode upgrade, it didn't
compile any more. Turned out that the "C" it generated as GNU C
with nested functions, and clang supports *some* GNU C extensions
but not that one.
Basically, there is NO way to protect yourself from changes to your
infrastructure. You can try to stick close to documented standards,
but things still change. And Erlang itself has dropped a number of
library modules over the years.
7stud
application:start(public_key), application:start(ssl),` before the start
of ranch. Or just `application:ensure_all_started(ranch)`.<
application:ensure_all_started(ranch)
Source code:
----------
start_link(Dispatch, Port) ->
io:format("Starting:~p~n",[file:get_cwd()]),
ok = application:start(crypto),
%%ok = application:start(ranch), % Line #22
application:ensure_all_started(ranch),
ok = application:start(cowlib),
ok = application:start(cowboy),
ok = web_server_start(Port, Dispatch).
----------
Output:
==========
../ezwebframe-master$ gmake
...
Erlang/OTP 19 [erts-8.2] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
a simple_demo of websockets....
Load the page http://localhost:1456/ in your browser
Starting:{ok,"/Users/7stud/erlang_programs/ezwebframe/ezwebframe-master/demos"}
Eshell V8.2 (abort with ^G)
Crash dump is being written to: erl_crash.dump...done
make[1]: *** [all] Error 1
gmake: *** [Makefile:4: all] Error 2
-----Original Message-----
From: "Tristan Sloughter" [t...@crashfast.com]
Date: 06/25/2017 05:14 PM
To: erlang-q...@erlang.org
Subject: Re: [erlang-questions] Programming Erlang: Chap 18, Websockets
The main issue here looks to be that the dep is tied to the master
branch of a git repo. So the dependency is defined as one that will
continually change over time.
The version of ranch used when the code was written probably didn't list
`ssl` as a dependency, now that it does `application:start(ranch)` will
fail. So the quick fix is to add `application:start(asn1),
application:start(public_key), application:start(ssl),` before the start
of ranch. Or just `application:ensure_all_started(ranch)`.
Moral of the story: lock your dependencies (even better if on a package
and not just a git reference) and use a release or
`application:ensure_all_started` for running a project :)
--
Tristan Sloughter
"I am not a crackpot" - Abe Simpson
t...@crashfast.com
On Sun, Jun 25, 2017, at 02:00 PM, Joe Armstrong wrote:
> Unfortunately cowboy (which I used) and the websockets standard have
> changed since I wrote the book. I also used make and not rebar.
>
> computer world - code that used to work no longer works despite the
> fact the code itself has not been changed.
>
7stud
{undef,[{cowboy,start_http,
points to:
web_server_start(Port, Dispatcher) ->
E0 = #env{dispatch=Dispatcher},
Dispatch = cowboy_router:compile([{'_',[{'_', ?MODULE, E0}]}]),
%% server is the name of this module
NumberOfAcceptors = 100,
Status =
cowboy:start_http(ezwebframe,
NumberOfAcceptors,
[{port, Port}],
[{env, [{dispatch, Dispatch}]}]),
...
...
Tristan Sloughter
Now it is broken because of the other issue I mentioned, the copy of
cowboy it is fetching is from github master and the API changed since
the code was written.
You'll either need to update the project's code or find what cowboy
version it worked with and update the dependencies to use that version.
7stud
> cowboy it is fetching is from github master and the API changed since
> the code was written.
> You'll either need to update the project's code or find what cowboy
> version it worked with and update the dependencies to use that version.
rebar.config:
========
{deps, [
{cowboy, "1.0.*", {git, "https://github.com/ninenines/cowboy/tree", "1.0.x"}}
]}.
========
original rebar.config:
=========
{deps, [
{cowboy, ".*", {git, "git://github.com/extend/cowboy.git", "master"}}
]}.
Nuku Ameyibor
Joe Armstrong
> The main issue here looks to be that the dep is tied to the master
> branch of a git repo. So the dependency is defined as one that will
> continually change over time.
even if I pointed to a version of cowboy/ranch (whatever) that I
tested against in 2013 it still would not work - to get it to work
you'd have to get a browser from 2013 and pray that the OS hasn't
changed that much.
This is only 4 years ago - now consider the problems people will have
in 200 years time trying to make programs written hundreds of years
ago work.
- or in 10,000 years time.
>
> The version of ranch used when the code was written probably didn't list
> `ssl` as a dependency, now that it does `application:start(ranch)` will
> fail. So the quick fix is to add `application:start(asn1),
> application:start(public_key), application:start(ssl),` before the start
> of ranch. Or just `application:ensure_all_started(ranch)`.
>
> Moral of the story: lock your dependencies (even better if on a package
> and not just a git reference) and use a release or
> `application:ensure_all_started` for running a project :)
smalltalk you could at least state that the SHA checksum of your image
was XYZ
and the behaviour might be reproducible.
Given that my machine has 500GB of state 2 ^ (500 * 10^9 * 8) - is an
incredibly large number (exercise: compare to #atoms in Universe)
Did my program work when I tested it? and if it did work did it work
because my machine was in some weird state? Will it work on another
machine?
The answer is "who knows" - a decent guess is that if my program works
on my machine it should work on a similar machine.
Horror story:
Once I and Anders Danne bought "identical" machines (at work) - we
wanted the same software on both machines.
We did the following:
1) I installed on my machine, when the SW worked
2) He installed on his machine
After about 20 minutes an install that worked on my machine failed on
his machine.
Why? my attempts to install failed occasionally - he only repeated the steps
that worked so our machines were in a different states
OR
The chipsets / BIOS (whatever) of the two machines were different
We'll never know.
When people say "my program does not work" the obvious question is
"in what state was your machine" - but the latter question has no answer, so
instead we say "have you tried doing X" then we vary X until until the
program works or we give up trying. And if it does work we do not know
if the
last thing we did caused it to work, or all or of all the things that
we did before (which did not work) were also needed in order to make
it work.
The study of programs goes by the quaint name of "computer science"
- "computer alchemy" might be a better name since we don't really know
why our magic spells sometimes work and sometimes don't.
Cheers
/Joe
Joe Armstrong
>
>> On 26/06/2017, at 9:00 AM, Joe Armstrong <erl...@gmail.com> wrote:
>> You've now run into what is one of the biggest problems in the
>> computer world - code that used to work no longer works despite the
>> fact the code itself has not been changed.
>
> One of my colleagues has an explicit "no third party libraries"
> policy for this very reason. Sadly, it does not protect against
> operating system or compiler changes.
"identical" OS and libraries - but the chipsets were different which
lead to strange failures. Debugging was extremely difficult and tooks
weeks because I falsely
assumed that the machines were the same and would fail in the same way.
Jesper Louis Andersen
Welcome to the world of broken software.
In my opinion this (making sure software evolution does not break
existing functioning software) is one of the biggest remaining
software problems - my guess is that this problem is getting worse
(due to the explosion in the numbers of programming languages,
libraries and frameworks and build systems) and will be even worse in
the IoT world.
Lots for you guys to work on :-)
I'm a big fan of something like Golang's `fix` tool. Before 1.0, you could run `go fix` and have your code rewritten to match the new spec automatically. Wrangler in the Erlang world by Simon Thompson can do the same for when things are altered. In the C world, semantic patches as in Coccinelle is highly useful for the same thing since you can apply a fix all over a code base easily.
Fred Hebert
But there is a deeper issue - the websockets standard *changed* so
even if I pointed to a version of cowboy/ranch (whatever) that I
tested against in 2013 it still would not work - to get it to work
you'd have to get a browser from 2013 and pray that the OS hasn't
changed that much.
This is only 4 years ago - now consider the problems people will have
in 200 years time trying to make programs written hundreds of years
ago work.
- or in 10,000 years time.
I'd like to refer to Programs, Life Cycles, and Laws of Software Evolution, a paper by MM. Lehman from 1980. For the vast majority of software (i.e. not programs implementing a formal specification nor those for a limited game, like chess), five laws of program evolution are proposed:
----
I. Continuing change
A program that is used and that as an implementation of its specification reflects some other reality, undergoes continual change or becomes progressively less useful. The change or decay process continues until it is judged more cost effective to replace the system with a recreated version.
II. Increasing Complexity
As an evolving program is continually changed, its complexity, reflecting deteriorating structure, increases unless work is done to maintain or reduce it.
III. The Fundamental Law of Program Evolution
The work output of a project is independent of the amount of resources employed --
the ability to be productive when maintaining a program is a function
of the program’s environment. How you begin your project, how you build
the code, refraction to change by programmers who feel changes are
making the platform unstable and whatnot will lead the project to have a
stable, predictable rate of change.
IV. Conservation of Organizational Stability
The world the program evolves in and/or its complexity tend to limit how useful the program can remain through change
V. Conservation of Familiarity
Similar to the fourth law, but based on the understanding of developers. If the system becomes too complex, developers slow down. Managing complexity and actively maintaining software is essential to help with this.
----
The idea that you can freeze software in time is a difficult one. Google, for example, vendors not only their own code and its dependencies, but also the compilers that were used to produce them. Of course, the compilers as they are may require specific operating systems, configurations, environments, or hardware to run. Maybe they keep enough of those around just in case.
Rebar3, even if we wanted to keep it as backwards compatible as we could, can no longer be useful on Erlang/OTP R15, because the SSL libraries coming with that version are no longer are considered secure enough to even talk to the package index it uses. It's interesting that even if the program compiles and behaves right, it cannot inter-operate safely with the real world anymore. The problem of course being that even if the standards for the crypto it uses are still supported, the package index evolves with the rest of the world, and the attackers for such a product in the rest of the world keep improving. The ecosystem in which the system runs makes it impractical to maintain old versions because pretty much nobody else runs older software along with you. They'd have to go through the effort of making and maintaining their own index to avoid upgrading software for which updates are available, while increasing their risks when it comes to security. That's a losing proposition!
We cannot reasonably decouple the program from the environment that contains it for a very long time: the environment is what likely defined what the best solution to a problem was in the first place; it likely even defined the problem itself. And if the program is good enough, it will impact the environment itself. MS Paint was real cool until everyone imagined making fancier drawings and then someone wanted Photoshop. As the environment changes, so the programs become irrelevant or impossible to run. You can delay the inevitable, but you can't isolate software forever and keep it useful, unless it is able to remain useful when running in isolation already.
If not maintained, it will at best remain a curiosity for future generations. Some people are too young to have ever seen a floppy disk in person, much less used one. They'll be more likely to think an actual 3.5" floppy is a 3D-printed rendition of the 'save icon' than a storage device.
So the true question is maybe: if I want to write a book that will be useful in 4 years, 200 years, or 10,000 years time, what should the book contain? The exercises and solutions will have to adapt themselves to changing contexts and environments. What do you assume is going to be around in 200 years? What will need to adapt? If that's not useful for the book you have in mind, then maybe a shorter time expectation is warranted. Maybe a more flexible medium is warranted. Who knows, it depends on what your objectives are.
Grzegorz Junka
> Unfortunately cowboy (which I used) and the websockets standard have
> changed since I wrote the book. I also used make and not rebar.
>
> You've now run into what is one of the biggest problems in the
> computer world - code that used to work no longer works despite the
> fact the code itself has not been changed.
>
> One way around this is to use zero dependencies - in fact several
> programs I wrote 30 years ago in Erlang still work fine with no
> changes because there are no external dependencies. (as an aside: this
> is why I like code with zero external dependencies - it takes a lot
> longer to write - but you don't have to support it into the future
> when they things you depend upon change in a manner that is
> incompatible manner)
:-) Old applications will still work as long as they run on the same
version of Erlang and use the same versions of dependencies as the
example code. But then, even a zere-dependency code isn't guaranteed to
run on a never version of Erlang as it already had some braking changes
between releases.
Grzegorz
o...@cs.otago.ac.nz
software not working in (very subtly) different environments.
As I understand it, this is one of the driving forces behind
the application = VM approach using Docker and the like:
make a VM containing the things your application needs and
as little else as you can manage, and then never change it.
If you want a new version of the application, make a new VM.
Marco Molteni
> As I understand it, this is one of the driving forces behind
> the application = VM approach using Docker and the like:
> make a VM containing the things your application needs and
> as little else as you can manage, and then never change it.
> If you want a new version of the application, make a new VM.
Then one realises that the container is exposed to a network, and so open to attack.
Then one understand that the less worse approach is to keep updating everything and fixing what breaks.
marco
Rick Pettit
Aren’t you forced to expose your Erlang application to a network, thus "opening it up to attack", regardless of whether or not docker or some VM is in the picture?
In what ways does adding Docker make that problem “worse”, in your opinion?
-Rick
Marco Molteni
re-reading what I wrote I have to admit I was not very clear. Let me try again:
I start from the assumption that the main problem today is resisting to attacks (or to be more realistic: to resist as much as possible to attacks).
In order to resist to attacks, besides clearly doing a proper threat modeling, mitigation and defence in depth, the accepted best practice is to keep updating the operating system you are using, also if updating might break your own application due to a dependency that breaks backward compatibility.
Now comes Docker. Docker has been mentioned as a way to "freeze" the dependencies (OS and libraries) needed to deploy an application, in order to be assured that said application will run today (the date it has been released) and, say, in one year time frame.
My point was that yes, if you use Docker in that way the application will keep working, and each day that passes it will become more vulnerable (either the app or any of the dependencies or OS onboard the container).
Does this mean that Docker is bad from a security point of view? No, but it means that Docker by itself is not enough and is even worse than a VM or an OS on bare metal, because it will not receive OS security updates, while at least a VM or a bare metal OS _can_ be configured to receive automatic security updates.
Using Docker securely requires an infrastructure that automatically builds an new Docker image each time a security update is available, automatically tests it and automatically deploys this new image in place of the old one.
Clearly if a VM or bare metal is, for fear of breaking the application, configured _not_ to receive automatic security updates, then the situation is the same (as today's Petya ransomware shows, for example).
Hope this explains better what I meant :-)
marco
Rick Pettit
I follow you now, and agree with the following:
> Using Docker securely requires an infrastructure that automatically builds an new Docker image each time a security update is available, automatically tests it and automatically deploys this new image in place of the old one.
>
> Clearly if a VM or bare metal is, for fear of breaking the application, configured _not_ to receive automatic security updates, then the situation is the same (as today's Petya ransomware shows, for example).
Thanks for clarifying,
-Rick
Richard A. O'Keefe
> On 28/06/2017, at 8:20 AM, Marco Molteni <marco....@laposte.net> wrote:
...
> I start from the assumption that the main problem today is resisting to attacks (or to be more realistic: to resist as much as possible to attacks).
> My point was that yes, if you use Docker in that way the application will keep working, and each day that passes it will become more vulnerable (either the app or any of the dependencies or OS onboard the container).
If any application, implemented and deployed by any means whatsoever,
is exposed *directly* to the network, then yes, it is exposed to all
attacks that flow over that network.
The context in which I became aware of Docker is a radio astronomy one
-- don't ask me about radio astronomy, I am on the very fringes of the
project. I can spell "AIPS" and have it on my machine but that is all
I know about the subject -- where the intention is to do vast amounts
of computation on a cluster which will have no *direct* network
connection. Instead all connection with the outside world will go
through a gateway machine. THAT machine has to be kept up to date.
But as long as the protocols between the cluster and the gateway are
are as limited as possible and do not change, the cluster software
will not need to change.
Of course, in such a context there are *still* changes that force
changes to software. Amongst other things
- mistakes in the code will be detected and corrected
- the hardware will change (ARM? intel? EPYC? Power?) over time
as computational needs grow
- new applications will be developed as astronomers' needs grow
But there will still be a need to be able to reproduce old work,
for *decades*.
Anthony Ramine
>
> But there is a deeper issue - the websockets standard *changed* so
> even if I pointed to a version of cowboy/ranch (whatever) that I
> tested against in 2013 it still would not work - to get it to work
> you'd have to get a browser from 2013 and pray that the OS hasn't
> changed that much.
>
> This is only 4 years ago - now consider the problems people will have
> in 200 years time trying to make programs written hundreds of years
> ago work.
> - or in 10,000 years time.
Just a friendly reminder that Web standards don't change "just because", but because they were broken in some sort or didn't reflect real world usage in the first place. That's why now DOM and HTML are Living Standards, for example.