New FFI API implemented on dev branch

[Evan Czaplicki]

I just finished implementing the ports API on the dev branch, replacing the "foreign export jsevent ..." API. Please let me know if you have questions or comments or whatever!

Some folks were worried about whether it's clear when ports are coming in or out. Do you feel this way? Do you have suggestions to make it better?

Todo List:

• The main thing left to do is to be a bit stricter about when errors are thrown. The Elm code knows when it is expecting a JSNumber, so I'd like to throw an error when someone gives null or '' or whatever even though JS would "figure it out".
• There are also a bunch of default ports: log, title, redirect, stdout, stderr. stdout and stderr are new and will do the right thing when you are running in node. Not sure if this is a good idea, let me know if you think there'll be problems! More generally, it's conceivable for people to distribute "port handlers" for various things, like audio.

[Peter Damoc]

Could you maybe produce some pretty pictures detailing how all this works?
I'm a little bit confused (partly because I have little experience with Elm). 

The idea of components is nice and I would like to understand it better. 

The way I view it is that it would be nice to have Elm components encapsulated and communicating with code written in other languages OR components written in other languages incapsulated communicating with Elm. 

In a way (to my limited understanding), this is what is happening now. There are some components implemented in other languages that have a series of "ports" which now are called Signals. e.g. "Mouse" is a component with "position" as a "port" that I can connect to other stuff. 

Question than becomes, how do you define a component? 

--
You received this message because you are subscribed to the Google Groups "Elm Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elm-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
There is NO FATE, we are the creators.
blog: http://damoc.ro/

[Evan Czaplicki]

Peter, that's a great idea! I think that'd help make this a lot clearer and make for a better presentation :)

A component is what you get when you compile an Elm program. Running elm --make SideBar.elm will produces a component called Elm.SideBar that may have a bunch of incoming ports and outgoing ports. This can be embedded in any JS/web project. The components can handle graphics or just be for logic. In the SideBar example, it would be a visual component embedded in a <div> that would act as the side bar for an application. The ports are ways to send info to and from that component.

Importantly, the port interface forces a strong API for each component, so if it depends on other parts of the program, all of those dependencies are explicit. There's no "just go find that object and look at what that field happens to be right now". I think this will lead to code that is more modular and maintainable. It'll also be more clear when you are messing up or are modeling the problem in a silly way.

That's probably not as helpful as pictures, so I'll try to get that done soon :) Thanks again for the suggestion!

[Max Goldstein]

Firstly, this looks a lot better than the initial draft shared privately.

A few questions:

• Presumably the error handling function can send a value on any port for which it has a variable to access it with. If it wants to try sending another value on the same signal (e.g., "oh whoops, try it now") is there a way to do this modularly? You can have the function return a new value to try (do nothing if it returns undefined) or perhaps set the this context to something useful. Of course, if this new value is also invalid, you can get an infinite loop very easily, which is a compelling argument to not support this behavior. Either way, a library of common error handlers would be cool.
• If I call unsubscribe on an output port with a function argument, can it really find which subscriber that is and unsubscribe them? Something something function equality undecidable. Can I (un)subscribe multiple functions as multiple arguments in one call? What happens if I subscribe the same callback twice? Etc.
• Calling unsubscribe with no arguments can (should?) clear all subscribers. If you want to emulate D3, calling subscribe with no arguments can return an array of all subscribers.

[Alex Neslusan]

This is cool and I've figured out how to get it working, but could you expand a bit on the error handler? I made an input port of type Signal Int, and when I send it a string instead, the console just says "undefined". What sort of thing do I put as the error handler? Right now I just have an empty block {};

[Alex Shroyer]

I haven't figured out how to get it working, care to share your example?

I am trying to convert this:

foreign import jsevent "fftEvents" (fromList [])

  fft : Signal (JSArray JSNumber)

...to the new Ports method, but this isn't working:

port fft : Signal (JSArray JSNumber)

port fft <- fromList []

This does compile, but it looks wrong:

port fft : Signal (JSArray JSNumber) 

port fft <- (fromList <~ constant []) 

Thanks in advance!

[John Mayer]

What if I don't want a conversion and would rather just work with JSValue? Can I use 'id' ?

--

[Evan Czaplicki]

Talking at Prezi today, we had some ideas about better names for ports and generalizing the idea.

Problems with the existing way:

1. Laszlo and Sean both think using the name "port" for both incoming and outgoing signals is confusing. I added the arrow to help distinguish, but I think I agree with them.

2. We'd like to be able to bring in static values and export static values and functions, which does not really fit with the port analogy.

So the idea is to use import/export as keywords for the FFI. This really captures "trading goods with foreign entities" better than any other terms we could think of.  So it'd be like this:

import userID : Int
import prices : Signal Int

export orders : Signal String

export factorial : Int -> Int

Notice that this proposal can deal with non-signals quite naturally. It's also fine to export functions (but not import them to maintain purity).

The downside is that the existing syntax for "import" would need to change. This sorta sucks, but it's a good opportunity to fix the issues with imports that Spiros pinted out. Viable replacements for the current meaning of import are using (from C#), include (from C++, PHP, SML, OCaml), and require (from JS). I can't really decide whether I prefer using or include.

Anyway, I'm curious what you think of this idea. Any comments?

--
Sent from Gmail Mobile

[Evan Czaplicki]

Sorry, just saw everyone's comments. I'm going to have really bad internet and response time for the next 4 days :/

I'll write an example that addresses these questions and share tomorrow.

Sent from Gmail Mobile

[John Mayer]

"using" is trendy :-) I say go for it, it's not a difficult thing for folks to patch.

[Max Goldstein]

I prefer include, but it's not a strong preference. Whatever the new keyword, let's have a very specific compiler error for the old usage of import. "You said import String. You probably meant ______ String."  Also, Ruby uses require, as does PHP if you want a fatal error and not a warning (typical PHP).

[Alex Shroyer]

I have no experience with C# but I think using sounds the most appropriate for what it's actually doing.  Or even uses. While we're at it, take away the requirement for the where in a module declaration and write:

module Foo

uses JavaScript as JS

...Which I think would be quite nice for both writing and reading.

[John Mayer]

Yeah definitely seconding the removal of "where" from modules. What's the point?

--

[Max Goldstein]

I'm also getting behind Alex's proposal, unless someone wants to make a case for my old favorite, include.

[Mads Flensted-Urech]

We could go for "use" as this matches "import" more than "using" which match "importing" and "uses" which matches "imports".

module MyModule

use List

use List as L

use List (*)               -- seems more natual with "use" (instead of open)

use List (map,foldl,foldr)

use List (map, foldl) as (lmap, lfoldl)   -- idea: would this be possible? or usefull?

I also support the removal of "where".

"include" indicates that the file is made part of this file before compiling/interpreting. That is not the case for Elm, since it just links up the the module at run time.

On Tue, Jan 7, 2014 at 3:24 AM, Max Goldstein <maxgol...@gmail.com> wrote:

I'm also getting behind Alex's proposal, unless someone wants to make a case for my old favorite, include.

--

[Mads Flensted-Urech]

A couple of questions:

a) would import and export be restricted to the top level module, just like main is?

b) can you post some examples of how you imagine the JS side of the interface for static values and exported functions?

[Evan Czaplicki]

I'm pleasantly surprised the reaction to the import/export idea is so positive :)

 

A couple of questions:

a) would import and export be restricted to the top level module, just like main is?

 

Laszlo has recommended that it is restricted in that way, at least in the first draft. It's currently implemented without this restriction, but I think adding the restriction is a good plan because it's not very clear what the implications of this decision will be.

 

b) can you post some examples of how you imagine the JS side of the interface for static values and exported functions?

I was thinking it'd be the same as in the ports proposal, so if you have the following imports and exports:

import userID : Int
import prices : Signal Int

export orders : Signal String

export factorial : Int -> Int

The initialization in JS would look like this.

var prices = Elm.signal(defaultValue, errorHandler);

var stocks = Elm.worker(Elm.Stocks, { prices:prices, userID:42 });

stocks.exports.orders.subscribe(handler);

stocks.exports.factorial(5) == 120

Does that sound okay? Any problems?

Again, this would throw errors really aggressively for type errors. Reminds me of a question from earlier about subscribe/unsubscribe: JS lets you do function comparison by comparing pointers rather than meaning (i.e. if you don't care about referential transparency, this is okay). So you can unsubscribe a function as long as you still have a reference to it. I think the .subscribe() and .unsubscribe() ideas would be okay. It's really "JS" but I think that's an emotional thing than a technical critique.

[Alex Shroyer]

I like use.  It's short.

I also really like this:

import Foo (bar) as (barAlias)

because it lets you write to suit your style rather than to suit the language, and allows you to write like this:

import JavaScript (toFloat) as (toFloatJS)

myFunction = toFloat someElmValue / toFloatJS someJSvalue  

or alternatively like this:

import JavaScript as JS

myFunction = toFloat someElmValue / JS.toFloat someJSvalue  

Which in some cases could make a big difference in terms of readability.

[Evan Czaplicki]

Agda has renaming and hiding for module imports. Their syntax is quite nice. It's cool, but I think it's decently complex and does not buy a ton.

I talked with Laszlo more today about the import/export idea and there's another draft. I'll start a new thread for that :)

--

[Adrian VELICU]

Hi Evan,

I hope you are well! We met in Edinburgh last year, I was telling you that I would be working on Elm and Robotics..

Thanks for all the hard work on the new ports API, I am using it in my current project.

I noticed that if I subscribe to a port for updates, the subscribed function will never get called for the initial value of that port, only when the value of the underlying signal changes.

Is there any way in the current dev version to bypass this problem, eg. by getting the value directly instead of subscribing to the port?

I also wanted to mention that I've modified the Elm runtime slightly to work under node.js instead of in a browser. The Events model in node.js is a bit different than the standard one, so I think it is not impossible for the above problem to be caused by my modifications. Please let me know if what I am trying to do is supposed to work out of the box with Elm in a browser.

Regards,

Adi

[Max Goldstein]

I can answer the question about initial values: it's a well-known (to veterans) quirk that initial values are not treated as events. So foldp (\_ _ -> 5) 2 (constant 0) stays at the initial value 2 rather than receiving a 0 and changing to 5. I'm starting to think we may need a "send the initial value of this signal as an event at program start" keyword.

Evan can say more about node, but the new repl is node-based so there may be a lot in common.

[Evan Czaplicki]

Hey again Adi! Very happy to hear from you :D

What do you need the initial value for? I think there are two ways to address this issue:

1. Provide a way to ask "What's the current value of the signal?"
2. Provide a handler for the initial value, so .subscribe would take two functions.

I think I prefer option one, but I don't really know what specific problems we'd like to solve with this (although I do see why it is important). Please tell me more about your situation and what you think the general use case might be.

What modifications did you need to make for node.js? I'd like to get Elm working there. I needed to make some stubs and dummy things to get the REPL working, but I have not done too much more than that. I suspect .addListener can easily be some sort of polyfill to get it working on node and in browser, but I've never focused on this. Point is, I'd like to merge these fixes in so tell me more about it or point me to some diffs :)

Max, it's a choice that it is that way. Saying it is a quirk implies that there's some well-thought out and plausible alternative. I think it also implies that I messed it up, but I have not seen coherent evidence to that effect. This discussion is relevant to the "primitveNode" idea I emailed about a while ago, so please continue the discussion there if you think I am confused.

On Wed, Jan 8, 2014 at 4:44 PM, Max Goldstein <maxgol...@gmail.com> wrote:

I can answer the question about initial values: it's a well-known (to veterans) quirk that initial values are not treated as events. So foldp (\_ _ -> 5) 2 (constant 0) stays at the initial value 2 rather than receiving a 0 and changing to 5. I'm starting to think we may need a "send the initial value of this signal as an event at program start" keyword.

Evan can say more about node, but the new repl is node-based so there may be a lot in common.

--

[Janis Voigtländer]

2014/1/8 Max Goldstein <maxgol...@gmail.com>

I'm starting to think we may need a "send the initial value of this signal as an event at program start" keyword.

Note that this (for signals/events, I don't know about ports) would be interdefinable with the startSignal function I wanted in this thread:

https://groups.google.com/forum/#!topic/elm-discuss/X4wmckEtMyg

In the sense that such startSignal could be defined with your hypothetical keyword; and on the other hand if startSignal were available, your keyword wouldn't need to be a keyword but could be defined from startSignal and sampleOn (and possibly merge).

Now, in above thread it was explained to me that and why such behavior is not really doable with how the Elm runtime currently (then, and still?) works. At least not with a guarantee that that "initial value event" really happens as first thing right after program start.

On the other hand, in that thread Tim Hobbs gave a code example of how to modify the runtime to at least get a weak version of said behavior, without an "absolute first" guarantee.

[Max Goldstein]

Forked to new thread. Let's get back to Adrian and node.

[Adrian VELICU]

Hi Evan, thanks for your quick reply.

I didn't have to do much to get Elm to work in Node.js. I will share with you the private Bitbucket repo in which I pushed my work immediately, and I will talk to my professor next week if it is OK to make the repository public before my project is finalised.

The reason why I was interested in the initial value for ports is that I was using ports to indicate from the inner Elm program (i.e. the one that users would write to control robots) to the outer Elm/JS robotics glue (i.e. the one that I would develop to enable usage of Elm for robotics) what kind of connection should be made to the robot, other parameters, etc. Eg. I define:

data Connection = Serial String | TCPIP String | Interactive

connectionToString : Connection -> String

connectionToString c = case c of

 Serial s -> "serial " ++ s

 TCPIP s -> "tcpip " ++ s

 Interactive -> "interactive"

port connection_ : Signal JS.JSString

port connection_ <- constant (JS.fromString (connectionToString connection))

Then the user just writes:

connection = TCPIP "127.0.0.1:10020"

Thinking about it, maybe this is not ideal; maybe the Elm program itself should only contain the logic to run the robot, and this meta-information should be configured somehow else.

Perhaps I should be aiming to create a binary such as elm-robot, which takes command line parameters to configure the connection and to run a particular robot (written in pure Elm).

I feel like this is somewhat logically equivalent to elm-server, which runs various Elm files for pages which do not have to be concerned with how they are being served.

Let me know what you think.

Adi

[Max New]

On Tuesday, January 7, 2014 2:43:50 AM UTC-6, Evan wrote:

I'm pleasantly surprised the reaction to the import/export idea is so positive :)

 

A couple of questions:

a) would import and export be restricted to the top level module, just like main is?

 

Laszlo has recommended that it is restricted in that way, at least in the first draft. It's currently implemented without this restriction, but I think adding the restriction is a good plan because it's not very clear what the implications of this decision will be.

 

If you left the restriction out, would it be possible to implement modules like Native.Signal using import/export? Then we'd be able to completely separate "elm the pure functional language" from "elm the frp system" which would really help on the node integration/testing front.

Though it seems like you intentionally left out "importing javascript functions" above so I assume you don't want to (effects and such). If you are leaving that out, does that mean you can only export first-order functions? I think exporting higher-order functions would be semantics-breaking:

runner : (Int -> Int) -> Int
runner f = f 1 + f 1

Without exporting this could be safely transformed into

runner f = let x = f 1 in x + x

but that's not a safe transformation if runner is exported since it could be called in JS with a stateful JS function.
 

[Evan Czaplicki]

If you left the restriction out, would it be possible to implement modules like Native.Signal using import/export? Then we'd be able to completely separate "elm the pure functional language" from "elm the frp system" which would really help on the node integration/testing front.

Yeah, I think a bunch of "inputs" could be implemented without corresponding Native implementations if that was allowed (Mouse, Touch, Window, and maybe some others). I don't know if I understand the testing aspect of this though. Is the idea that you could more easily mock out Mouse inputs?

Though it seems like you intentionally left out "importing javascript functions"

Yeah, I do not want to do that. If impurity is going to be introduced, I want it to be in a more principled way.

above so I assume you don't want to (effects and such). If you are leaving that out, does that mean you can only export first-order functions? I think exporting higher-order functions would be semantics-breaking:

This is a great point, I did not think of that! I was only vaguely imagining first-order cases. I think restricting it to first-order functions is a good compromise, and I suspect that solving higher-order exports would come with a solution for function imports. In other words, I don't know when something like that will be possible.

[Evan Czaplicki]

Oh, oops, here's what I meant to say to Adrian! I think the API that comes in 0.11 will let you bring non-signals through. We did a small proof-of-concept within Prezi, and very quickly, we needed to bring in a "userID" which is a fixed string that does not change throughout the whole program. It could be done with a signal, but it just kind of sucked. So I'm trying to figure out nice syntax for more general imports and exports (for some non-signals).

Would that address your particular issue?

I still get the sense that it'd maybe be valuable to do something with initial values, but my feeling now is to wait until there are a few examples where it's definitely the only way to get something done to see which approach is best.

[Max New]

On Friday, January 10, 2014 5:21:43 AM UTC-6, Evan wrote:

If you left the restriction out, would it be possible to implement modules like Native.Signal using import/export? Then we'd be able to completely separate "elm the pure functional language" from "elm the frp system" which would really help on the node integration/testing front.

Yeah, I think a bunch of "inputs" could be implemented without corresponding Native implementations if that was allowed (Mouse, Touch, Window, and maybe some others). I don't know if I understand the testing aspect of this though. Is the idea that you could more easily mock out Mouse inputs?

No, I just meant it would make it easier to produce a runtime without the browser-specific stuff and then introduce server/node-specific stuff the same way. But like you said this would mean handling possible impurity.

[Evan Czaplicki]

Gotcha, that makes sense :) I don't get the handling impurity part though.

--

You received this message because you are subscribed to the Google Groups "Elm Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elm-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Sent from Gmail Mobile

[Evan Czaplicki]

New ports syntax that allows non-signals through is ready. I'll email describing this soon.