HTML5 drag drop events

[Kurt Harriger]

I was trying to implement drag and drop using elm-html rather than the Graphics library.  

I looked around a bit for some example code and found this library jvoigtlaender/elm-drag-and-drop recently mentioned here, but this seems to only work with the graphics elements not html. When I tried to use html elements rather than graphics library it did not seem to work any more.  If the element was marked as draggable then it only fired nothing events, if I marked the element as draggable = false I would occasionally get lift and move by events but it was very inconsistent and mostly still did not work.  

My attempt to use this library are here: https://gist.github.com/kurtharriger/abaa0cf4f08027b0b02c

HTML5 in theory supports drag and drop events, so I figured this would be simple, but it appears that elm-html does not expose these events.  It does however expose "on"  so I figured I could just copy a bit of code and use the native dragstart and drop events for my purposes.  It seemed easy enough to add the event however the drop event never fires. 

I found a jsFiddle example where drag drop works:

http://jsfiddle.net/6Lyk5kbh/0/ 

If I then modify the example and comment out the dataTransfer object in dragstart and run it again the drop event no longer fires here either: 

http://jsfiddle.net/6Lyk5kbh/1/

So I believe that if I am to use the native drag drop events then I also need to set a dataTransfer object in the dragstart event, but I'm not really sure how to do that in elm? 

Here is my attempt to use HTML5 events for drag drop in elm: 

https://gist.github.com/kurtharriger/9ed34731be0737b06a89

Thanks!

[Kurt Harriger]

Additionally was trying to inspect the value that was passed to dragstart function. Had some trouble figuring out how to use debug without actually using the result and came up with this: 

onDragStart addr msg =

  on "dragstart" value (\e -> Signal.message addr (fst (msg, Debug.log "e" e)))

This however generated a javascript exception. 

Uncaught TypeError: Cannot use 'in' operator to search for 'ctor' in null

It seems that whatever e is in this case, it doesn't appear to be useable from elm

[Ryan Rempel]

It looks like it would be impossible to implement native drag-and-drop without changes to elm-html.

The problem is that the Javascript side of things requires you to call `ev.dataTransfer.setData(...)`, where `ev` is the event which is supplied to your "dragstart" event handler, before the event handler returns. (Actually, I'm not entirely sure if it has to be before the event handler returns, but that's probably true ... even if it weren't, it doesn't fundamentally change the problem).

However, elm-html doesn't give you direct access to the event handler -- it constructs one, based on the options you supply, but none of the options currently will call dataTransfer.setData(...). (And, although you get the event object as a `Json.Decode.Value`, nothing about that will let you call an arbitrary Javascript function).

So, one way to make it work would be to add a new option to the `Options` type in Html.Events. Or, one might construct a specialized `onDragStart` function.

It looks as though the `dragover` event could be handled from elm-html, since the only interesting thing is whether to call `preventDefault()` or not, and elm-html can handle that.

The `drop` event would also seem to require some changes to elm-html, because you have to call a function on the event object -- i.e. `ev.dataTransfer.getData(...)`. Now, elm-html will give you the event as a `Json.Decode.Value`, so you can decode it etc. But I don't think that lets you call an arbitrary Javascript function ... that is, you could get the `dataTransfer` as a `Json.Decode.Value`, but you can't call a function on it.

So, the long and the short of it is that the native drag-and-drop events can't be used without changes to elm-html. Or, of course, other native code of one kind of another. Unless I'm missing something.

[Ryan Rempel]

You know, part of a possible solution for this would be a (possibly weird) enhancement to Json.Decode and Json.Encode, to deal with functions.

To back up a little, Json.Decode and Json.Encode are interesting for more than just what we traditionally think of as JSON. Instead, they are a kind of general-purpose way of moving values between arbitrary Javascript objects (Json.Encode.Value or Json.Decode.Value -- they are synonyms) and Elm types. And, that's exactly how elm-html uses them -- it supplies the event object, and we can construct Elm stuff from the event object by using Json.Decode.

However, Json.Decode and Json.Encode don't deal with functions ... that is, they aren't conscious of Javascript functions that might be attached to the Json.Encode.Value.

But, my intuition is that you could construct a perfectly reasonable API within Json.Decode for calling such functions. Here's a quick stab at it:

{-| Opaque type representing a Javascript function -}

type Function = Function

{-| Type representing a Javascript exception }

type Error = Error

... insert some API for extracting interesting stuff from an Error

{-| Extract a function -}

function : Decoder Function

{-| Call a function, using the supplied parameters. -}

call : List Json.Encode.Value -> Function -> Task Error Json.Decode.Value 

Now, I'm sure there would be some complications, but I'm pretty confident that I can see how to implement all of that (using native code, of course). And, it may be a bit weird, but it might even be Elm-ish -- really, the only un-Elmish thing about it is that the Function can't be guaranteed to be pure, but that's handled by making calling the Function a Task.

Now, to get weirder yet, you could even imagine a similar API on the Json.Encode side, couldn't you? Something like:

{-| Encode a function -}

function : Function -> Value

{-| Produce a Javascript function given the function body. -}

makeFunction : String -> Result Error Function

{-| Produce a function given an Elm implementation. -}

elmFunction : (List Json.Encode.Value -> Json.Decode.Value) -> Function

Again, I'm sure there would be some complications, but I'm pretty sure I can see how to implement this.

Now, if we had stuff like this, I think it would actually eliminate the need for some kinds of "native" code. If there is stuff in Javascript-land we want to get, we can use `makeFunction` and then `call` to use it. And, if we need to supply an Elm function to Javascript in a way that Javascript can call, we can use `elmFunction`. And the use of Json.Encode/Decode for parameters and return types covers the type issues. Now, of course calling these functions could produce errors. But the `call` Task can wrap exceptions and do the right thing. So, we wouldn't get unhandled runtime exceptions.

Actually, I suppose `makeFunction` could lead to unhandled runtime exceptions if the resulting function is handed off to javascript as a callback. However, I suppose that `makeFunction` could also wrap the function with some exception handling, so that exceptions are logged and don't crash. Or, one could leave out `makeFunction` -- it's probably the weirdest part, and the other stuff would be useful even without it.

Now, this is actually pretty far from the original post -- my apologies for that! But, it set off a train of thought I might try to explore further.

[Kurt Harriger]

Thanks for the quick feedback.  

I'm just getting started with elm so I think the Json.Encode/Decode functions solution is probably beyond my current skill level.  I could get by without drag drop for my toy application right now, but if I have a little extra time this weekend I'm pretty interested in learning more about the JS interop capabilities. 

My initial thought would be that I probably could just pass in some arbitrary value for the dataTransfer object just so that the drop event works and handle the actual data using Signal.forward (always ..) in elm instead.  Thoughts?  

I started looking at docs on interop and looks like interop is in theory to be done via ports, but ports appear to be expressly async in nature not sure they would work in this case?  

virtual-dom appears that it just imports Native.VirtualDom.function and I'm guessing this is synchronous introp with assumptions that native js code is exported correctly?  I don't really see this documented but I think I could probably figure it out by looking over some of the core libraries.  

I wouldn't mind sending a PR for additional drag drop events when I figure it out, but still not quite sure the most directionally correct way to approach this.  I think I saw something on an issue somewhere that there are plans to change how interop works in 0.17, but I haven't actually found much for details on the proposed changes.  I also saw that elm-html needs to be updated to use Signal.Message instead of Signal.Address -> a  to support changing of Signal.address to Signal.dispatch so still kinda on the fence if anything I do here would actually be relevant, but might be good learning experience regardless.

--
You received this message because you are subscribed to the Google Groups "Elm Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elm-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ryan Rempel]

Yes, my apologies again, what I posted wasn't really meant as a practical solution for your problem -- it's more of an interesting train of thought.

Essentially, you've come across some events that elm-html can't handle without changes, and the kind of solution I was thinking about is not a reasonable short-term solution -- it's more of a train of thought for a general class of things.

The problem with "pass in some arbitrary value for the dataTransfer object" is that I don't think the html5 apis allow you to provide your own dataTransferObject ... it is provided for you on the event object, and all you can do is call functions on it. Even if I'm wrong about that, Html.Events would allow you to get the event object, but not to change it ... especially, not before the event handler has returned.

So, I'm pretty sure that the short answer is "it can't be done without changes to elm-html" -- though perhaps others will have ideas.

[Ryan Rempel]

One additional thought.

In theory, there might be a way to make drag-and-drop work by using the "mousedown" "mousemove" and "mouseup" events with elm-html. There would be some potentially tricky aspects, in terms of which elements need which event handlers etc. and coordinating everything, but it's probably not impossible.

That is, the part that seems to need changes in elm-html is handling "dragstart" and "drop". If you're willing to work with "mousedown" "mousemove" and "mouseup", then theoretically elm-html should have what you need. However, there would be much more logic to implement in order to make it work.

[Janis Voigtländer]

I was trying to implement drag and drop using elm-html rather than the Graphics library.
I looked around a bit for some example code and found this library jvoigtlaender/elm-drag-and-drop recently mentioned here, but this seems to only work with the graphics elements not html. When I tried to use html elements rather than graphics library it did not seem to work any more. If the element was marked as draggable then it only fired nothing events, if I marked the element as draggable = false I would occasionally get lift and move by events but it was very inconsistent and mostly still did not work.

My attempt to use this library are here: https://gist.github.com/kurtharriger/abaa0cf4f08027b0b02c

The library jvoigtlaender/elm-drag-and-drop is specifically written for use with Graphics. It might be possible to also find a use case for it with elm-html, but you would need to use onHover instead of onClick events (what you did in your gist). And to make real use of it, you would need some free form positioning like Graphics has, but elm-html hasn’t.

The library is really for stuff like this (which was implemented as a student exercise using it; try to drag the nodes around). For use of HTML5 drag-and-drop events, you will have to use another approach.

See also this earlier comment.

​

--

[Kurt Harriger]

It turns out writing native js methods is pretty straightforward.  The biggest issue I had figuring out how to import native methods. It took me awhile to discover that I needed to enable native-modules in elm-package.json before it would load them them. 

Then after hard coding a call to e.dataTransfer.setData with some arbitrary values I discovered it still didn't work.  After taking another look at the example I realized my problem wasn't with dragStart at all, but that I needed to preventDefault using onDragOver.  This is mostly already supported with onWithOptions.  

Here is the revised gist with where the drop event actually fires.   

https://gist.github.com/kurtharriger/9ed34731be0737b06a89/revisions

[Daniel Bachler]

Kurt, if I understand you correctly you found a workaround for the problem of calling setData? Unless I am missing something that native code is not part of your gist, is it?

I am thinking about rewriting a js project I did in Elm and for that DnD would be essential, so I am very interested in any ways that would make this possible right now, without depending on major changes to Elm.Html.

[Kurt Harriger]

So yeah I found that it was unnecessary to call setData.  

The reason the drop event did not fire is because one must preventDefault behavior in the ondragover event of the drop target before the browser will fire the drop event.  This is possible using Html.Events.onWithOptions.  

At some point you'll actually want to know what was dropped, but you can easily add this data to the dragstart event handler and maintain this state in elm.  So I don't think you actually need to setData in practice.  If you are looking for a bit more than what is in the gist you can look at how I used it to allow users to setup the game board in my battleship game here: 

https://github.com/kurtharriger/elm-battleship

The drag-n-drop experience is still not perfect however.  I did notice that the default drag and drop behavior inserts a little (+) icon over the image being dragged which in this case is probably undesired but still functional. 

Additionally, the drag start and drop occurs at the cursor position. Try to grab a ship from the end and drop it on the gameboard, when it drops it drops into the square your mouse is over not into the square at upper left corner of the image.  Ideally I would fix this by changing dragstart to set upper left of the drag image to be at the cursor position so that it drops where you would expect.  

Adding native code was not all that hard to do, so when I have some time I'll finish updating my battleship example with with some native code to fix the 2 issues I mentioned above and post link to result here with result when done. 

[Kurt Harriger]

Here is quick update that modifies the dataTransfer object using native code:

https://github.com/kurtharriger/elm-battleship/commit/d523653b8ac4ec463bd188ef41f0ff864ca6d5a9

Perhaps ideally one might just specify the desired mutations to apply in via options to make it more general. That would be pretty simple to do with simple properties like effectAllowed, but if you want to do something crazy with the dragimage you might still need native code. 

[Daniel Bachler]

Oh I didn't realize the call to setData was redundant. That should make the whole architecture quite a bit easier. Thanks for the example with the native code, that might still come in handy at some point!