MS Edge support works only icons are broken

[marian lux]

It seems that mgwt works out of the box with Microsofts new Edge browser (included in Windows 10). There is only one issue. Icons and Iconbuttons will not shown correctly. 

I tested this with your project (which works with the current mgwt master branch) and get the same issues as shown in the following mgwt showcase screenshot (http://mobilegwt.appspot.com/showcase/):

The iconbutton sample application (https://github.com/ruhaim/mgwt-imagebuttons-demo http://mgwt-2-0-image-buttons.appspot.com/) is also not working on Edge as you can see in this screenshot:

Mgwt has a new color approach for image buttons which allows to set the color of an icon without changing the icon-image (https://github.com/mgwt/mgwt/wiki/ImageButton). I think this is broken with Edge.

Can anyone post a workaround for this or if someone has the deep understanding how the color approach works, could fix this in mgwt and make a pull request?!

Thank you very much.

Marian

[paul]

Unfortunately I have windows professional 8.1 and it is horrible. I haven't managed to install a windows 10 VM to test the IE edge browser.

I'll borrow a windows 7 laptop and try again.

Just one thought (I assume ie edge supports viewing source) Could you set a break point in ImageConverter.loadImage to see if it is called.

Also set a break point on img.onload to see if the callback is called.

I'm wondering whether we need to add the Image to the DOM to get it to load the dataURL ??

Make sure you have the meta tag ie compatibilty ie10 set in the host page otherwise the os detection is all wrong.

Cheers

[marian lux]

Hi Paul,

thank you very much for looking at this.
If I understand you right I should set a break point in SDM on ImageConverter.loadImage.
I use a external test device with Windows 10 edge browser where I have no access to install my IDE environment.
Trying to call SDM with the external IP on the Win 10 device is not working. The only IP which works is 127.0.0.1  on SDM. Is there a way to test SDM on another device IP inside my local network? How to figure out this. Then I could set the break point as you mentioned.

Thanks,
Marian

[Jordi G. March]

Hi Marian,

do you have these lines on your XML?

  <set-configuration-property name="mgwt.superdevmode" value="on" />
  <set-configuration-property name="devModeUrlWhitelistRegexp" value=".*" />
  <set-configuration-property name="mgwt.superdevmode_host" value="http://192.168.7.193:9876" />

   

[paul]

In eclipse for example we just add to the run configuration on the  "Arguments" tab:

-bindAddress 0.0.0.0

That way SDM will start up and listen on all interfaces and not just 127.0.0.1 on your server/desktop/laptop

So you can then use SDM from any device that has access to your application.

What Jordi mentions, I think was a convenience method in mgwt to use SDM when using GWT 2.6. We just do the above for GWT 2.7.

Cheers

[Jordi G. March]

Oups, yes, forgot to mention the bindAddress.... Yes, we still use 2.6...

[marian lux]

Thank you.
Remote debugging works now after I added

-bindAddress 0.0.0.0

The in ImageConverter.java, the native JSNI method loadImage(String dataUrl, int width, int height, LoadImageCallback callback) will be called only if I am on IE 11. In Google Chrome it will be never called.

Hope this helps.
Regards Marian

Am Mittwoch, 9. September 2015 16:50:55 UTC+2 schrieb paul:

[paul]

....but on IE Edge is onSuccess in the below called? I'm guessing not.

    loadImage(resource.getSafeUri().asString(), width, height, new LoadImageCallback() {

      @Override

      public void onSuccess(ImageElement imageElement) {

        

        Canvas canvas = Canvas.createIfSupported();

        canvas.getElement().setPropertyInt("height", height);

        canvas.getElement().setPropertyInt("width", width);

        Context2d context = canvas.getContext2d();

        context.drawImage(imageElement, 0, 0);

        ImageData imageData = context.getImageData(0, 0, width, height);

        CanvasPixelArray canvasPixelArray = imageData.getData();

        for (int i = 0; i < canvasPixelArray.getLength(); i += 4) {

          canvasPixelArray.set(i, red);

          canvasPixelArray.set(i + 1, green);

          canvasPixelArray.set(i + 2, blue);

          canvasPixelArray.set(i + 3,

          canvasPixelArray.get(i + 3));

        }

        context.putImageData(imageData, 0, 0);

        imageConverterCallback.onSuccess(new ConvertedImageResource(

                canvas.toDataUrl("image/png"), resource.getWidth(),

                resource.getHeight()));

      }

    });

  }

[marian lux]

Sorry. I forgot to mention MS Edge in my last post.

After a retest:
onSuccess is called only on IE11.
MS Edge is not called, as you said. (but also on Chrome browser onSuccess is not called)

[paul]

Okay managed to take a quick look on a windows 10 vm on a colleagues laptop.

It is an evaluation copy so these results may change.

Like you said showcase works pretty well apart from icons. The reason is it is using the safari permutation.

MGWT assumes safari (webkit) can support masked icons which ie10/11 does not and I assume IE edge does not either.

When I add the meta tag...

 <meta http-equiv="x-ua-compatible" content="IE=10" />

IE Edge seems to ignore it and is still identified as safari by MGWT.

I've just read IE Edge supports touch events on mobile (as well as pointer events) so longer term it maybe easier to use the safari permutation for IE Edge. However as always there is bound to be differences.

I'm sure we can fix mgwt in IE edge desktop for you. Give us a few hours.

As for IE Edge on windows mobile 10 we can deal with that another day (It may just work but we will see!!)

[paul]

The pull request

https://github.com/mgwt/mgwt/pull/262

...although not merged into the main branch yet holds additional fixes to the WP8/8.1 work I have done.

I have added a fix for IE Edge (desktop) for the icons. I've added a runtime detection method for IEEdge to OsDetection and so now emulate the IconHandler the same way as done on IE10/11

However I leave IEEdge to run as the safari permutation which seems to work so far.

I gave showcase a quick test and all looks good.

You also no longer need the meta tag ...

<meta http-equiv="x-ua-compatible" content="IE=10">

...in your host page if you merge in this pull request since I added support for the latest pointer event model in IE11

Cheers

[marian lux]

Thank you so much paul for your effort! I don't want to wait until your extreme useful pull requests find their way into the official master branch. (I think Daniel has no time because Google and the current GWT development steals his time. Hope he will spend some more time for this project in future.) So I am working now with my own local git repo where your (and other useful) pull requests are merged in my local mgwt master branch until official support follows.

I tested it against IE 11 and MS Edge.

All seems to work fine expect one thing. The search-icon in the mgwt serachbox is not displayed correct on MS Edge browser on Windows 10.

See the following screenshot (taken with showcase):

Could you fix this, paul?

THX Marian

[paul]

Okay, did a quick fix for ie10/11 and WP8.1

I've committed to the same pull request above.

Does not work for IEEdge. The way the widget has been designed it gets it's images from css which is for safari and so applies webkit mask. We will need to move the images out of css and into the widget and then use the IconHandler so we can emulate as we do now else where.

I don't have time right now.

Shorter term, you could apply a different appearance to the widget at runtime (using MGWT.getOsDetection().isIEEdge() ) and just apply css using the ie10 settings from the searchbox.css file

Cheers

[marian lux]

If you add/alter the following in your searchbox.css it works on all bowsers (IE10,11 IEEdge, Mobile Safari, Chrome):

/*ie10 + 11 support*/
@if user.agent ie10 {
  .mgwt-SearchBox-input {
    top: 10px;
  }
}

@if user.agent ie10 {
  .mgwt-SearchBox-icon {
    background-image: searchImage;
    background-repeat: no-repeat;
    background-color: inherit;
  }
}

/*IEEdge fix*/
@if user.agent safari {
  .mgwt-SearchBox-icon {
    -webkit-mask-image: searchImage;
    -webkit-mask-repeat: no-repeat;
   
    background-image: searchImage;
    background-repeat: no-repeat;
    background-color: inherit;
  }
}

Could you please make a new commit on your pull request for this if it also works on your project/test environment (showcase)?

[paul]

The below partially breaks for safari (chrome, android etc), the colour being masked through is the inherited background colour i.e white

I've put in a runtime check into the css for IEEdge and so do not affect other safari permutations that do support -webkit-mask-image

I've committed to the pull request.

Cheers

[marian lux]

Thank you very much. I will test it with my project!

By the way: I did not know that the following line is possible in GWT - CSS:

@if (com.googlecode.mgwt.ui.client.MGWT.getOsDetection().isIEEdge()) {

Very nice to know this.

[Marc]

Life is good! I resolved the whole icons issue... testing it still but hopefully it can work.

[marian lux]

It seems that the pull request 262 - which is merged now into the master branch - works no longer on MS Edge browser (but it works on IE 11).

I can see see the black squares (as showed in my first post) instead of the images on mgwt showcase, compiled with the current master branch (and also in my mgwt project).

I also tried to add:

<meta http-equiv="x-ua-compatible" content="IE=10">

But the issue is still there.

Paul, do you know what could be responsible for this?
I also filed a bug for this: https://github.com/mgwt/mgwt/issues/282

THX
Marian

[paul]

Will take a look later today

[marian lux]

Hi Paul,

today I did a little investigation and testing on this (broken Icons on MS Edge) and I found out the following:

It is a problem that appears after MS Edge update. The current version is not working any more.

My first test was on MS Edge 20.10240.16384.0
The method onSuccess will be called (see code from ImageConverter.java in mgwt below).

My second test was on MS Edge 25.10586.0.0 after Upgrading Windows 10 to the current version.
The method onSuccess will never called (see code from ImageConverter.java in mgwt below).

So it seems there was a change on MS Edge update.

@Paul, I think you know how to fix this. Would it be possible for you to make an another pull request for fixing this issue??

loadImage(resource.getSafeUri().asString(), width, height, new LoadImageCallback() {

      @Override

     public void onSuccess(ImageElement imageElement) {

       

       Canvas canvas = Canvas.createIfSupported();

       canvas.getElement().setPropertyInt("height", height);

       canvas.getElement().setPropertyInt("width", width);

        Context2d context = canvas.getContext2d();

       context.drawImage(imageElement, 0, 0);

       ImageData imageData = context.getImageData(0, 0, width, height);

        CanvasPixelArray canvasPixelArray = imageData.getData();

        for (int i = 0; i < canvasPixelArray.getLength(); i += 4) {

         canvasPixelArray.set(i, red);

         canvasPixelArray.set(i + 1, green);

         canvasPixelArray.set(i + 2, blue);

         canvasPixelArray.set(i + 3,

         canvasPixelArray.get(i + 3));

       }

       context.putImageData(imageData, 0, 0);

       imageConverterCallback.onSuccess(new ConvertedImageResource(

               canvas.toDataUrl("image/png"), resource.getWidth(),

               resource.getHeight()));

     }

   });

 }

THX
Marian

[marian lux]

The User Agent changes on MS Edge from 12 to 13.
After changing the Line #161 in OsDetectionRuntimeImpl.java to

    if (userAgent.contains("edge/12") || userAgent.contains("edge/13")) {

it works again on MS Edge.

Is there a better / more elegant solution for this? So we have to update / make changes in the mgwt framework after every mayor edge update.

[paul]

I'm sure there is a more elegant solution.Will take a look ASAP

[marian lux]

Maybe we could design mgwt without looking for user agents? Maybe we can look for missing functionality and handle it until it's done on all browsers ... something like the intention of Phonegap ... If it works, we don't need the check it any more and we could remove it from the framework.

Have you tried to run the showcase on a Windows 10 Touch Device on MS Edge? I found out that scrolling is not working without mouse (Touch gestures are not scrolling as on Google Chrome on Windows 10) on our mgwt project.

[paul]

Without user agents etc how do you decide what code to include into which permutation? There is a trade off as always, code size will increase if you want to do runtime feature detection.

For the IEEdge user agent issue it is sufficient to simply test for "edge/". I'll create a pull request shortly once I work out a few other things....

For devices that support touch and mouse simultaneously it gets tricky. I've had a quick play:

I can get chrome on windows 10 to work fine for touch and mouse. Basically we have to cater for both touch and mouse events and not one or the other. However if touch is used we need to do a touchEndEvent.preventDefault() to stop the mouse events firing otherwise you get double firing of the event handlers. There is a still an issue with event capture which I'm looking at where for example the slider does not work very well with the mouse.

For IEdge on windows 10, you are correct, mouse and touch work fine, but touch sliding does not. I've not dug into this yet. It should be easier to fix since we can hook into standard pointer events on IEEdge which are the same for both mouse and touch interaction. 

Sorry been a bit quiet lately, just too much work going on at the moment! I'll do my best to follow all this up shortly.

Ta

[paul]

Hello Marian,

Give https://github.com/paulfrench/mgwt/tree/touch_and_mouse

...a try and let me know. Showcase seems to work okay on the following:

windows 10 (chrome, ie11 and edge) - device supports touch and mouse (either can be used)

I did a quick regression test of chrome on windows 8.1 desktop as well.

I don't have time to do exhaustive testing. Can you try the branch above and let me know if it solves your issues? One word of warning I have had to introduce another binding to distinguish between browsers that use the pointer model and those that do not "mgwt.pointermodel". I'll turn this into a runtime config parameter at a later date. 

[marian lux]

Hi Paul!

Thank you so much. I will test it ASAP and report to you!

[marian lux]

Hi Paul,

I think your branch works fine. I tested it on:
- Android 6 - Chrome
- iOS 9.3.1 Mobile Safari
- Win8 IE 11
- Win8 Chrome
- Win10 Edge
- Win10 Chrome
- Linux Chrome

It seems that all works fine. Expect one thing (but this appeared also before):
Win8 IE 11 and Win10 Edge don't handle mouse wheel scrolling!
If this would be possible, the desktop support would be fine.

But THX again Paul! And maybe you have a solution for the scrolling handling (issue) on Windows Browsers (IE 11 / Edge) that it works as on Google Chrome?!

[paul]

It needs more thinking still. I've broken other functionality like clicking on a MListBox using touch. Since I call TouchEndEvent.preventDefault() this stops mouse events firing but also stops the browser doing other default behaviour like showing the list in a pop up. Same issue for MDateBox. Will try a few things when I get a chance.

[marian lux]

Okay. I tested it on my project which does not contain a MListBox.
To get a more complete picture, I will also test it against Showcase with our test devices.

[marian lux]

I figured out that also MSearchBox seems to be broken.

When a user tries to click on an  MSearchBox, the soft keyboard will never pop up so it is not possible (on Android 6 Chrome + iOS 9 Safari) to enter some text into the search field.

Am Montag, 25. April 2016 19:40:35 UTC+2 schrieb paul:

[paul]

MGWT wrongly assumes that if the browser has ontouchstart defined in the documentElement then it must be a touch device. Chrome on windows 10 desktop has ontouchstart defined even when the device does not support touch. Hence you get a mgwt showcase application you cannot use if the device hardware does not support touch. You can still use IE Edge okay.

Currently MGWT will bind touch handlers to touch events or mouse events, but not both. So we have a problem when we consider a device that supports both mouse and touch simultaneously (more common these days).

So firstly MGWT will need to listen to both mouse and touch events in case you use touch one minute and then your mouse next. Plus I don't think it is possible to reliably detect at runtime if a device actually does support touch. You still just register you usual touch handlers and MGWT will call your handler as expected regardless if you use touch or the mouse. Due to compatible mouse events being fired on a touch device after a touch we need to make sure the mgwt touch handlers are only called once. If you use your mouse then all is fine since no touch events are fired.

On a touchstart, touchmove or touchend event we could call preventDefault and this stops any compatible mouse events being fired. That would be a good solution if it did not also stop other desired default browser behaviour happening e.g scrolling, text selection, showing a drop down list etc. Hence this is not an option!

The only solution I can think of would be to call stopPropagation and preventDefault on all compatible mouse events fired (apart from the mouse click which we class as a gesture). The compatible mouse events we call stop and prevent on are:

"mousedown"

"mousemove"

"mouseup"

"mouseover"

"mouseout"

So what we need to do is when the primary touch has changed (the only touch that can fire compatible mouse events) we ensure we stop propagation and prevent default on the above events that immediately follow the primary touch change.

At a guess we may need to record the primary touch ID in progress and stop/prevent all mouse events for a fixed amount of time after??

Anyway, just letting you know where I am up to. I'll have to park it for now.

[paul]

Hello Marian,

Hello Marian,

Give it another go https://github.com/paulfrench/mgwt/tree/touch_and_mouse

We should be a lot closer to something that works on all devices/browsers and supports touch and mouse simultaneously

Cheers

[marian lux]

Hi Paul,

thank you very much for working on this!

My first test results:
I checked out your your newest branch below but it seems that it i figure out the same problem as on current mgwt master branch:
On my Windows 10 tablet on IE Edge 13.x, scrolling works only with mouse. Touch scrolling (on scrollpanel) is not working. When I try to scroll, the whole content of the window is moving and not the scroll panel. So it seems the event propagation is not working correctly.
On Google Chrome all works as it should!

[paul]

I've dug out a windows10 laptop with EdgeHTML 13.10586 (if that helps) and it works fine for me apart from scrolling with the mouse wheel. I can scroll using the mouse and via touch.

I'm running mgwt showcase.

Have you added ie10 as user agent permutation to ShowCase.gwt.xml ? This shouldn't make any difference in this case.

  <set-property name="user.agent" value="safari,ie10" />

I've also updated the showcase pom to use java 7 and gwt 2.7

When you run showcase in super dev mode (windows 10 IEEdge13) do you see:

      binding: mgwt.density=mid (could be high or xhigh for you)

      binding: mgwt.formfactor=desktop

      binding: mgwt.pointermodel=pointer

      binding: user.agent=safari

If you don't then have you <meta http-equiv> tag specifying a version of IE in the showcase index.html file? If so you need to remove it (we originally required it a while back to get ie11 to work but it is no longer required) 

[marian lux]