API URLs work in browsers but NOT in remote app...

[Thomas Albanese]

Hi Folks,

I haven't beat my head against a wall like this in some time.  I have a perplexing and maddening problem.

I have googled my brains out for three (3) days.

SYSTEMS INVOLVED:

Initially I was running Win2008 IIS7 with Railo 3.3 over Tomcat & idapi dll that comes with it.

Now, after troubleshooting, I am running the server with Railo 4.0.3 and it's revised IIS connector.  (It made not difference with regard to this problem).

I am using Taffy 1.3 beta. 

FIRST PROBLEM:  I had the problem where the slashes in the query of the URLs was causing blank output in browser and/or other errors when I tested even though it worked in the dashboard mock client.

     After searching the net for various fixes and tweaks, I was able to get my API URLs to now work in the browser whether I use the .../index.cfm/... OR the /index.cfm?endpoint= methods.

Of course I was thrilled after two days of hard effort. HOWEVER, much to my chagrin, when I went to test the mobile client application I am developing...

SECOND AND CURRENT PROBLEM:  When I make the HTTP Client api request (using Appcelerator's Titanium 3.0 which is javascript based) it comes back with 400 Bad Request with no further detail.

  AGAIN, I am using the EXACT same URL on both sides.

1. This Works:  API URLS entered directly in any PC browser, Mac or Windows
2. This Works:  The Dashboard Mock Client
3. This DOES NOT Work:   Taking that same URL and executing via the httpclient methodology in my mobile app on either iOS or Android.      
4. But This Works:  API URLS entered in any mobile device or emulator browser that I have tried.  They all work.
5. And This Works:  Entering other URLs in the mobile app including the dashboard come back with no error in the mobile app. In other words, URLs that don't have slashes in the query string.

NOTE:  The same symptom presents using either URL endpoint convention.  Same behavior either way.

NOTE:  The same symptom presents whether I use the 8888 port to go directly to Railo or whether I go through the standard IIS port 80.  Same behavior either way.

WHAT I HAVE TRIED:

1. TWEAKS:  
• Did various suggestions made online with regarding to the web.config on the IIS side.  That helped fixing the first problem but not the second.
  
• Did various suggestions made regarding the Tomcat INIT tweaks regarding IGNORING slashes in the URI query string.  None have had any effect whatsoever.
2. Upgraded Railo - had no effect at all.
   
3. Upgraded Taffy - 1.3 has the ability use the endpoint as a parameter but I get the error either way.
4. Implemented various URL Rewrites that, again, work in the browser and dashboard mock client but NOT the client app.
5. LOGS: View the Trace log for failed URL requests.  The problem with this is that it appears to give a lot of detail but when you get down to where the error is, it just says what I already knew: 400 Bad Request although it does say that the "module name" is "ManagedPipelineHandler".
• On Railo 3.3 the log said the error occurred in the connector on the Railo/Tomcat side.
• On Railo 4.0.3 the log says the error occurred in the AspNetHttpHandler.
6. Added various HTTP headers thinking that the mobile app was not supplying something that the browser does.  None of these made any difference.
7. I disabled anonymous authentication in IIS7 and that did return a different error to the mobile app ("unauthorized").  I turned it back on and get the "bad request" again.
8. I tried GET and POST, with and without json send parameters.
9. I tried other URLs from the HTTPClient tools within the titanium code and those all worked.
10. The differences in the web log entries for the various clients are:
• PC Firefox (result 200 OK):    
  
• 2013-03-09 22:37:49 W3SVC1 S15951825 216.250.115.211 GET /api/index.cfm endpoint=/swatchsets/62001 80 - 67.174.76.227 HTTP/1.1 Mozilla/5.0+(Windows+NT+6.1;+WOW64;+rv:19.0)+Gecko/20100101+Firefox/19.0 CFID=45e2e1d2-098b-47da-945d-085bd9fa8edd;+CFTOKEN=0;+RAILO_ADMIN_LANG=en;+RAILO_ADMIN_LASTPAGE=services.datasource - swatchman.servatus.biz 200 0 0 3589 461 656
• 2013-03-09 22:38:53 W3SVC1 S15951825 216.250.115.211 GET /api/index.cfm/swatchsets/62001 - 80 - 216.250.115.211 HTTP/1.1 Mozilla/5.0+(Windows+NT+6.1;+WOW64)+AppleWebKit/537.22+(KHTML,+like+Gecko)+Chrome/25.0.1364.152+Safari/537.22 RAILO_ADMIN_LANG=en;+RAILO_ADMIN_LASTPAGE=services.datasource;+CFID=e86a500d-5d58-4e7c-8f30-1dc4aa41e33f;+CFTOKEN=0 - swatchman.servatus.biz 200 0 0 3613 540 238
  
• iPhone Safari (result 200 OK):
  
• 2013-03-09 22:38:53 W3SVC1 S15951825 216.250.115.211 GET /api/index.cfm/swatchsets/62001 - 80 - 216.250.115.211 HTTP/1.1 Mozilla/5.0+(Windows+NT+6.1;+WOW64)+AppleWebKit/537.22+(KHTML,+like+Gecko)+Chrome/25.0.1364.152+Safari/537.22 RAILO_ADMIN_LANG=en;+RAILO_ADMIN_LASTPAGE=services.datasource;+CFID=e86a500d-5d58-4e7c-8f30-1dc4aa41e33f;+CFTOKEN=0 - swatchman.servatus.biz 200 0 0 3613 540 238
• Mobile App (result 400 Bad Request)
• 2013-03-09 22:41:57 W3SVC1 S15951825 216.250.115.211 GET /api/index.cfm endpoint=/swatchsets/62001 80 - 67.174.76.227 HTTP/1.1 SwatchMan/1.1.0+CFNetwork/609.1.4+Darwin/12.2.0 CFID=1945a41f-9e6a-4ee5-8642-1a80fc5a45bb;+CFTOKEN=0 - swatchman.servatus.biz 400 0 0 257 287 109

IN A NUTSHELL:

1. The problem clearly revolves around the "/" (slash) in the query URI convention that is implemented by Taffy and it does not appear to be fully addressed by the new endpoint variation.  Again, all that works in the browser but not the app.
2. There is obviously some difference between how the server is seeing a straight browser GET versus what the mobile app is sending in terms of either headers or cookies or whatever. 
   
3. The problem occurs when either Tomcat is parsing the URL when IIS is preparing it.  
4. What is so vexing is that the problem appears, for all the world, to be the noted SLASH problem but if that were the case, the same problem would show on the regular browser clients, NO?
5. So the fixes I have done have fixed the SLASH problems but not when using a remote mobile app.

I can supply my Titanium javascript code, if necessary, but it is not any different than the hundreds of examples one can google on the net.  I have duplicated each minor variation here and there on the next with absolutely NO Effect whatsoever.

I am going to inquire at Appcelerator also and see if they have run into this before.  I can't believe they haven't but, for the life of me, I cannot find anything out there about this specific issue.

Thanks to all in advance for any ideas you all may have.

Tom

[Thomas Albanese]

Oh, also, I have not implemented any real security other than parameters themselves as far as that goes.

[denstar]

On 3/10/13 3:32 PM, Thomas Albanese wrote:
> Hi Folks,
>
> I haven't beat my head against a wall like this in some time. I have a
> perplexing and maddening problem.

Whenever I encounter something like this, and I've racked my brain and
come up empty, I drop to watching the network traffic.

Eclipse has a decent TCP Monitor plugin (not sure which tooling it is a
part of-- Test Platform maybe) that lets you set up proxies and monitor
traffic through them, and compare the requests. It's quick to set up a
proxy, and I don't leave the IDE.

There's Wireshark on windows, and a grip of tools for other OSes.
Sometimes a quick tcpdump is enough to see something.

And sometimes seeing the traffic is not helpful at all.

Just tossing it out there, couldn't think of anything else. :)

:den

--
When tempted to fight fire with fire, remember that the fire department
generally uses water.

[Thomas Albanese]

I have figured out the problem and will post it as a bug in the bug area.

[Adam Tuttle]

Sorry for my inactivity here on the list. Been a bit swamped by life lately.

If someone wants to send a pull request for this bug, I'd be happy to review it! :)

Adam

Thomas Albanese wrote:

I have figured out the problem and will post it as a bug in the bug area.

On Sunday, March 10, 2013 5:32:53 PM UTC-4, Thomas Albanese wrote:

    Hi Folks,

    I haven't beat my head against a wall like this in some time. I
    have a perplexing and maddening problem.

    I have googled my brains out for three (3) days.

    *SYSTEMS INVOLVED:*
    Initially I was running *Win2008 IIS7 with Railo 3.3 over Tomcat &
    idapi dll* that comes with it.
    Now, after troubleshooting, I am running the server with *Railo
    4.0.3 and it's revised IIS connector*. (It made not difference

    with regard to this problem).
    I am using Taffy 1.3 beta.

    *FIRST PROBLEM:* I had the problem where the slashes in the query
    of the URLs was causing *blank output in browser and/or other
    errors* when I tested even though *it worked in the dashboard mock
    client*.

    After searching the net for various fixes and tweaks, I was able

    to get my API URLs to *now work in the browser whether I use the
    .../index.cfm/... OR the /index.cfm?endpoint= methods.*
    *
    *

    Of course I was thrilled after two days of hard effort. HOWEVER,
    much to my chagrin, when I went to test the mobile client
    application I am developing...

    *SECOND AND CURRENT PROBLEM:* When I make the HTTP Client api

    request (using Appcelerator's Titanium 3.0 which is javascript

    based) it comes back with *400 Bad Request* with no further detail.

    AGAIN, I am using the EXACT same URL on both sides.

         1. *This Works*: API URLS entered directly in any PC browser,
            Mac or Windows
         2. *This Works: *The Dashboard Mock Client
         3. *This DOES NOT Work: *Taking that same URL and executing
            via the httpclient methodology *in my mobile app* on
            either iOS or Android.
         4. *But This Works*: API URLS entered in any mobile device or

            emulator browser that I have tried. They all work.

         5. *And This Works:* Entering other URLs in the mobile app

            including the dashboard come back with no error in the

            mobile app. In other words,*URLs* that don't have slashes

            in the query string.

    NOTE: The same symptom presents using either URL endpoint
    convention. Same behavior either way.
    NOTE: The same symptom presents whether I use the 8888 port to go
    directly to Railo or whether I go through the standard IIS port
    80. Same behavior either way.

    *WHAT I HAVE TRIED:*

     1. *TWEAKS:*
          * Did various suggestions made online with regarding to the
            *web.config* on the IIS side. That helped fixing the first

            problem but not the second.

          * Did various suggestions made regarding the Tomcat INIT

            tweaks regarding IGNORING slashes in the URI query string.
            None have had any effect whatsoever.

     2. Upgraded Railo - had no effect at all.
     3. Upgraded Taffy - 1.3 has the ability use the endpoint as a

        parameter but I get the error either way.

     4. Implemented various URL Rewrites that, again, work in the

        browser and dashboard mock client but NOT the client app.

     5. LOGS: View the Trace log for failed URL requests. The problem

        with this is that it appears to give a lot of detail but when
        you get down to where the error is, it just says what I
        already knew: 400 Bad Request although it does say that the
        "module name" is "ManagedPipelineHandler".

          * On Railo 3.3 the log said the error occurred in the

            connector on the Railo/Tomcat side.

          * On Railo 4.0.3 the log says the error occurred in the
            *AspNetHttpHandler.*
     6. Added various HTTP headers thinking that the mobile app was

        not supplying something that the browser does. None of these
        made any difference.

     7. I disabled anonymous authentication in IIS7 and that did

        return a different error to the mobile app ("unauthorized"). I
        turned it back on and get the "bad request" again.

     8. I tried GET and POST, with and without json send parameters.
     9. I tried other URLs from the HTTPClient tools within the

        titanium code and those all worked.

    10. The differences in the web log entries for the various clients
        are:
          * PC Firefox (result*200 OK*):
              * 2013-03-09 22:37:49 W3SVC1 S15951825 216.250.115.211

                GET /api/index.cfm endpoint=/swatchsets/62001 80 -
                67.174.76.227 HTTP/1.1
                Mozilla/5.0+(Windows+NT+6.1;+WOW64;+rv:19.0)+Gecko/20100101+Firefox/19.0
                CFID=45e2e1d2-098b-47da-945d-085bd9fa8edd;+CFTOKEN=0;+RAILO_ADMIN_LANG=en;+RAILO_ADMIN_LASTPAGE=services.datasource
                - swatchman.servatus.biz

                <http://swatchman.servatus.biz>*200* 0 0 3589 461 656
              * 2013-03-09 22:38:53 W3SVC1 S15951825 216.250.115.211

                GET /api/index.cfm/swatchsets/62001 - 80 -
                216.250.115.211 HTTP/1.1
                Mozilla/5.0+(Windows+NT+6.1;+WOW64)+AppleWebKit/537.22+(KHTML,+like+Gecko)+Chrome/25.0.1364.152+Safari/537.22
                RAILO_ADMIN_LANG=en;+RAILO_ADMIN_LASTPAGE=services.datasource;+CFID=e86a500d-5d58-4e7c-8f30-1dc4aa41e33f;+CFTOKEN=0
                - swatchman.servatus.biz

                <http://swatchman.servatus.biz>*200* 0 0 3613 540 238
          * iPhone Safari (result *200 OK*):
             o
              o 2013-03-09 22:38:53 W3SVC1 S15951825 216.250.115.211

                GET /api/index.cfm/swatchsets/62001 - 80 -
                216.250.115.211 HTTP/1.1
                Mozilla/5.0+(Windows+NT+6.1;+WOW64)+AppleWebKit/537.22+(KHTML,+like+Gecko)+Chrome/25.0.1364.152+Safari/537.22
                RAILO_ADMIN_LANG=en;+RAILO_ADMIN_LASTPAGE=services.datasource;+CFID=e86a500d-5d58-4e7c-8f30-1dc4aa41e33f;+CFTOKEN=0
                - swatchman.servatus.biz

                <http://swatchman.servatus.biz> *200 *0 0 3613 540 238
          * Mobile App (result *400 Bad Request*)
             o
              o 2013-03-09 22:41:57 W3SVC1 S15951825 216.250.115.211

                GET /api/index.cfm endpoint=/swatchsets/62001 80 -
                67.174.76.227 HTTP/1.1
                SwatchMan/1.1.0+CFNetwork/609.1.4+Darwin/12.2.0
                CFID=1945a41f-9e6a-4ee5-8642-1a80fc5a45bb;+CFTOKEN=0 -
                swatchman.servatus.biz

                <http://swatchman.servatus.biz>*400 *0 0 257 287 109

    *IN A NUTSHELL:*

     1. The problem clearly revolves around the "/" (slash) in the

        query URI convention that is implemented by Taffy and it does
        not appear to be fully addressed by the new endpoint
        variation. Again, all that works in the browser but not the app.

     2. There is obviously some difference between how the server is

        seeing a straight browser GET versus what the mobile app is
        sending in terms of either headers or cookies or whatever.

     3. The problem occurs when either Tomcat is parsing the URL when
        IIS is preparing it.
     4. What is so vexing is that the problem appears, for all the

        world, to be the noted SLASH problem but if that were the
        case, the same problem would show on the regular browser
        clients, NO?

     5. So the fixes I have done have fixed the SLASH problems but not

        when using a remote mobile app.

    I can supply my Titanium javascript code, if necessary, but it is
    not any different than the hundreds of examples one can google on
    the net. I have duplicated each minor variation here and there on
    the next with absolutely NO Effect whatsoever.

    I am going to inquire at Appcelerator also and see if they have
    run into this before. I can't believe they haven't but, for the
    life of me, I cannot find anything out there about this specific
    issue.

    Thanks to all in advance for any ideas you all may have.


    Tom

--
You received this message because you are subscribed to the Google
Groups "Taffy Users" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to taffy-users...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.