WiFly RN-134 board (not the sparkfun shield) problem

[trandi]

Hi guys,

I usually hate bothering others with my technical issues so I tend to
search and try by myself until I find a solution... however this time
I can't take anymore, I've already spent more than an entire week-end
on this "silly" problem ....
So here I am, maybe somebody on this group can hep... (by the way, I
couldn't find a dedicated WiFly forum, am I missing something ?)


I'll start straight away buy insisting that my question / problem is
about the WiFly RN-134 board, which contains the SAME RovingNetworks
131 module but its is NOT the Sparkfun Arduino shield.

I'm using (or trying to :) ) this with an Atmega 168 micro
controller, same as for the Arduino but programmed directly with avr-
gcc.

I bought last week a RN-134 evaluation board (from mouser
http://uk.mouser.com/Search/ProductDetail.aspx?R=RN-134virtualkey66230000virtualkey765-RN-134
) for my project (here you can have a look at various stuff I blog
about : http://trandi.wordpress.com )

It came with firmware version 2.23 and after several days of
frustration I've updated it to 2.28 which seems to be the latest, but
is has NOT changed anything to my problem(s).

I basically need to connect to a http web-service on the internet to
get some weather information: http://weather.yahooapis.com/forecastrss?w=44418

I initially connected to the board (through the TTL serial port using
a (FTDI 3.3V cable) and TeraTerm.), set up the IP protocol at 18 to
support HTTP beyond IP, configured the wireless ssid and
authentication and SAVED everything.

Everything is ok, I get into command mode (with "$$$") and I type
this:


set dns name weather.yahooapis.com \n
set comm remote GET$/forecastrss?w=44418 \n
open \n \n

And then I get the XML file AS EXPECTED.
So this proves that the whole configuration is WORKING OK! The board
is able to go on the internet and download the requested XML from
yahoo's website...


Now, here are my 2 issues, quite independent of each other, or so I
think:

1) If I do

set dns name weather.yahooapis.com \n
set comm remote 0 \n
open \n
GET /forecastrss?w=44418 \n

Then it does NOT work. It complains about "Yahoo! - 404 Not Found".
Exactly as if the actual page name sent had something added or
substracted from it...


2) If I take the snippet that does work from TeraTerm and I send the
exact same instructions from my Atmega 168 micrcontroller, all the
commands work ok (I can get the AOK back, and even the "*OPEN*") I get
again "Yahoo! - 404 Not Found" !!!

I can actually parse the 404 message on the microcontroller, so
there's no connectivity problem and there's no error from the WiFly
board coming back...



I know it's a long description... To sum it up, some how, in these 2
situations, it seems that the firmware on the WiFly is adding
something to the HTTP request which makes Yahoo not recognise it....
I have also tried in both cases:

- to play with more or less "\n" at the end of the commands or even
add "\r"
- to add "HTTP/1.0" at the end of the page name both when it's set up
in comm remote and when it's send after open

To no avail !!!


This really is becoming huuugely frustrating, as I've already spend
several entire days on this problem (yes that's how desperate I am to
get that weather information on my VFD clock
http://trandi.wordpress.com/2011/09/10/vacuum-fluorescent-display-clock-based-on-soviet-iv-18-tube/
)

I've also had a look at the WiFly arduino library for the Sparkfun
shield, and checked what they are doing in the code... it's the same
thing, no extra commands or tricks that I could see...

Many many thanks for your help,
Dan

[Mark Steward]

Just guessing, because I don't have a WiFly, and haven't read the docs, but If you're sending HTTP manually, you'll need to follow the HTTP protocol:

GET /forecastrss?w=44418 HTTP/1.0

Host:  weather.yahooapis.com

The Yahoo APIs seem to return 404 when they actually mean 400.

Mark

[trandi]

Thanks for your quick reply.

This is a good point, but the WiFly normall adds this automatically.

The worst of the 2 problems, is the 1st one, whereby the exact thing
that works from TeraTerm does NOT when I connect on the same serial
line the Atmega 168.

I know, I know, you'll say that there's obviously a problem with the
serial connection but no: the atmega gets back the whole text with the
404 error, so the communication is ok...

And I also get 400 errors when I mess with the spaces around the
"GET"...

thanks,
dan

On Sep 20, 10:01 am, Mark Steward <markstew...@gmail.com> wrote:
> Just guessing, because I don't have a WiFly, and haven't read the docs, but
> If you're sending HTTP manually, you'll need to follow the HTTP protocol:
>
> GET /forecastrss?w=44418 HTTP/1.0
> Host:  weather.yahooapis.com
>
> The Yahoo APIs seem to return 404 when they actually mean 400.
>
> Mark
>
>
>
>
>
>
>
> On Tue, Sep 20, 2011 at 2:56 PM, trandi <robo.tra...@gmail.com> wrote:
> > Hi guys,
>
> > I usually hate bothering others with my technical issues so I tend to
> > search and try by myself until I find a solution... however this time
> > I can't take anymore, I've already spent more than an entire week-end
> > on this "silly" problem ....
> > So here I am, maybe somebody on this group can hep... (by the way, I
> > couldn't find a dedicated WiFly forum, am I missing something ?)
>
> > I'll start straight away buy insisting that my question / problem is
> > about the WiFly RN-134 board, which contains the SAME RovingNetworks
> > 131 module but its is NOT the Sparkfun Arduino shield.
>
> > I'm using (or trying to  :) ) this with an Atmega 168 micro
> > controller, same as for the Arduino but programmed directly with avr-
> > gcc.
>
> > I bought last week a RN-134 evaluation board (from mouser
>

> >http://uk.mouser.com/Search/ProductDetail.aspx?R=RN-134virtualkey6623...

> >http://trandi.wordpress.com/2011/09/10/vacuum-fluorescent-display-clo...

[Mark Steward]

I don't think it can add the Host header. How would it know when it's meant to do it?

Mark

[trandi]

That's an interesting point, but how do you explain that it works when
I conect to the WiFly from the PC through TeraTerm !???

As mentioned, in that case, with the exact same config, I get back the
right XML !

dan

[Mark Steward]

Not sure, but I agree that's a different issue.  Is there an application note/programming guide for the board around anywhere?

Mark

[Mark Steward]

It's just occurred to me that Host is optional for HTTP 1.0, so maybe 404's not out of place.

However, I wouldn't be surprised if a simple device like that only spoke 1.0.  If there's no public documentation, do you have any way of sniffing traffic to the access point?

Cheers,

Mark

[trandi]

Of course, they have this quite detailed user manual:
http://www.rovingnetworks.com/Docs/WiFly-RN-UM.pdf

I think I've learned their examples / text relevant to HTTP submission
by heart ! :)

thanks,
dan

[Martin Klang]

On 20 Sep 2011, at 16:18, Mark Steward wrote:

> do you have any way of sniffing traffic to the access point?

Excellent suggestion IMO.
If you don't know what goes over the wire you will have a hard time fixing it.

404 makes sense if you have omitted Host -
If you try:
telnet weather.yahooapis.com 80
GET /forecastrss?w=44418
then you also get a 404.

if you try:
telnet weather.yahooapis.com 80
GET /forecastrss?w=44418 HTTP/1.1
<newline>
then you get a 400:
client sent HTTP/1.1 request without hostname (see RFC2616 section 14.23).

but if you try:
telnet weather.yahooapis.com 80
GET /forecastrss?w=44418 HTTP/1.1
Host: weather.yahooapis.com
then you get the weather.

Oh and HTTP mandates CRLF for line and request termination, but many servers will accept a simple LF.
Ideally use:
open weather.yahooapis.com 80 \n
GET /forecastrss?w=44418 HTTP/1.1\r\n
Host: weather.yahooapis.com \r\n
\r\n
but of course \r and \n must be the actual characters, not the escape sequence. And the final <CRLF> must be present.
Or just skip the \r if you can't be bothered.

hth,

/m

[trandi]

Ok, I think I start to understand what's going on:

- I'm using the WiFly board in "HTTP" mode (set ip 18) which is
supposed to add this "HTTP/1.0 and the Host: parameter automatically)

-----------------------------------------------------------------------------------------------------------
<remote string><your data><SPACE>HTTP/1.0\nHost:<SPACE><the hostname
in config>\n\n

-----------------------------------------------------------------------------------------------------------

- something is going wrong there and I think the easiest option is to
go back into pure IP mode and construct myself the whole request as
you have just explained ...


I'll try it tonight after work and let you know...

dan


P.S. I've also got an answer from RovingNetworks support, slightly
dry / condescendent, BUT really really quick !
It basically mentions these strings that the HTTP mode adds to the
request and how in pure IP mode the WiFly won't touch the message
sent...

I've never seen any support answering your problem in a few hours !!
Thumbs up, they seem to have a great team there and caring about their
clients !

[Mark Steward]

Yeah, it looks like a nice piece of kit, which is why I'm so interested in it. Having read the manual, it seems it's not a very sophiaticated client, so I'd either try TCP mode or setting up an async wifi network so you can sniff it.

[trandi]

OK, after a quick test tonight it turns out that indeed going into
"pure IP" mode (so that the WiFly doesn't add anything by itself) and
sending:

"
open weather.yahooapis.com 80 \n
GET /forecastrss?w=44418 HTTP/1.1\r\n
Host: weather.yahooapis.com \r\n
\r\n
"

Does work indeed !!!

Exactly as Martin has mentioned...

Thank you everybody for your quick replies and all your help !

dan

On Sep 20, 6:39 pm, Mark Steward <markstew...@gmail.com> wrote:
> Yeah, it looks like a nice piece of kit, which is why I'm so interested in
> it. Having read the manual, it seems it's not a very sophiaticated client,
> so I'd either try TCP mode or setting up an async wifi network so you can
> sniff it.

> On Sep 20, 2011 6:09 PM, "trandi" <robo.tra...@gmail.com> wrote:
>
> > Ok, I think I start to understand what's going on:
>
> > - I'm using the WiFly board in "HTTP" mode (set ip 18) which is
> > supposed to add this "HTTP/1.0 and the Host: parameter automatically)
>

> --------------------------------------------------------------------------- --------------------------------> <remote string><your data><SPACE>HTTP/1.0\nHost:<SPACE><the hostname

[trandi]

Actually...

Everything works fine now (in pure TCP mode) as I've already mentioned
if I send:

"
open weather.yahooapis.com 80\r\n

GET /forecastrss?w=44418 HTTP/1.1\r\n
Host: weather.yahooapis.com\r\n
\r\n
"

HOWEVER, and I really don't understand it, if I want to do the same
thing on another website it does NOT work:

"
open search.twitter.com 80\r\n
GET /search.json?q=arduino&rpp=1 HTTP/1.0\r\n
Host: search.twitter.com\r\n
\r\n
"

I'm getting "Http 405 method not allowed allow get, head, post"

It's IDENTICAL code, except for the host and the page, and when using
telnet it works for both !!

Maybe I'm missing something really obvious...

Thank you again for your help !
Dan

[Mark Steward]

Sorry, no idea this time. If you really can't sniff it, change the domain to hack.rs and I'll have a look at what it's sending.

Mark

[Martin Klang]

> Maybe I'm missing something really obvious...

it's right there in your email,
and the server response:
HTTP/1.0 400 Bad Request

/m

On 21 Sep 2011, at 21:32, trandi wrote:

> GET /forecastrss?w=44418 HTTP/1.1\r\n

[trandi]

???? am I missing something ?

It's a 405 "Http 405 method not allowed allow get, head,
post" ...

I have actually tried with both HTTP/1.0 and 1.1 for both hosts and it
doesn't seem to do any difference...

dan

[Martin Klang]

ah sorry, I thought it was the 1.0 messing things up.
The Host header is not part of HTTP 1.0, is it?

Also there's a \r\n after the open command, which shouldn't be necessary - right?
Though I understand you say that it works for the other server.

Sending the same request over telnet results in a 200 OK.
What does the actual code look like? What are you writing this in?

/m

[Elliot West]

On 22 September 2011 10:11, Martin Klang <ma...@pingdynasty.com> wrote:

ah sorry, I thought it was the 1.0 messing things up.
The Host header is not part of HTTP 1.0, is it?

The internet suggests that a lot of proxies/servers like/want the Host header - even for HTTP/1.0

Dan, the simplest thing you can do at this point is sniff the network and compare at the byte level - anything else is just bumbling about in the dark.

[trandi]

Hi guys,

I've finally found the problem and finished my project:
http://trandi.wordpress.com/2011/09/26/vfd-clock-connects-to-the-internet/

Here's the short abstract of the issue, in case anybody on this forum
is interested or other people search it up in the future:

- the HTTP mode was working when connected to the WiFly board from the
PC but NOT when it was the AtMega micrcontroller sending the
commands.. still don't know why
- the solution was to go in TCP mode and construct the HTTP request
myself (you'll see the code in the wiFly.c file posted on my blog)
- then the problem with that was simply that when resetting the board
to its factory settings I had forgotten to clear the "com remote"
string, and the board would add "*HELLO*" to all my transmissions
(again, confusingly enough yahooapis.com would work just fine with
this, whereas search.twitter.com would complain !)

Hope this helps, let me know if you have any similar problems and need
help !

Thanks,
dan

On Sep 22, 10:19 am, Elliot West <tea...@gmail.com> wrote: