Debugging Localhost with Charles (or fiddler)
280 views
Skip to first unread message
Maverix
Dec 8, 2013, 7:03:29 PM12/8/13
to rav...@googlegroups.com
Hi all,
I am trying to see what is going across the wire on my local dev box using Charles (very simular to Fiddler)
Just like fiddler they recommend changing the host by appending a fullstop after the localhost address
Great now I can see the requests in Charles but the result being passed back from Ravendb is:
HTTP Error 400. The request hostname is invalid
I'm sure this is something simple to fix... Any ideas?
Maverix
Dec 8, 2013, 7:19:24 PM12/8/13
to rav...@googlegroups.com
Yeah, wish I had bothered to not reveal my ignorance!
The answer? simply change the connection string to the machine name (in my case it is http://developer-pc:8080)
Oh well hopefully this will help someone else.
Chris Marisic
Dec 9, 2013, 2:20:03 PM12/9/13
to rav...@googlegroups.com
For future reference to others, when possible, the best possible address to use with fiddler is http://ipv4.fiddler/foo
Antoine
Nov 13, 2017, 4:59:31 AM11/13/17
to RavenDB - 2nd generation document database
Hello,
Friday I started having issues with My RavenFS instance. When uploading files using a code similar to this:
var store = new FilesStore();
store.Url = "http://localhost:9000/";
store.DefaultFileSystem = "foo";
store.Initialize();
var session = store.OpenAsyncSession();
var path = "tmp/240CA775-8288-4638-BB02-02F041748FFA";
var ms = new MemoryStream(bytes);
var metadata = new RavenJObject() { ... };
session.RegisterUpload(path, ms, metadata);
await session.SaveChangesAsync();
I got stuck on a strange parse error (RavenDB.Client 3.0.30000 and Server 30155 (maybe)).
I decided to update to RavenDB.Client 3.5.4 and Server 35215.
Now the error was more obvious to debug.
Raven.Imports.Newtonsoft.Json.JsonReaderException: Unexpected character encountered while parsing value: <. Path '', line 0, position 0.
at Raven.Imports.Newtonsoft.Json.JsonTextReader.ParseValue() in C:\Builds\RavenDB-Stable-3.5\Imports\Newtonsoft.Json\Src\Newtonsoft.Json\JsonTextReader.cs:line 974
PUT http://localhost:9000/fs/foo/files?name=tmp%2F240CA775-8288-4638-BB02-02F041748FFA
Server: Microsoft-HTTPAPI/2.0
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request - Invalid Hostname</h2>
<hr><p>HTTP Error 400. The request hostname is invalid.</p>
</BODY></HTML>
It seemed nothing was wrong in the code or configuration.
Today I found this post. I changed my url to "http://127.0.0.1:9000/" and the exception did not occur any more.
I am sure something is going wrong within the webserver with the Host header. I don't know what.
Changing the URL fixes the problem. The bug has a workaround so it does not need much attention.
I'm posting this to help users find this information.
Friday I started having issues with My RavenFS instance. When uploading files using a code similar to this:
var store = new FilesStore();
store.Url = "http://localhost:9000/";
store.DefaultFileSystem = "foo";
store.Initialize();
var session = store.OpenAsyncSession();
var path = "tmp/240CA775-8288-4638-BB02-02F041748FFA";
var ms = new MemoryStream(bytes);
var metadata = new RavenJObject() { ... };
session.RegisterUpload(path, ms, metadata);
await session.SaveChangesAsync();
I got stuck on a strange parse error (RavenDB.Client 3.0.30000 and Server 30155 (maybe)).
I decided to update to RavenDB.Client 3.5.4 and Server 35215.
Now the error was more obvious to debug.
Raven.Imports.Newtonsoft.Json.JsonReaderException: Unexpected character encountered while parsing value: <. Path '', line 0, position 0.
at Raven.Imports.Newtonsoft.Json.JsonTextReader.ParseValue() in C:\Builds\RavenDB-Stable-3.5\Imports\Newtonsoft.Json\Src\Newtonsoft.Json\JsonTextReader.cs:line 974
PUT http://localhost:9000/fs/foo/files?name=tmp%2F240CA775-8288-4638-BB02-02F041748FFA
Server: Microsoft-HTTPAPI/2.0
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request - Invalid Hostname</h2>
<hr><p>HTTP Error 400. The request hostname is invalid.</p>
</BODY></HTML>
It seemed nothing was wrong in the code or configuration.
Today I found this post. I changed my url to "http://127.0.0.1:9000/" and the exception did not occur any more.
I am sure something is going wrong within the webserver with the Host header. I don't know what.
Changing the URL fixes the problem. The bug has a workaround so it does not need much attention.
I'm posting this to help users find this information.
Oren Eini (Ayende Rahien)
Nov 13, 2017, 6:25:25 AM11/13/17
to ravendb
<BODY><h2>Bad Request - Invalid Hostname</h2>
It looks like you added another website to IIS, which cause it to do validation of the website host header
Hibernating Rhinos Ltd
Oren Eini l CEO l Mobile: + 972-52-548-6969
Office: +972-4-622-7811 l Fax: +972-153-4-622-7811
--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Antoine
Nov 13, 2017, 2:28:43 PM11/13/17
to RavenDB - 2nd generation document database
Hello Oren,
I considered that. But the following elements contradicts the idea.
I considered that. But the following elements contradicts the idea.
- Raven is running as a service (not in IIS)
- The document database responds well at the same URL
- Browsing the problematic URL with a web browser works well
- GET'ing the problematic url in Fiddler works
- PUT'ting the problematic url in Fiddler triggers the HTTP 400
That is quite a strange thing.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.
Oren Eini (Ayende Rahien)
Nov 13, 2017, 4:45:06 PM11/13/17
to ravendb
Check if you have some sort of firewall / anti virus in the middle.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.
Antoine
Nov 14, 2017, 1:30:46 PM11/14/17
to RavenDB - 2nd generation document database
Hello,
After some investigations, I found more stuff.
First, please consider the previous post as incorrect. Some things were not verified enough; I was having trouble trying too many thing concurrently (my brain is not thread-safe obviously).
A colleague found an intriguing workaround. He removed the metadata object that I was using and the request works.
Now I used Wireshark to determine what is happening on the wire.
Below is a working transaction that does not post metadata followed by a transaction with file metadata. It seems everything was done in a unique TCP stream.
It looks like metadata are given in HTTP headers and are problematic when they contain newlines (and other unexpected characters). Note the unexpected line returns between Remark and OriginalName at "tcp stream 2 | seq 977734" highlighted in red.
Before you look at the wireshark sample, here is how I create the metadata object. Note the newlines that I put in a metadata property.
Now the wireshark sample:
tcp stream 2 | seq 1 | raven-to-client
HTTP/1.1 100 Continue
tcp stream 2 | seq [210..934610] | client-to-raven (+TCP ACKs)
[chunks of POST file]
tcp stream 2 | seq 26 | raven-to-client
HTTP/1.1 201 Created
[submit file with metadata]
tcp stream 2 | seq 977734 | client-to-raven
PUT /fs/test/files?name=Temp%2Fb90b22a2-5e20-43ab-9f5b-e7afd84d9f2d HTTP/1.1
tcp stream 2 | seq 188 | raven-to-client
HTTP/1.1 400 Bad Request
It now looks obvious that metadata are not sent in a safe way.
What do you think?
After some investigations, I found more stuff.
First, please consider the previous post as incorrect. Some things were not verified enough; I was having trouble trying too many thing concurrently (my brain is not thread-safe obviously).
A colleague found an intriguing workaround. He removed the metadata object that I was using and the request works.
Now I used Wireshark to determine what is happening on the wire.
Below is a working transaction that does not post metadata followed by a transaction with file metadata. It seems everything was done in a unique TCP stream.
It looks like metadata are given in HTTP headers and are problematic when they contain newlines (and other unexpected characters). Note the unexpected line returns between Remark and OriginalName at "tcp stream 2 | seq 977734" highlighted in red.
Before you look at the wireshark sample, here is how I create the metadata object. Note the newlines that I put in a metadata property.
// UI application code
model.Remark = "VCC temporary file. \r\nSessionKey: " + sessionKeyForThisFile + "\r\n";
domain.StoreTemporaryFile(model); // domain invocation
// domain code (inlined StoreTemporaryFile method and DAL methods)
var meta = new Dictionary<string, string>();
meta.Add("ExpireDate", model.ExpireDate.ToString("o"));
meta.Add("MimeType", model.MimeType);
meta.Add("Remark", model.Remark);
meta.Add("OriginalName", model.OriginalName);
var metadata = new RavenJObject();
if (data != null)
{
foreach (var item in data)
{
metadata.Add(item.Key, item.Value);
}
}
session.RegisterUpload(path, ms, metadata);
meta.Add("ExpireDate", model.ExpireDate.ToString("o"));
meta.Add("MimeType", model.MimeType);
meta.Add("Remark", model.Remark);
meta.Add("OriginalName", model.OriginalName);
var metadata = new RavenJObject();
if (data != null)
{
foreach (var item in data)
{
metadata.Add(item.Key, item.Value);
}
}
session.RegisterUpload(path, ms, metadata);
await session.SaveChangesAsync(); // crash
Now the wireshark sample:
[submit file with null metadata]
tcp stream 2 | seq 1 | client-to-raven
PUT /fs/test/files?name=Temp%2F0637bd45-5ec1-4f19-9dc3-a0df12fd3be2 HTTP/1.1
tcp stream 2 | seq 1 | client-to-raven
PUT /fs/test/files?name=Temp%2F0637bd45-5ec1-4f19-9dc3-a0df12fd3be2 HTTP/1.1
Raven-Client-Version: 3.5.4.0
RavenFS-Size: 977510
Host: pc-2:9000
Transfer-Encoding: chunked
Expect: 100-continue
tcp stream 2 | seq 1 | raven-to-client
HTTP/1.1 100 Continue
tcp stream 2 | seq [210..934610] | client-to-raven (+TCP ACKs)
[chunks of POST file]
tcp stream 2 | seq 26 | raven-to-client
HTTP/1.1 201 Created
Content-Length: 0
Server: Microsoft-HTTPAPI/2.0
Raven-Server-Build: 30115
Temp-Request-Time: 331
Date: Tue, 14 Nov 2017 17:48:02 GMT
[submit file with metadata]
tcp stream 2 | seq 977734 | client-to-raven
PUT /fs/test/files?name=Temp%2Fb90b22a2-5e20-43ab-9f5b-e7afd84d9f2d HTTP/1.1
Raven-Client-Version: 3.5.4.0
ExpireDate: 2017-12-14T17:48:08.7185602Z
MimeType: text/plain
Remark: VCC temporary file.
SessionKey: User/D05691506832B4D69D17044464B5/VCC/VC-33/File/b90b22a2-5e20-43ab-9f5b-e7afd84d9f2d
OriginalName: 20080212_1200_1210.stm
RavenFS-Size: 23639
Host: pc-2:9000
Transfer-Encoding: chunked
Expect: 100-continue
tcp stream 2 | seq 188 | raven-to-client
HTTP/1.1 400 Bad Request
Content-Type: text/html; charset=us-ascii
Server: Microsoft-HTTPAPI/2.0
Date: Tue, 14 Nov 2017 17:48:12 GMT
Connection: close
Content-Length: 334
<!DOCTYPE .....> (see previous post HTTP 400 response)
It now looks obvious that metadata are not sent in a safe way.
- Am I using them wrong?
- If I would fix that myself:
- I would consider sending metadata in a different dedicated query (big code time), or sending them with the file in a multipart request (non-obvious code change).
- I would also consider checking for unauthorized metadata characters during the upload phase with an exception thrown beforehand.
- Otherwise:
- Is UpdateMetadataAsync safe to call with those metadata?
What do you think?
Oren Eini (Ayende Rahien)
Nov 15, 2017, 1:15:04 AM11/15/17
to ravendb
Very interesting, that is indeed a bug.
Can you work around that by avoid new lines there?
Hibernating Rhinos Ltd
Oren Eini l CEO l Mobile: + 972-52-548-6969
Office: +972-4-622-7811 l Fax: +972-153-4-622-7811
Antoine
Nov 29, 2017, 12:22:38 PM11/29/17
to RavenDB - 2nd generation document database
Yes, I can.
Reply all
Reply to author
Forward
0 new messages