New (relative) Indy is very slow...
dk_sz
Using an old version of Indy 10.x it is blazingly fast...
However, with the newest...
It is veeeery slow (like 1/10 or even slower)
Has there recently (within the last year or) been
added some new options or similar?
best rgeards
Thomas
dk_sz
I am starting to suspect:
function TIdHTTPProtocol.ProcessResponse(AIgnoreReplies: array of SmallInt):
TIdHTTPWhatsNext;
it enters
"
end else begin
if (Request.Method = Id_HTTPMethodHead) or (LResponseCode = 204) then
begin
// Have noticed one case where a non-conforming server did send an
// entity body in response to a HEAD request, so just ignore
anything
// the server may send by accident
DiscardContent(5000); // Lets wait 5 seconds for any kind of content
end else begin
FHTTP.ReadResult(Response);
"
every time I have ResponseCode = 200 and is
making a HEAD requests (I make lots! of these)
At least the thread sometimes seems toe wait here some
seconds... Other (or all?) times EIdConnClosedGracefully
I am still in my beginning research...
And I am still in "shotgun" problem mode :-)
That said, my earlier Indy version did not have
"DiscardContent" code... So that is another reason why I
suspect the problem is related to that / HTTP 1.1 support (?)
I am sorta worried though... Because there
is no-way I can have it 10+ times slower :-(
My first guess was there were some
cross-all-connections indy critical section locks
which "stall" during some slow http reads/waits...?
However, I have only limited knowledge of where to look :-|
best regards
Thomas
"dk_sz" <dk...@hotmail.com> skrev i en meddelelse
news:4561...@newsgroups.borland.com...
Remy Lebeau (TeamB)
"dk_sz" <dk...@hotmail.com> wrote in message
news:4561f8d3$1...@newsgroups.borland.com...
> I am starting to suspect:
>
> function TIdHTTPProtocol.ProcessResponse(AIgnoreReplies: array of
SmallInt):
> TIdHTTPWhatsNext;
That is a known issue that has not been addressed yet.
> That said, my earlier Indy version did not have
> "DiscardContent" code... So that is another reason why I
> suspect the problem is related to that / HTTP 1.1 support (?)
It is not related to HTTP 1.1.
> My first guess was there were some
> cross-all-connections indy critical section locks
> which "stall" during some slow http reads/waits...?
Nope. You already nailed the problem head-on, - the DiscardContent()
function is the cause of the slow-down.
Gambit
dk_sz
Does Indy have a public bug-tracker and status system?
best regards
Thomas Schulz
dk_sz
I am torn since there are other Indy HTTP
client bug-fixes / improvements I am interested in :-)
best regards
Thomas Schulz
Remy Lebeau (TeamB)
> I am torn since there are other Indy HTTP
> client bug-fixes / improvements I am interested in :-)
Such as?
Gambit
Remy Lebeau (TeamB)
> Does Indy have a public bug-tracker and status system?
It used to, but it hardly gets used anymore, and in fact the links to it on
the Indy website are not working right now.
Gambit
dk_sz
>> client bug-fixes / improvements I am interested in :-)
>
> Such as?
* I think. e.g. IIS (-1) response codes...?
* Better HTTP 1.1 support (?)
best regards
Thomas
Remy Lebeau (TeamB)
> * I think. e.g. IIS (-1) response codes...?
-1 is not a valid response code. HTTP response codes MUST be 3-digit
non-negative numbers in the range of 1xx-5xx. What exactly are you
referring to?
> * Better HTTP 1.1 support (?)
Define "better" exactly.
Gambit
Andrew Fiddian-Green
>> cross-all-connections indy critical section locks
>> which "stall" during some slow http reads/waits...?
>
> Nope. You already nailed the problem head-on, - the DiscardContent()
> function is the cause of the slow-down.
Remy, you probably recall that I raised this (or a similar) issue a few
months back on the Indy HTTP NG?
I think we concluded it was a trade off between a) being "safe" on non
compliant servers and b) being "fast" on servers that are known to be
compliant. The current Indy code errs on the side of "safe" rather than
"fast".
Personally I modified my instance of the Indy source to resolve my speed
issue...
Regards,
AndrewFG
dk_sz
> non-negative numbers in the range of 1xx-5xx. What exactly are you
> referring to?
I know. But I could swear my memory tells me I still have seen -1
But, I will keep that thought for myself until I can show an example.
>> * Better HTTP 1.1 support (?)
>
> Define "better" exactly.
Well, I suppose e.g. stuff like IIS servers and
RawStrInternetToDateTime / ParseDayOfMonth
But you are saying that, as far as you know, there have
been no important HTTP client related updates? If so, I
suppose it is not important for me to update my Indy :-)
best regards
Thomas
dk_sz
> months back on the Indy HTTP NG?
What newsgroups is that???
> I think we concluded it was a trade off between a) being "safe" on non
> compliant servers and b) being "fast" on servers that are known to be
> compliant. The current Indy code errs on the side of "safe" rather than
> "fast".
What about adding an optioin...?
The speed difference in my application
is at least 10 times, probably even more.
best regards
Thomas
Remy Lebeau (TeamB)
> Well, I suppose e.g. stuff like IIS servers and
>
> RawStrInternetToDateTime / ParseDayOfMonth
Again, please be more specific. What EXACTLY are you referring to? You
need to provide actual details.
> But you are saying that, as far as you know, there have
> been no important HTTP client related updates? If so, I
> suppose it is not important for me to update my Indy :-)
I did not say that. There have been many changes in TIdHTTP over time.
Gambit
dk_sz
>>
>> RawStrInternetToDateTime / ParseDayOfMonth
> Again, please be more specific. What EXACTLY are you referring to? You
> need to provide actual details.
I posted about it about a week ago:
http://groups.google.dk/group/borland.public.delphi.internet.winsock/browse_thread/thread/beee7b444aa84dbb/bbb68eea702f6a9d
In that thread you wrote:
"
What you showed earlier is NOT what is in Indy 10 right now (not even
close). This is what the current code looks like (which I fixed in January
2006):
// RLebeau 01/23/2006 - IIS fix
lValue := Values['Expires']; {do not localize}
if IsNumeric(lValue) then
begin
"
> I did not say that. There have been many changes in TIdHTTP over time.
I am just in general interested. I use TIdHTTP
in some shareware programs where I encounter
a wide range of websites and webservers... I do
not expect I will hear about all errors my users encounter.
Still, I am quite content as it is, but would of course
like to update to newer (stable) versions when/if possible.
best regards
Thomas Schulz
Remy Lebeau (TeamB)
> I posted about it about a week ago:
Then why are you bringing it up again as a "need better support" issue? We
already discussed it last week. You were using an old version that no
longer applied. Did you try the new version yet?
> Still, I am quite content as it is, but would of course
> like to update to newer (stable) versions when/if possible.
Then why don't you?
Gambit
dk_sz
You asked for examples?
> "need better support" issue?
No!
> longer applied. Did you try the new version yet?
I can not because it is 10+ times slower (in my usage of it)
...
The conversation in this thread has gone like this:
ME:
I am torn since there are other Indy HTTP
client bug-fixes / improvements I am interested in :-)
YOU:
Such as?
ME:
* I think. e.g. IIS (-1) response codes...?
* Better HTTP 1.1 support (?)
YOU:
Define "better" exactly.
ME:
Well, I suppose e.g. stuff like IIS servers and
RawStrInternetToDateTime / ParseDayOfMonth
YOU:
Again, please be more specific. What EXACTLY are you referring to? You
need to provide actual details.
ME:
I posted about it about a week ago:
[COMMENT: Where you wrote this was fixed in
newer versions. That is why I refer to it! As an example!]
YOU:
Then why are you bringing it up again as a "need better support" issue? We
already discussed it last week. You were using an old version that no
longer applied. Did you try the new version yet?
best regards
Thomas Schulz
dk_sz
> I am torn since there are other Indy HTTP
> client bug-fixes / improvements I am interested in :-)
I can see where the misunderstanding perhaps occured.
I meant in context:
Old Indy10 version versus new Indy10 version.
I.e. I am torn between keeping the old Indy10 version
compared to the new Indy10 (since I would like the many,
I assumed, bug-fixes and improvements in the newer versions.)
I can not currently upgrade because
of the speed issue. (the DiscardContent problem)
best regards
Thomas Schulz
Andrew Fiddian-Green
The NG is news:atozedsoftware.indy.protocol.http
(the server is news.atozedsoftware.com)
> What about adding an optioin...?
You can try assigning the following call back to THttp.OnHeadersAvailable
procedure TMyThread.HeadersAvailable(Sender: TObject; AHeaders:
TIdHeaderList;
var VContinue: Boolean);
var
s: string;
const
te = 'Transfer-Encoding';
begin
// if we get a 20x response then just close the connection...
VContinue := ((fHTTP.ResponseCode div 100) <> 2);
// or (fHTTP.Request.Method <> Id_HTTPMethodHead);
// freebie: while we are at it, convert 'Chunked' to 'chunked'
s := LowerCase(aHeaders.Values[te]);
if IndyPos('chunked', s) > 0 then
begin
aHeaders.Values[te] := s;
end;
end;
dk_sz
> (the server is news.atozedsoftware.com)
>> What about adding an optioin...?
> You can try assigning the following call back to THttp.OnHeadersAvailable
Thanks! I found your posts in the above
newsgroup. Thank you so much for the
code snippet! I know have hope :-)
There is no way I could handle a 1000%
increase in HTTP communication time for
my programs... That was simply unbearable :-)
best regards
Thomas Schulz