Wininet, FTP, bogus 12029 error in passive mode calling FtpFindFirstFile
Darren Greenwald
google confirms we are not the only programmers affected.
The problem seems to be restricted to using Passive mode FTP.
If you call FtpFindFirstFile() twice in a row the first time works, the
second time returns error 12029. Now 12029 should be an error returned from
InternetConnect, but the fact is we are connecting OK. It is a bogus error.
And worse, once this happens there seems to be no fix short of closing down
Wininet.dll and reloading the DLL.
Now another programmer posted here a couple of months back that if he called
FtpGetCurrentDirectory between FtpFindFirstFile() calls it seems to mask the
bug - in fact it does seem to mask the bug and so we using it as a temporary
kludge.
A little more info:
1.) I set up the G6 FTP server and observed that when Wininet goes into the
12029 state like this it connects fine and goes into passive mode, but never
sends the LIST command to the FTP server.
2.) This bug has bit us in 3 ways. All of interest.
a.) Calling FtpFindFirstFile on an empty folder - since there is no
intervening call to InternetFindNextFile() the next call to FtpFindFirstFile
fails with error 12029. The kludge of calling FtpGetCurrentDirectory()
masks this.
b.) If two (or more) threads call FtpFindFirstFile at about the same time,
the second will fail with error 12029. A lock is required to single thread
calls to FtpFindFirstFile to prevent this.
c.) If FtpFindFirstFile returns the file you are looking for, normally you
would not then call InternetFindNextFile() and so again, next call to
FtpFindFirstFile() fails with error 12029.
3.) A programmer at work just informed me has discovered yet another API
call that once called, any call after that to FtpFindFirstFile() will return
error 12029. To be looked at in the morning.
4.) Once this occurs we are screwed - must shutdown and restart Wininet
(though we have to test that - some postings that Wininet leaks a thread
local storage slot and so cannot be reopened more than 64x).
5.) Haven't seen the problem occur using active mode.
6.) I am using IE6 - dont know yet if it is IE6 only or IE5 is included.
7.) Read my message carefully - I said it is connecting fine, the error
12029 is bogus - it is being returned when we call FtpFindFirstFile, not
when we call InternetConnect.
Darren Greenwald
"Darren Greenwald" <darre...@earthlink.net> wrote in message
news:#v4UcF8dBHA.1924@tkmsftngp02...
Brian Combs
Check to see if the issue happens in sync mode or is it only in async. You
could use netmon to get a trace to see what is going on at the network
level.
Q283679 Information on the IIS File Transmission Protocol (FTP) Service
http://support.microsoft.com/support/kb/articles/q283/6/79.asp
Q172712 INFO: Limitations of WinInet FTP Functions
http://support.microsoft.com/support/kb/articles/q172/7/12.asp
Netmon will give you the best information to see what is happening and
where the problem is.
Thanks
Brian
Microsoft Developer Support
This posting is provided "AS IS" with no warranties, and confers no rights.
You assume all risk for your use. © 2001 Microsoft Corporation. All rights
reserved.
Darren Greenwald
result of such an experiment won't matter. Unfortunately no time to
peform such experiments either. I can tell you that we already know
that when Wininet gets into this state it never sends the LIST command
to the FTP server, and once this happens it sticks. It is possible this
is related to the FtpFindFirstFile caching - something we have just run
up against and is quite a story in itself...
We are using async mode, and we have a work around, and as I mentioned
it doesn't happen in active mode, but it does in passive mode. It turns
out that the work around dove-tails with another work around so the work
is tolerable. The work around is we have a higher level wrapper around
Wininet, and a lock around FtpFindFirstFile so that only one thread can
call it at a time.
It turns out that FtpFindFirstFile does not work on all FTP servers if
there is a space in a folder name, even though it is sending the space
in the folder name to the FTP server. If you call FtpFindFirstFile on a
folder named "My Folder" it does send a "LIST My Folder string" to the
FTP server, but at least 3 FTP servers we have tried including IIS fail,
IIS fails with a user error indicating "My" is not found - the part
after the space is ignored. The G6 FTP server doesn't do this, but some
FTP servers do. So this seems to be a limitation of the FTP server, yet
all programmers that use Wininet to do FTP need to be aware of it.
The work around for that is to get the current directory, set the
current directory to the sub-folder (which does work on all FTP servers
we have tried, you can CWD in to a folder whose name has spaces), and
then use FtpFindFirstFile, and then back out to the original directory.
As a side effect these extra calls masked the bug with 12029 on the
second FtpFindFirstFile call.
But they also expose another gotcha with FtpFindFirstFile, something we
have never observed until after we put this work around. Since doing
the work around we found that FtpFindFirstFile is caching! folder
listings. That really threw us because we would get a folder listing,
then delete a file on the FTP server via Wininet, then get another
folder listing and suddenly we get the old folder listing. Yet we never
saw this behavior before adding the extra calls to set the current
folder.
Even more strange, if we called FtpFindFirstFile but we don't enumerate
to the end of the list, then it doesn't cache. But if we call
FtpFindFirstFile and then enumerate to the end of the list it caches.
That just makes no sense and doesn't seem to be documented.
As an experiment the programmer threw the following flags in:
flags |= INTERNET_FLAG_RELOAD;
flags |= INTERNET_FLAG_RESYNCHRONIZE;
flags |= INTERNET_FLAG_DONT_CACHE;
flags |= INTERNET_FLAG_NEED_FILE;
and indeed one of these stops the caching, but the unexpected behaviors
are:
1.) It was never caching before we put in the kludge to change the
current folder... and we have been using Wininet for quite a long time.
Or maybe it was caching if but only caches if you do FtpFindFirstFile on
the current directory and we have been lucky and have always been
calling FtpFindFirstFile on a sub-folder - dont know for sure.
2.) Caching is very broken or at least as far as we can tell, useless.
Why? Well consider the following:
a.) We connect to an FTP server, we get a folder listing with
FtpFindFirstFile, we disconnect.
b.) We connect (again) to the same FTP server, we delete a file using
Wininet's FtpDeleteFile(), we disconnect, so the now the folder is empty
on the FTP server.
c.) We connect (again) to the same FTP server, we get folder listing
with FtpFindFirstFile, and it returns us a listing that includes the
file we deleted. And I can see in the FTP server logs, Wininet never
sent the LIST command, this is old stale data coming out of its cache.
The cache seems to persists across connections, yet it is dumb because
it doesn't even detect that we deleted a file, via Wininet no less. I
can't see any value in all in returning a stale folder listing. It can
never lead to any good, just confusion.
3.) I have no idea why FtpFindFirstFile caching is enabled by default
anyway - the opposite would be better, with a strong warning that cached
data may be stale data. Caching could go in to the app, no need for it
in a low level function. Caching is a wonderful thing as long as it
isn't stale data. But stale data makes for a questionable cache at
best.
I think we need to go searching for a good FTP client, source code we
can integrate in to our code, reasonable licensing fee, something that
supports modern features like resumed downloads. We have spent way too
much time chasing mysterious bugs in Wininet and without source code and
the poor documentation every time we run in to a problem we have no
choice but to experiment to find out how things really work vs how they
are (not) documented to work. Without source we are at Microsoft's
mercy and I doesn't look like the FTP client portion of Wininet is on
their radar right now.
"Brian Combs" <bcombs...@microsoft.com> wrote in message
news:OBBlofnfBHA.1480@cpmsftngxa09...
Brian Combs
It sounds like you are pushing WinInet past what it was designed for.
WinInet was designed to be used with IE in a client environment. See
"Q238425 INFO: WinInet Not Supported for Use in Services" which talks about
this. Also see
Q172712 INFO: Limitations of WinInet FTP Functions
Thanks
Brian [MS]
Darren Greenwald
my original posting. You sure you were responding to me? Or where you
just cut/copy/pasting an automated reply?
Regarding the link to q238425, we are not using Wininet from a service
but even so how is it pushing Wininet to call FtpFindFirstFile() two
times without making an intervening call? I think if you go back and
read my postings and look at the problems I listed you will see that the
articles you provided links too do not address these matters. Also if
you search www.google.com you will find that we are not the only
programmers affected by the bogus 12029 error when calling
FtpFindFirstFile twice in a row using passive mode FTP.
"Brian Combs" <bco...@online.microsoft.com> wrote in message
news:HnZUjnBiBHA.2108@cpmsftngxa07...
Brian Combs
To address your specfic issue.
"The problem seems to be restricted to using Passive mode FTP.
If you call FtpFindFirstFile() twice in a row the first time works, the
second time returns error 12029"
"a.) Calling FtpFindFirstFile on an empty folder - since there is no
intervening call to InternetFindNextFile() the next call to FtpFindFirstFile
fails with error 12029."
"b.) If two (or more) threads call FtpFindFirstFile at about the same time,
the second will fail with error 12029."
"c.) If FtpFindFirstFile returns the file you are looking for, normally you
would not then call InternetFindNextFile() and so again, next call to
FtpFindFirstFile() fails with error 12029."
I tested all of the above from a client box with IE 6.0 to FTP server on
Win 2000 server sp2.
I took a network trace and saw the LIST command for each FtpFindFirstFile
call.
I could not repro any of the above.
I used the following code.
// *** Start Code ***
#include <windows.h>
#include <wininet.h>
#include <iostream.h>
HINTERNET hOpen;
// HINTERNET hConnect; // if you don't want to use the threads then
uncomment this
// HINTERNET hFind; // if you don't want to use the threads then uncomment
this
// WIN32_FIND_DATA pData; // if you don't want to use the threads then
uncomment this
DWORD WINAPI ThreadFunc( LPVOID lpParam )
{
HINTERNET hFind;
HINTERNET hConnect;
WIN32_FIND_DATA pData;
if ( !(hConnect = InternetConnect ( hOpen, "YourServerName" ,
INTERNET_INVALID_PORT_NUMBER, "FTP", "FTP",
INTERNET_SERVICE_FTP, INTERNET_FLAG_PASSIVE , 0) ) )
{
cout << GetLastError()<< "InternetConnect"<< endl;
return 1;
}
// change the *.* if looking for a specfic file
if ( !(hFind = FtpFindFirstFile (hConnect,TEXT ("*.*") , &pData, 0, 0)
))
if (GetLastError() == ERROR_NO_MORE_FILES)
{
cout << "This directory is empty or the file is not there" << endl;
InternetCloseHandle (hConnect); // need to add error checking
return 0;
}
else
{
DWORD dwError = GetLastError();
// need ot add error checking
cout << GetLastError () << "FindFirst error: " << endl;
if( ERROR_INTERNET_EXTENDED_ERROR == dwError)
{
char szBuffer[1024] = "";
DWORD dwLength = 1024;
DWORD dwIntError = 0;
InternetGetLastResponseInfo (&dwIntError, szBuffer, &dwLength);
cout << szBuffer << endl;
}
InternetCloseHandle (hConnect); // need to add error checking
return 1;
}
cout << pData.cFileName;
if (pData.dwFileAttributes == FILE_ATTRIBUTE_DIRECTORY)
// Entry is a directory, mark it as such
cout << " <DIR> " << endl;
else
cout << endl;
if (!InternetCloseHandle (hFind) )
{
cout << GetLastError() << "InternetCloseHandle error findfirst";
return 1;
}
if (!InternetCloseHandle (hConnect) )
{
cout << GetLastError() << "InternetCloseHandle error Connect";
return 1 ;
}
return 0;
}
void main()
{
if ( !(hOpen = InternetOpen ( "Passive FTP Test",
INTERNET_OPEN_TYPE_PRECONFIG , NULL, 0, 0) ) )
{
cout << GetLastError() << "InternetOpen" << endl;
return ;
}
// if you don't want to use the threads comment the thread code
DWORD dwThreadId, dwThrdParam = 1;
HANDLE hThread[3];
int i = 0;
for ( i = 0; i < 3; i++)
{
hThread[i] = CreateThread(
NULL, // no security attributes
0, // use default stack size
ThreadFunc, // thread function
&dwThrdParam, // argument to thread function
0, // use default creation flags
&dwThreadId); // returns the thread identifier
// Check the return value for success.
if (hThread == NULL)
{
cout << "CreateThread failed." << endl;
return;
}
}
DWORD dwWait = WaitForMultipleObjects(3, hThread, TRUE, 30000);
for ( i = 0; i < 3; i++)
{
CloseHandle( hThread[i] );
}
// end of thread code
// uncomment this block if you don't want to use threads.
/* if ( !(hConnect = InternetConnect ( hOpen, "YourServerName" ,
INTERNET_INVALID_PORT_NUMBER, "FTP", "FTP",
INTERNET_SERVICE_FTP, INTERNET_FLAG_PASSIVE , 0) ) )
{
cout << GetLastError()<< "InternetConnect"<< endl;
return ;
}
for (int i = 0; i < 3; i++)
{
if ( !(hFind = FtpFindFirstFile (hConnect, TEXT ("*.*"), &pData, 0, 0) ))
if (GetLastError() == ERROR_NO_MORE_FILES)
{
cout << "This directory is empty or the file is not there" << endl;
//return ;
continue;
}
else
{
cout << GetLastError () << "FindFirst error: ";
return ;
}
cout << pData.cFileName;
if (pData.dwFileAttributes == FILE_ATTRIBUTE_DIRECTORY)
// Entry is a directory, mark it as such
cout << " <DIR> " << endl;
else
cout << endl;
if (!InternetCloseHandle (hFind) )
{
cout << GetLastError() << "InternetCloseHandle error findfirst";
return ;
}
}
if (!InternetCloseHandle (hConnect) )
{
cout << GetLastError() << "InternetCloseHandle error Connect";
return ;
}*/ // end of non-thread code
if (!InternetCloseHandle (hOpen) )
{
cout << GetLastError() << "InternetCloseHandle error Open";
return ;
}
}
//*** end code ***
Darren Greenwald
for examination and compare it with what our code is doing.
For your info here is a link to another person who has seen the problem:
http://groups.google.com/groups?q=12029+FtpFindFirstFile&hl=en&rnum=2&se
lm=c5c22590.0109280744.639206fb%40posting.google.com
One difference is we are using Async mode.
"Brian Combs" <bco...@online.microsoft.com> wrote in message
news:fT7z51yjBHA.1792@cpmsftngxa09...
Alex Koshterek
Hi, Darren.
It`s me found solution about putting FtpGetCurrentDirectory between
FtpFindFirstFile calls.
I`m also working in async passive mode.
WinInet looks very bugy.
I`m also got 12003 error without any usefull info from
InternetGetLastResponseInfo while uploading file under Win98 SE with
not updated IE. In active mode all works fine.
I`m not experienced this problem with Win2k.
Brian Combs
I looked at that post and changed my code to call InternetGetNextFile if
needed. I still did not have any problem. I did not test with asynchronous.
I find that if I am using worker threads to do the downloading I don't need
to use WinInet Asynchronously. The worker thread provides much of that
functionality. I find asynchronous WinInet useful in enduser apps where I
may not want to use a worker thread and I don't want to lock the
application while I down load data.
If I get a chance I will change my code to use asynchronous calls to see if
that makes a difference.
Darren Greenwald
Our Wininet programmer modified your example to use async mode and
demonstrated the problem. In retrospect we might have used worker
threads had we know that async mode was going to be an issue. We have
no problem with starting multiple threads and would have worked this way
had we known async mode would have issues.
The only concern then would be that presumably one can still abort FTP
functions in synchronous mode by closing the parent handle. It does
seem to work in Async mode, but I wouldn't trust it until we have
actually tried it in sync mode.
"Brian Combs" <bco...@online.microsoft.com> wrote in message
news:LoYSYlUlBHA.2048@cpmsftngxa07...
Darren Greenwald
Yes, it seems that this particular bug does not occur in active mode, or
in sync mode. For today we use your work around. For tomorrow I think
we are going to search for a modern FTP client that we can
buy/license/use the source code for and integrate that in to our own
code. Simultaneously we may look at rewriting our Wininet wrappers to
use sync mode and start multiple threads. Thanks for your feedback
Alex.
"Alex Koshterek" <ha...@od.anything3d.com> wrote in message
news:c5c22590.02010...@posting.google.com...
Brian Combs
If you want to post the changed code sample I will be glad to take a look
at it.
Closing the handle should kill the request. See Q224318 HOWTO: Control
Connection Timeout by Creating Second Thread