Question about ftp useage of vxworks (relating to ftpXfer, read, close)

[Michael.Jiang]

Thank you for looking of my question.

I'm using Tornado 2.0 and Vxworks 5.4. (Begining to use ftpLib in
recent days)
Two problems: :<

1. Read size cannot exceed 3KB. (The largest read size is limited to
2920 always)

2. If a lot of files' procession as (ftpXfer-->read-->close-->ftpXfer
...... ), There is a break happened after some files readed (Each
file's size ia about 64KB). On the FTP server's log, there are error
messages as
" USER anonymous
331 User name okay, please send complete E-mail address as
password.
PASS anonymous
230 User logged in, proceed.
TYPE I
200 Type set to I.
CWD .
250 Directory changed to /
PORT 10,0,0,1,5,104
200 PORT Command successful.
RETR file_004.bin
150 Opening BINARY mode data connection for file_004.bin (56284
bytes).
425-Maximum disk quota limited to 0 Kbytes
Used disk quota 5171132 Kbytes, available -5171132 Kbytes
425 Cannot open data connection.
QUIT
221 Goodbye!"

Thanks in advance for any opinion.

Procession is described as belows:

if (ftpXfer (... ,"RETR %s", ".", fname,&ctrlSock, &dataSock) == -1)
{ //error process }
else
{
// malloc ptr
read(dataSock, ptr, size);
...
// processing data
...
free(ptr);
close (dataSock);
if (ftpReplyGet (ctrlSock, TRUE) != FTP_COMPLETE)
printf("Unable to close %s\n", fname);
if (ftpCommand (ctrlSock, "QUIT", 0, 0, 0, 0, 0, 0) !=
FTP_COMPLETE)
printf("Unable to close %s\n", fname);
close (ctrlSocket);
taskDelay(20);
}
// redo as up.

[John]

Hello,

> 1. Read size cannot exceed 3KB. (The largest read size is limited to
> 2920 always)

Probably connected to the way the data is buffered in the network
stack, but why does it matter? read() does not guarantee that it will
return as much data as you ask for. Your application must expect,
especially when using network connections, to get the data in a number
of small blocks.

> 2. If a lot of files' procession as (ftpXfer-->read-->close-->ftpXfer
> ...... ), There is a break happened after some files readed (Each
> file's size ia about 64KB).

I assume that you mean you are doing this very fast (i.e. continually
getting new files without any significant delay between them). In this
case, you have to understand a limitation of TCP itself. When any
socket is closed, there is a delay of 2 * MSL (1 minute total for
VxWorks) before that socket descriptor can be re-used. In the network
code, if a request for a socket descriptor discovers that there are
none available, it blocks until the next timeout when another becomes
available. This will seem to be a pause, and then it will probably run
slowly.

If you are literally continually opening and closing connections, then
you have a problem. If you know the maximum number of files you will
transfer, or you have some regularity to the rate (e.g. one per
second), then you can set NUM_FILES and the NUM_SYS_xxx macros large
enough to ensure that there are always enough descriptors.

You can read a little more about this in my online book at:

http://www.bluedonkey.org/cgi-bin/twiki/bin/view/Books/VxWorksCookbookNetworking#Configuration

Look in the section under "Related OS Parameters" for a note on the
2*MSL issue.

> On the FTP server's log, there are error
> messages as
> " USER anonymous
> 331 User name okay, please send complete E-mail address as
> password.
> PASS anonymous
> 230 User logged in, proceed.
> TYPE I
> 200 Type set to I.
> CWD .
> 250 Directory changed to /
> PORT 10,0,0,1,5,104
> 200 PORT Command successful.
> RETR file_004.bin
> 150 Opening BINARY mode data connection for file_004.bin (56284
> bytes).
> 425-Maximum disk quota limited to 0 Kbytes
> Used disk quota 5171132 Kbytes, available -5171132 Kbytes
> 425 Cannot open data connection.
> QUIT
> 221 Goodbye!"

This is a problem on the server end! The user you are trying to log in
as appears to have exceeded a disk space quota on the server. The
server's quota for this user appears to be zero bytes. Looks like a
server configuration problem to me.

HTH,
John...

=====
Contribute to the VxWorks Cookbook at
http://www.bluedonkey.org/cgi-bin/twiki/bin/view/Books/WebHome

[Junichiro Hamaguchi]

Hello,

> Thank you for looking of my question.
>
> I'm using Tornado 2.0 and Vxworks 5.4. (Begining to use ftpLib in
> recent days)
> Two problems: :<
>
> 1. Read size cannot exceed 3KB. (The largest read size is limited to
> 2920 always)

Where does this limitation come from?

> 2. If a lot of files' procession as (ftpXfer-->read-->close-->ftpXfer
> ...... ), There is a break happened after some files readed (Each
> file's size ia about 64KB). On the FTP server's log, there are error
> messages as
> " USER anonymous
> 331 User name okay, please send complete E-mail address as
> password.
> PASS anonymous
> 230 User logged in, proceed.
> TYPE I
> 200 Type set to I.
> CWD .
> 250 Directory changed to /
> PORT 10,0,0,1,5,104
> 200 PORT Command successful.
> RETR file_004.bin
> 150 Opening BINARY mode data connection for file_004.bin (56284
> bytes).
> 425-Maximum disk quota limited to 0 Kbytes
> Used disk quota 5171132 Kbytes, available -5171132 Kbytes
> 425 Cannot open data connection.
> QUIT
> 221 Goodbye!"
>
> Thanks in advance for any opinion.

The error 425 came from the server. Have you checked quota
on the server side?

> Procession is described as belows:
>
> if (ftpXfer (... ,"RETR %s", ".", fname,&ctrlSock, &dataSock) == -1)
> { //error process }
> else
> {
> // malloc ptr
> read(dataSock, ptr, size);
> ...
> // processing data
> ...
> free(ptr);
> close (dataSock);
> if (ftpReplyGet (ctrlSock, TRUE) != FTP_COMPLETE)
> printf("Unable to close %s\n", fname);
> if (ftpCommand (ctrlSock, "QUIT", 0, 0, 0, 0, 0, 0) !=
> FTP_COMPLETE)
> printf("Unable to close %s\n", fname);
> close (ctrlSocket);
> taskDelay(20);
> }
> // redo as up.

It looks fine though, what does the 20 ticks taskDelay make different?

-hama