Delays in download VxWorks symbol table using FTP
Tom Fuda
I am using 486 targets (VME based single board computers from Logic Design
Group with on board Ethernet and IDE disk drive) in a Sun host environment.
I have noticed that if I try to download the VxWorks image (separate image
and symbol table) to my target using FTP it takes approximately 1 minute to
download the symbol table while the image itself is downloaded in a matter
of seconds. If I use RSH to download the symbol table from the Sun host the
transfer only takes a few seconds. This problem is observed whether my boot
host is the Sun or another VxWorks target (using either Ethernet or shared
memory network). I have also noticed that is two VxWorks targets
simultaneously attempt to download images from the same host using FTP, one
download usually fails while the other one succeeds after about a minute.
Has anyone else experienced this problem? If so, do you know of a workaround
or fix? Thanks in advance.
Thomas Fuda
Northrop Grumman Norden Systems
(203) 852-7991
fu...@norden.com
Greg Rasche
Tom,
I've experienced similar problems with i960 targets on the VME bus loading
from a SCSI disk which resides on one of the targets...but have not seen the
problem when loading from a Sun Host's disk.
We believe that our problem is related to the FTP Server running on one of
the VxWorks targets. WRS is currently investigating solving this problem.
Greg Rasche
gra...@procy.gi.com
John Gordon
I have six PPC boards sitting in a rack which all try
to boot simultaneously from a SparcStation 10 over
ethernet. I have noticed that my chances of all six
booting correctly when the VME reset is asserted (via
the slot one card reset) is slim. Assuming they get
as far as running the bootrom (not always the case)
they seem to fail at some stage either during kernel
download or symbol table download.
This is currently on my list of things to investigate!
John...
PS It is VxWorks 5.2 for PowerPC / Solaris 2.4 on the
Sparc
Greg Kowis
I've seen a similar delay using a DY4 SVME 162 booted from a Sun host. I
don't have a work around yet.
Greg Kowis
Texas Instruments
(214)952-4336
ko...@dlep1.itg.ti.com
In article <DuL0E...@tron.bwi.wec.com>, fu...@norden.com says...
Victor Franco
In article <DuL0E...@tron.bwi.wec.com>, Tom Fuda <fu...@norden.com> writes:
>I am using 486 targets (VME based single board computers from Logic Design
>Group with on board Ethernet and IDE disk drive) in a Sun host environment.
>I have noticed that if I try to download the VxWorks image (separate image
>and symbol table) to my target using FTP it takes approximately 1 minute to
>download the symbol table while the image itself is downloaded in a matter
>of seconds. If I use RSH to download the symbol table from the Sun host the
>transfer only takes a few seconds. This problem is observed whether my boot
>host is the Sun or another VxWorks target (using either Ethernet or shared
>memory network). I have also noticed that is two VxWorks targets
>simultaneously attempt to download images from the same host using FTP, one
>download usually fails while the other one succeeds after about a minute.
>Has anyone else experienced this problem? If so, do you know of a workaround
>or fix? Thanks in advance.
Tom,
I saved the following article from a few months back. It seems to address
the problem you are experiencing. I hope this helps. Credit, of course,
goes to the author, Jeffrey Szczepanski.
Good luck,
Victor Franco
Xerox Corp.
---- Included article ----
From j...@eroch.mc.xerox.com (Jeffrey R. Szczepanski)
Newsgroups: comp.os.vxworks
Subject: FREE CODE- You can't load symbol table via Network?
Date: Thu Apr 18 09:30:02 PDT 1996
Organization: Lawrence Berkeley Laboratory, Berkeley CA
Hello All!
Fix your symbol table loading problems?!
There has been talk here about the problem of loading the VxWorks symbol
table immediately after a network boot of VxWorks. This problem occurs
because the FTP load of the VxWorks image itself conflicts with the
following network load of the symbol table. The usuall symptom is a ERROR
loading the symbol table after a long timeout OR a successful load after
a long timeout. Neither do I find acceptable.
Some have explained this as a Sun problem, since this seems to be mostly
associated with Sun, as far as I have heard. Although I have heard others
complain about NT which has even a longer timeout....We have Suns here.
Anyway, after doing some "homework" on TCP, I believe Sun implements
correctly, what is known as the TIMED_WAIT phase or state of the TCP
protocol.
The problem arises because of the VxWorks implementation of TCP and how they
assign the port numbers on the TARGET end. The boot code's use of FTP to
load the VxWorks "application" image will use ports 1024 & 1025 for the
FTP transfer (Control on 1024, data on 1025). This assumes that no other
connects have been made by (or with) the target at the TCP level.
When VxWorks restarts on the newly loaded image, the starting port numbers
are reset back to the 1024 starting number. This means that the FTP transfer
that will take place to load the symbol table will, again, occur quickly
thereafter on 1024 & 1025 ports on the target. At this point, the HOST system
is still in a TIMED_WAIT for a virtual connect between the FTP ports on the
hosts and these specific port numbers on the target. The Sun is ignoring
the requests in TIMED_WAIT, as it should.
To eliminate this conflict you need to simply connect from the target to
the FTP server on the host FROM a different set of ports. The simplest
solution is to talk to something else first to increment the port numbers
on your target. The code attached below, connects first, to TCP network
standard TIME port on the host first. Any valid port connect will work,
you can pick your favorite!!! (Besides, FTP, obviously)
To use this patch, simply call ftpFix() immediately before attempting
to load the symbol table.
Good Luck,
Jeff
------------------------------------------------------------------------------
Jeffrey R. Szczepanski Phone: (716) 264-6613
Xerox Engineering Systems
300 Main Street
East Rochester, NY 14445 Email: j...@eroch.mc.xerox.com
------------------------------------------------------------------------------
#include "vxWorks.h"
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <stdio.h>
#define TCP_TIME_PORT 37
/*
**********************************************************************
*
* NAME: ftpFix
*
* DESCRIPTION: Work around the issue with timeout
* on FTP data sockets. Specifically,
* the reuse of TCP port #1024 is avoided
* when talking to the host FTP data port.
*
* PARAMETERS: void
*
* RETURN CODES: STATUS
*
* NOTES: This routine is used to work around
* the issue of loading the symbol table
* immediately after system boot. Generally,
* the load will hang because the host
* system has not timed out the socket "link"
* to the same VxWorks target port number.
**********************************************************************
*/
static STATUS
ftpFix(void)
{
struct sockaddr_in serverAddr; /* Server Port info */
int sock; /* Socket file descriptor */
if ((sock = socket(AF_INET, SOCK_STREAM, 0)) < 0)
{
ERROR();
return(FAILURE);
}
bzero((char *) (&serverAddr), sizeof(serverAddr));
serverAddr.sin_family = AF_INET;
serverAddr.sin_port = htons(TCP_TIME_PORT);
serverAddr.sin_addr.s_addr = hostGetByName(SysBootParams.hostName);
if (connect(sock, &serverAddr, sizeof(serverAddr)) != OK)
{
ERROR();
return(FAILURE);
}
close(sock);
return(SUCCESS);
}
Brian Watkins
I have also observed the FTP delay correctly diagnosed by Jeffrey
Szczepanski. I believe his solution does not entirely fix the problem; if
the target system is re-booted before the host exits the TIME_WAIT state
(usually 1-2 minutes) then the load of the VxWorks image or the symbol table
may be delayed.
I chose a different approach. It turns out that only one "side" of a TCP
connection ends up in the TIME_WAIT state when a connection is released.
The other side may re-use its socket immediately. The side that ends up in
the TIME_WAIT state is the first one that attempts to release the
connection. If the target were always the first to release the FTP
connection, the host would never enter the TIME_WAIT state.
In the TCP protocol, the CLOSE operation means "I have no more data to
send" (see RFC 793 sec. 3.5). Therefore, if data will never be written to a
socket, it is perfectly valid to CLOSE the connection before reading the
data!
I modified the bootrom of my mv162-523 board by adding a CLOSE of the FTP
data socket after the transfer is initiated but before the data is read as
follows in the netLoad() function in bootConfig.c:
...
{
if (bootFtp)
{
if (ftpXfer (hostName, usr, passwd, "", "RETR %s", "", fileName,
&errFd, &fd) == ERROR)
return (ERROR);
/* BDW 7-14-95
* Disable further writes on the data socket. This will
* hopefully initiate the graceful close of the socket
* and prevent the FTP server from entering the TCP
* TIME_WAIT state.
*/
shutdown (fd,1);
}
else
{
bootRsh = TRUE;
sprintf (command, "cat %s", fileName);
fd = rcmd (hostName, RSHD, usr, usr, command, &errFd);
if (fd == ERROR)
return (ERROR);
}
}
if (bootLoadModule (fd, pEntry) != OK)
goto readErr;
...
This gets us to the same point as Mr. Szczepanski's solution; we must also
modify the VxWorks image to ensure that the sockets used for the symbol
table and the startup files get CLOSED (via shutdown()) before the data is
read. Unfortunately these files are read via the network file driver
(netDrv) which cannot be modified (by those of us without a source code
license). Luckily netDrv uses the ftpDataConnGet() function to open the FTP
data connection, and this function is simple enough to reverse-engineer and
modify. I have written the following module to replace ftpDataConnGet():
/****
*
* $Program name: ftpPatch.c $
*
* $Description: Patch for VxWorks FTP client $
*
* This file contains two functions that are used to alter the behavior
* of the VxWorks FTP client code to eliminate (or at least reduce) the
* possibility of the FTP server data port passing through the TCP TIME_WAIT
* state. This state is entered when the FTP server is the first to close
* the data port. We will attempt to create a pending close using the
* shutdown() function for sockets that are only opened for reading.
*
* $Written by: Brian Watkins $
*
* $Source$
* $Locker$
* $Modified: Mon Jul 17 10:32:32 1995 (watkins) watkins@hpspk7df $
****/
static char * const rcsid = "$Header$";
#include "vxWorks.h"
#include "ftpLib.h"
#include "sockLib.h"
#include "unistd.h"
#include "stdio.h"
u_short ftpReadOnly = 0;
/**********************************************************************/
static int ftpPatchDataConnGet (int dataSock)
/* This function duplicates the functionality of the VxWorks
* ftpDataConnGet() function, with an additional call to
* shutdown() if an external variable indicates that the FTP
* data socket will only be used for reading.
*/
{
int newSock;
struct sockaddr addr;
int addrlen;
addrlen = sizeof (addr);
newSock = accept (dataSock, &addr, &addrlen);
close (dataSock);
if ((newSock != ERROR) && (ftpReadOnly != 0))
{
/* Initiate a graceful shutdown of the socket. */
shutdown (newSock,1);
}
return newSock;
}
/**********************************************************************/
void ftpPatchInit (void)
{
#define JMP_CODE 0x4ef9
u_short *short_ptr;
u_long *long_ptr;
/* Overwrite the beginning of the original ftpDataConnGet()
* function with an absolute jump to the patched routine.
* WARNING: This will not work if VxWorks is running out of
* ROM.
*/
short_ptr = (u_short *) ftpDataConnGet;
*short_ptr = JMP_CODE;
if (*(short_ptr++) == JMP_CODE)
{
long_ptr = (u_long *) short_ptr;
*long_ptr = (u_long *) ftpPatchDataConnGet;
puts ("FTP client patch installed");
}
else
{
puts ("Cannot patch FTP client");
}
}
/**********************************************************************/
Notice that the installation function is 680x0 specific, but it can be
easily adapted to other CPUs. This function also assumes that VxWorks is
running in RAM; the installation will fail if VxWorks is ROM based. The
installation function should also invalidate the instruction cache in case
the original ftpDataConnGet() function is in the cache.
The FTP modifications are completed with the following changes in the
usrRoot() function in usrConfig.c:
...
/* initialize network */
#ifdef INCLUDE_NET_INIT
usrBootLineInit (sysStartType); /* crack the bootline */
usrNetInit (BOOT_LINE_ADRS); /* initialize network support */
ftpPatchInit (); /* install FTP client patch */
ftpReadOnly = 1; /* Mark all FTP data sockets as */
/* read-only */
#endif /* INCLUDE_NET_INIT */
...
#ifdef INCLUDE_STARTUP_SCRIPT /* run a startup script */
if (sysBootParams.startupScript [0] != EOS)
usrStartupScript (sysBootParams.startupScript);
#endif /* INCLUDE_STARTUP_SCRIPT */
ftpReadOnly = 0; /* restore normal FTP */
/* client operation */
shellInit (SHELL_STACK_SIZE, TRUE); /* create the shell */
/* only include the simple demo if the shell is NOT included */
#else
ftpReadOnly = 0; /* restore normal FTP */
/* client operation */
#if defined(INCLUDE_DEMO) /* create demo w/o shell */
taskSpawn ("demo", 20, 0, 2000, (FUNCPTR)usrDemo, 0,0,0,0,0,0,0,0,0,0);
#endif /* mips cpp no elif */
#endif /* INCLUDE_SHELL */
...
Using the modified bootrom and VxWorks image, I rarely (if ever) observe
FTP delays.
Feel free to play around with this technique and/or e-mail me if you have
any questions.
--------------------------
Brian Watkins
wat...@spk.hp.com
Scott K Boehmke
I too have experienced similar SymbolTable load delays,
but I am using rsh to download... do your explanations
hold water for such a case? I've found it to be somewhat
random when trying to determine exactly when the board
will enter the delay... it usually occurs exactly when
I need the board the most. I scramble to figure out what's
wrong, and eventually it works fine on its own.
Watching the system with "snoop" shows that the VxWorks
target continues to request things from the host, but that
the host ignores (doesn't reply) until a timeout has occured.
Scott
--
Scott Boehmke (412) 268-5679 Voice
Research Engineer (412) 268-5895 Fax
Field Robotics Center
Carnegie Mellon University s...@cmu.edu
5000 Forbes Ave
Pittsburgh, PA 15213
Fabrice Cancre
Re: Delays in download VxWorks symbol table using FTP
I also had the same kind of problem with an MVME167 booting from a
Windows/NT FTP server.
The solution has been found by our Wind River Support in Chicago,
Ed Klein.
After the load of the kernel, the IP connection is closed by the
target which will reboot and restart immediatly. Depending on the
FTP server, the server will have or not time to actually close the
connection before the target reboots. If it has not time, the next
FTP transfer will hang up.
We solved this problem by inserting a delay between the kernel
download (ended by a FTP close) and the reboot. As far as I
remember, this has been donne in bootConfig.c in /config/all
directory. We actually use a delay of one second which is way to
large but very efficient.This requires the creation of new
bootroms.
Good luck
Fabrice Cancre
R/D Tech
1200 saint Jean Baptiste
Quebec, G2E 5E8
Canada
Tel : +1 (418)872-1155
Fax : +1 (418)872-5431
E-Mail : 10345...@compuserve.com
Home E-Mail : famille...@sympatico.ca