Google Группы больше не поддерживают новые публикации и подписки в сети Usenet. Опубликованный ранее контент останется доступен.

Dismiss

freebsd-hackers Digest, Vol 13, Issue 9

3 просмотра

Перейти к первому непрочитанному сообщению

freebsd-hac...@freebsd.org

не прочитано,

22 июн. 2003 г., 15:03:0222.06.2003

–

Send freebsd-hackers mailing list submissions to
freebsd...@freebsd.org

To subscribe or unsubscribe via the World Wide Web, visit
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
or, via email, send a message with subject or body 'help' to
freebsd-hac...@freebsd.org

You can reach the person managing the list at
freebsd-ha...@freebsd.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of freebsd-hackers digest..."

Today's Topics:

1. Re: Ftpd (option -h not working) (Socketd)
2. FreeBSD CVSROOT scripts and spaces in directory and file
names (Ruslan Ermilov)
3. rc.sendmail (John Von Essen)
4. Re: Ftpd (option -h not working) (Brandon D. Valentine)
5. Re: Replacing GNU grep revisited (David Schultz)
6. Re: rc.sendmail (Makoto Matsushita)
7. Re: open() and ESTALE error (Don Lewis)
8. Re: open() and ESTALE error (Andrey Alekseyev)
9. ld: memory exhausted (Marco van de Voort)
10. Re: Out of Office AutoReply: LinkSys WPC54G PCCARD (fwd)
(Scott Mitchell)
11. Re: Replacing GNU grep revisited (Sean Farley)
12. prism/2 firmware upload&download question (Dirk-Willem van Gulik)
13. Re: prism/2 firmware upload&download question (Matthew N. Dodd)

----------------------------------------------------------------------

Message: 1
Date: Sat, 21 Jun 2003 21:25:59 +0200
From: Socketd <d...@traceroute.dk>
Subject: Re: Ftpd (option -h not working)
To: Maxim Konovalov <ma...@macomnet.ru>, hac...@freebsd.org
Message-ID: <2003062121255...@traceroute.dk>
Content-Type: text/plain; charset=US-ASCII

On Sat, 21 Jun 2003 22:57:13 +0400 (MSD)
Maxim Konovalov <ma...@macomnet.ru> wrote:

> > Will someone please forward this message to the maintainer of ftpd
> > (can't find any mail is the source files)?
>
> http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/50690

Ah ok, strange it haven't been fixed jet then, but my bad for reporting
it without checking first :-)

br
socketd

------------------------------

Message: 2
Date: Sun, 22 Jun 2003 00:22:33 +0300
From: Ruslan Ermilov <r...@FreeBSD.org>
Subject: FreeBSD CVSROOT scripts and spaces in directory and file
names
To: Josef Karthauser <j...@FreeBSD.org>, Peter Wemm <pe...@FreeBSD.org>
Cc: hac...@FreeBSD.org
Message-ID: <20030621212...@sunbay.com>
Content-Type: text/plain; charset="us-ascii"

Hi there!

Our local CVSROOT scripts are based on a 1997 year version of FreeBSD
CVSROOT scripts, slightly modified to fit local needs, and we recently
noticed a problem with not getting email notifications if the affected
repository path has spaces in its name. I have verified that the
modern CVSROOT scripts (which I am planning to migrate to soon) are
subject to this problem as well.

There is a couple of problems actually.

The first problem turns out to be a known limitation in cvs(1), in
a way how it constructs the argument to the log_accum.pl script
(using the %s in CVSROOT/loginfo). With %s, both repository path
and the list of files (added, removed, or changed) in that directory
are passed as a single argument to the log_accum.pl script, separated
by a single whitespace character, which obviously creates a problem
for a parser in log_accum.pl that does a simple split().

The solution that works here and was inspired by reading the following
comment in src/contrib/cvs/src/logmsg.c, describing the % format
specifiers,

/* All other characters, we insert an empty field (but
we do put in the comma separating it from other
fields). This way if future CVS versions add formatting
characters, one can write a loginfo file which at least
won't blow up on an old CVS. */
/* Note that people who have to deal with spaces in file
and directory names are using space to get a known
delimiter for the directory name, so it's probably
not a good idea to ever define that as a formatting
character. */

was to use %{ s} to get a known " ," delimiter. (Chances still
are that the file/directory name will contain the " ," sequence,
but this is less likely.)

With this fix, no emails were lost anymore, but the contents were
still damaged. If a directory name had a space in its name, the
log_accum.pl::get_revision_number() that does a simple split()
when parsing the "Repository revision" string from "cvs -Qn status",
hits the whitespace problem. That was easy to fix, by limiting
splitting to four elements only. Then I hit a real PROBLEM.

The log_accum.pl script parses the standard log message generated
by logmsg.c, to build three lists of added, removed, and modified
files, each hashed by the "tag" (yes, it's possible to commit to
different branches in one commit, and even to commit to a single
file twice in a single commit). I first thought that I could just
take the list of files from the argument to the script (generated
by %{ s}), but there is no indication of "added/removed/changed",
and, more importantly, there is no "tag" information. And the
problem with building these lists by parsing the log message is
that the lists of added, removed, and modified files are formatted
by logmsg.c as separated by a single whitespace character (and to
make it fit the 70-column width), which creates the same problem
if a file has a space in a name.

My first hack was to patch logmsg.c to have it list files one per
line, so, for example, instead of

Modified files:
a b c

I have now been getting

Modified files:
a
b
c

This worked well, but I wanted a real SOLUTION. The solution
I have found is to implement the %T modifier for "loginfo"
scripts, which prints the tag (or HEAD for trunk). Combined
with the %V and %v modifiers which print "NONE" for added
and removed files, respectively, I have been able to fix the
log_accum.pl's logic to not depend on the contents of the log
message to build lists of added, removed, and modified files.

Attached in this email are:

- a patch for src/contrib/cvs that implements the %T (tag)
modifier for CVSROOT/loginfo scripts,

- a patch for log_accum.pl and loginfo that uses %T to deal
with whitespaces in directory and file names.

I'd like some input here before I submit this patch to CVS
developers.

P.S. The patch also removes two /usr/local/bin/awake uses
from log_accum.pl, as these made it unuseable for third
party application. Josef, Peter suggested me to ask you
to please address this particular problem with "awake",
and to make this script useful for third-parties again.

Cheers,
--
Ruslan Ermilov Sysadmin and DBA,
r...@sunbay.com Sunbay Software Ltd,
r...@FreeBSD.org FreeBSD committer
-------------- next part --------------
Index: doc/cvs.texinfo
===================================================================
RCS file: /home/ncvs/src/contrib/cvs/doc/cvs.texinfo,v
retrieving revision 1.1.1.13
diff -u -p -r1.1.1.13 cvs.texinfo
--- doc/cvs.texinfo 2 Dec 2002 03:13:37 -0000 1.1.1.13
+++ doc/cvs.texinfo 21 Jun 2003 20:21:34 -0000
@@ -12888,6 +12888,8 @@ separators. The format characters are:
@table @t
@item s
file name
+@item T
+tag
@item V
old version number (pre-checkin)
@item v
Index: src/logmsg.c
===================================================================
RCS file: /home/ncvs/src/contrib/cvs/src/logmsg.c,v
retrieving revision 1.11
diff -u -p -r1.11 logmsg.c
--- src/logmsg.c 2 Dec 2002 03:17:48 -0000 1.11
+++ src/logmsg.c 21 Jun 2003 20:24:12 -0000
@@ -683,6 +683,15 @@ title_proc (p, closure)
strlen (str_list) + strlen (p->key) + 5);
(void) strcat (str_list, p->key);
break;
+ case 'T':
+ str_list =
+ xrealloc (str_list,
+ (strlen (str_list)
+ + (li->tag ? strlen (li->tag) : 0)
+ + 10)
+ );
+ (void) strcat (str_list, (li->tag ? li->tag : "HEAD"));
+ break;
case 'V':
str_list =
xrealloc (str_list,
@@ -767,6 +776,7 @@ logfile_write (repository, filter, messa
`}' as separators. The format characters are:

s = file name
+ T = tag
V = old version number (pre-checkin)
v = new version number (post-checkin)

Index: src/mkmodules.c
===================================================================
RCS file: /home/ncvs/src/contrib/cvs/src/mkmodules.c,v
retrieving revision 1.12
diff -u -p -r1.12 mkmodules.c
--- src/mkmodules.c 2 Sep 2002 05:57:13 -0000 1.12
+++ src/mkmodules.c 21 Jun 2003 20:16:33 -0000
@@ -69,6 +69,7 @@ static const char *const loginfo_content
"# characters are:\n",
"#\n",
"# s = file name\n",
+ "# T = tag\n",
"# V = old version number (pre-checkin)\n",
"# v = new version number (post-checkin)\n",
"#\n",
-------------- next part --------------
diff --exclude=CVS -ru CVSROOT-freebsd/loginfo CVSROOT/loginfo
--- CVSROOT-freebsd/loginfo Fri Feb 28 05:35:48 2003
+++ CVSROOT/loginfo Sat Jun 21 22:55:15 2003
@@ -27,4 +27,4 @@
#DEFAULT (echo ""; id; echo %s; date; cat) >> $CVSROOT/CVSROOT/commitlog
# or
#DEFAULT (echo ""; id; echo %{sVv}; date; cat) >> $CVSROOT/CVSROOT/commitlog
-DEFAULT $CVSROOT/CVSROOT/log_accum.pl %s
+DEFAULT $CVSROOT/CVSROOT/log_accum.pl %{ TVvs}
diff --exclude=CVS -ru CVSROOT-freebsd/log_accum.pl CVSROOT/log_accum.pl
--- CVSROOT-freebsd/log_accum.pl Fri Feb 28 20:28:11 2003
+++ CVSROOT/log_accum.pl Sat Jun 21 23:00:25 2003
@@ -34,10 +34,7 @@
#
############################################################
my $STATE_NONE = 0;
-my $STATE_CHANGED = 1;
-my $STATE_ADDED = 2;
-my $STATE_REMOVED = 3;
-my $STATE_LOG = 4;
+my $STATE_LOG = 1;

my $BASE_FN = "$cfg::TMPDIR/$cfg::FILE_PREFIX";
my $LAST_FILE = $cfg::LAST_FILE;
@@ -233,7 +230,7 @@
while (<RCS>) {
if (/^[ \t]*Repository revision/) {
chomp;
- my @revline = split;
+ my @revline = split(" ", $_, 4);
$revision = $revline[2];
$revline[3] =~ m|^$CVSROOT/+(.*),v$|;
$rcsfile = $1;
@@ -635,8 +632,7 @@
# Initialize basic variables
#
my $input_params = $ARGV[0];
-my ($directory, @filenames) = split " ", $input_params;
-#@files = split(' ', $input_params);
+my ($directory, @fileinfo) = split " ,", $input_params;

my @path = split('/', $directory);
my $dir;
@@ -647,6 +643,23 @@
}
$dir = $dir . "/";

+my @filenames;
+my %added_files; # Hashes containing lists of files
+my %changed_files; # that have been changed, keyed
+my %removed_files; # by branch tag.
+foreach (@fileinfo) {
+ my ($tag, $oldrev, $newrev, $filename) = split(/,/, $_, 4);
+ push @filenames, $filename;
+ if ($oldrev eq "NONE") {
+ push @{ $added_files{$tag} }, $filename;
+ } elsif ($newrev eq "NONE") {
+ push @{ $removed_files{$tag} }, $filename;
+ } else {
+ push @{ $changed_files{$tag} }, $filename;
+ }
+ &append_line($TAGS_FILE, $tag);
+}
+
#
# Throw some values at the developer if in debug mode
#
@@ -697,7 +710,6 @@

&do_changes_file(@text);
&mail_notification(@text);
- system("/usr/local/bin/awake", $directory);
&cleanup_tmpfiles();
exit 0;
}
@@ -705,49 +717,26 @@
#
# Iterate over the body of the message collecting information.
#
-my %added_files; # Hashes containing lists of files
-my %changed_files; # that have been changed, keyed
-my %removed_files; # by branch tag.
-
my @log_lines; # The lines of the log message.

-my $tag = "HEAD"; # Default branch is HEAD.
my $state = $STATE_NONE; # Initially in no state.

while (<STDIN>) {
s/[ \t\n]+$//; # delete trailing space

- # parse the revision tag if it exists.
- if (/^Revision\/Branch:(.*)/) { $tag = $1; next; }
- if (/^[ \t]+Tag: (.*)/) { $tag = $1; next; }
- if (/^[ \t]+No tag$/) { $tag = "HEAD"; next; }
-
- # check for a state change, guarding against similar markers
- # in the log message itself.
- unless ($state == $STATE_LOG) {
- if (/^Modified Files/) { $state = $STATE_CHANGED; next; }
- if (/^Added Files/) { $state = $STATE_ADDED; next; }
- if (/^Removed Files/) { $state = $STATE_REMOVED; next; }
- if (/^Log Message/) { $state = $STATE_LOG; next; }
+ # don't do anything if we're not in a state.
+ if ($state == $STATE_NONE) {
+ $state = $STATE_LOG if /^Log Message/;
+ next;
}

- # don't so anything if we're not in a state.
- next if $state == $STATE_NONE;
-
# collect the log line (ignoring empty template entries)?
if ($state == $STATE_LOG) {
next if /^(.*):$/ and $cfg::TEMPLATE_HEADERS{$1};

push @log_lines, $_;
}
-
- # otherwise collect information about which files changed.
- my @files = split;
- push @{ $changed_files{$tag} }, @files if $state == $STATE_CHANGED;
- push @{ $added_files{$tag} }, @files if $state == $STATE_ADDED;
- push @{ $removed_files{$tag} }, @files if $state == $STATE_REMOVED;
}
-&append_line($TAGS_FILE, $tag);

#
# Strip leading and trailing blank lines from the log message.
@@ -896,7 +885,6 @@
&mail_notification(@log_msg);
}

-system("/usr/local/bin/awake", $directory);
&cleanup_tmpfiles();
exit 0;
# EOF
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20030622/90711b1a/attachment-0001.bin

------------------------------

Message: 3
Date: Sat, 21 Jun 2003 17:33:49 -0400
From: John Von Essen <jo...@essenz.com>
Subject: rc.sendmail
To: freebsd-...@freebsd.org
Cc: freebsd...@freebsd.org
Message-ID: <0BCBDE1C-A430-11D7...@essenz.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed

Could someone please explain rc.sendmail to me? I am unclear why it
does what it does. I currently have everything enabled in rc.conf:

mta_start_script="/etc/rc.sendmail"
sendmail_enable="YES"
(1) sendmail_flags="-L sm-mta -bd -q30m"
sendmail_submit_enable="YES"
(2) sendmail_submit_flags="-L sm-mta -bd -q30m
-ODaemonPortOptions=Addr=localhost"
sendmail_outbound_enable="YES"
(3) sendmail_outbound_flags="-L sm-queue -q30m"
sendmail_msp_queue_enable="YES"
(4) sendmail_msp_queue_flags="-L sm-msp-queue -Ac -q30m"

With the above settings, when I do a 'make start', only (1) and (4) get
started. If I set sendmail_enable="NO", then only (2) and (4) start. If
I set sendmail_enable="NO" and sendmail_submit="NO", then only (3) and
(4).

This doesn't make any sense to me. For starters, why would I ever want
just (3) and (4) running? Furthermore, I can't seem to get (1), (3),
and (4) to all start together. I imagine people would want those three
since you need your main sendmail running, and you could have a need
for an "always-on" queue runner for mqueue and clientmqueue.

Thanks.
John

------------------------------

Message: 4
Date: Sat, 21 Jun 2003 16:36:57 -0500
From: "Brandon D. Valentine" <bra...@dvalentine.com>
Subject: Re: Ftpd (option -h not working)
To: Socketd <d...@traceroute.dk>
Cc: hac...@freebsd.org
Message-ID: <20030621213...@geekpunk.net>
Content-Type: text/plain; charset=us-ascii

On Sat, Jun 21, 2003 at 09:25:59PM +0200, Socketd wrote:
> On Sat, 21 Jun 2003 22:57:13 +0400 (MSD)
> Maxim Konovalov <ma...@macomnet.ru> wrote:
>
> > > Will someone please forward this message to the maintainer of ftpd
> > > (can't find any mail is the source files)?
> >
> > http://www.freebsd.org/cgi/query-pr.cgi?pr=bin/50690
>
> Ah ok, strange it haven't been fixed jet then, but my bad for reporting
> it without checking first :-)

It has been fixed. Please read the entire audit trail of the PR. =)

On 16 Jun 2003 it was committed to -CURRENT, which was after the date of
5.1-RELEASE. The audit trail indicates that yar plans to MFC this at
some point RSN. I would doubt that this will make it into the
RELENG_5_1 branch unless you can convince the Security Officer team that
this is a serious problem for users of 5.1.

Brandon D. Valentine
--
bra...@dvalentine.com http://www.geekpunk.net
Pseudo-Random Googlism: brandon is a tax attorney

------------------------------

Message: 5
Date: Sat, 21 Jun 2003 17:58:53 -0700
From: David Schultz <d...@FreeBSD.ORG>
Subject: Re: Replacing GNU grep revisited
To: Sean Farley <sean-f...@farley.org>
Cc: freebsd...@FreeBSD.ORG
Message-ID: <20030622005...@HAL9000.homeunix.com>
Content-Type: text/plain; charset="us-ascii:iso-8859-1"

On Sat, Jun 21, 2003, Sean Farley wrote:
> In January, Pedro Giffuni started a thread about replacing GNU's grep in
> the system. Interestingly, I did not know about the grep thread on
> hackers until later. James Howard had interjected in February with
> mention that he had gotten patches to speed up freegrep. I had sent him
> those patches in late January, but I have not heard from him since.
> Hopefully, he is OK and just busy.
>
> I have placed the patches up on Geocities¹ for others to try out. They
> get freegrep fairly close to the performance of GNU's grep. Also
> included is a small patch to regex to squeak a bit more performance out
> of it, but I am not certain if it actually helps or not.
>
> BTW, Postgres is using a newer version of the regex library--swiped from
> TCL--that FreeBSD uses. It supports multibyte characters. Is it time
> for an update?

dds@ has expressed some interest in compiling the FSMs for regexps
into native code, which would make it blazingly fast. See
cvs-all@. As a practical matter, there are only a couple of
zealots who care what kind of license grep is under, so replacing
GNU grep with something that's ``almost as good as GNU grep'' is a
regression IMO. If we were talking about a kernel module or
library, of course, that would be a different matter.

------------------------------

Message: 6
Date: Sun, 22 Jun 2003 11:53:31 +0900
From: Makoto Matsushita <matu...@jp.FreeBSD.org>
Subject: Re: rc.sendmail
To: jo...@essenz.com
Cc: freebsd...@freebsd.org
Message-ID: <200306221153...@jp.FreeBSD.org>
Content-Type: Text/Plain; charset=us-ascii

john> Could someone please explain rc.sendmail to me?

Is rc.sendmail(8) not enough for you?

-- -
Makoto `MAR' Matsushita

------------------------------

Message: 7
Date: Sat, 21 Jun 2003 23:35:33 -0700 (PDT)
From: Don Lewis <truc...@FreeBSD.org>
Subject: Re: open() and ESTALE error
To: ui...@blackflag.ru
Cc: freebsd...@FreeBSD.org
Message-ID: <200306220635....@gw.catspoiler.org>
Content-Type: TEXT/plain; charset=us-ascii

On 21 Jun, Andrey Alekseyev wrote:
> Don,
>
>> old vnode and its associated file handle. If the file on the server was
>> renamed and not deleted, the server won't return ESTALE for the handle
>
> I'm all confused and messed up :) Actually, a rename on the server is not
> the same as sillyrename on the client. If you rename a file on the
> server for which there is a cached file handle on the client, next time
> the client will use its cached file handle, it'll get ESTALE from the server.
> I don't know how this happens, though. Until I dig more around all the
> rename paraphernalia, I won't know. If someone can clear this out, please
> do. It'll be much appreciated. At this time I can't link this with the
> inode generation number changes (as there is no new inode allocated when
> the file is renamed).

When a file is renamed on the server, its file handle remains valid.

I had some time to write some scripts to exercise this stuff and
discovered some interesting things. The NFS server is a 4.8-stable box
named mousie, and the NFS client is running 5.1-current. The tests were
run in my NFS-mounted home directory.

Here's the first script:

#!/bin/sh -v
rm -f file1 file2
ssh -n mousie rm -f file1 file2
echo foo > file1
echo bar > file2
ssh -n mousie cat file1
ssh -n mousie cat file2
tail -f file1 &
sleep 1
cat file1
cat file2
ssh -n mousie 'mv file1 tmpfile; mv file2 file1; mv tmpfile file2'
cat file1
cat file2
echo baz >> file2
sleep 1
kill $!
ssh -n mousie cat file1
ssh -n mousie cat file2

Here's the output of the script:

#!/bin/sh -v
rm -f file1 file2
ssh -n mousie rm -f file1 file2
echo foo > file1
echo bar > file2
ssh -n mousie cat file1
foo
ssh -n mousie cat file2
bar
tail -f file1 &
sleep 1
foo
cat file1
foo
cat file2
bar
ssh -n mousie 'mv file1 tmpfile; mv file2 file1; mv tmpfile file2'
cat file1
bar
cat file2
foo
echo baz >> file2
sleep 1
baz
kill $!
Terminated
ssh -n mousie cat file1
bar
ssh -n mousie cat file2
foo
baz

Notice that immediately after the files are swapped on the server, the
cat commands on the client are able to immediately detect that the files
have been interchanged and they open the correct files. The tail
command shows that the original handle for file1 remains valid after the
rename operations and when more data is written to file2 after the
interchange, the data is appended to the file that was formerly file1.

My second script is an attempt to reproduce the open() -> ESTALE error.

#!/bin/sh -v
rm -f file1 file2
ssh -n mousie rm -f file1 file2
echo foo > file1
echo bar > file2
ssh -n mousie cat file1
ssh -n mousie cat file2
sleep 1
cat file1
cat file2
ssh -n mousie 'mv file1 file2'
cat file2
cat file1

And its output:

#!/bin/sh -v
rm -f file1 file2
ssh -n mousie rm -f file1 file2
echo foo > file1
echo bar > file2
ssh -n mousie cat file1
foo
ssh -n mousie cat file2
bar
sleep 1
cat file1
foo
cat file2
bar
ssh -n mousie 'mv file1 file2'
cat file2
foo
cat file1
cat: file1: No such file or directory

Even though file2 was unlinked and replaced by file1 on the server, the
client immediately notices the change and is able to open the proper
file.

Since my scripts weren't provoking the reported problem, I wondered if
this was a 4.x vs. 5.x problem, or if the problem didn't occur in the
current working directory, or if the problem only occurred if a
directory was specified in the file path. I modified my scripts to work
with a subdirectory and got rather different results:

#!/bin/sh -v
rm -f dir/file1 dir/file2
ssh -n mousie rm -f dir/file1 dir/file2
echo foo > dir/file1
echo bar > dir/file2
ssh -n mousie cat dir/file1
foo
ssh -n mousie cat dir/file2
bar
tail -f dir/file1 &
sleep 1
foo
cat dir/file1
foo
cat dir/file2
bar
ssh -n mousie 'mv dir/file1 dir/tmpfile; mv dir/file2 dir/file1; mv dir/tmpfile dir/file2'
sleep 120
cat dir/file1
bar
cat dir/file2
bar
echo baz >> dir/file2
sleep 1
kill $!
Terminated
ssh -n mousie cat dir/file1
bar
baz
ssh -n mousie cat dir/file2
foo

Even after waiting long enough for the cached attributes to time out,
the one of cat commands on the client opened the incorrect file and when
the shell executed the echo command to append to one of the files, the
wrong file was opened and appended to. Conclusion, the client is
confused and retrying open() on an ESTALE error is insufficient to fix
the problem.

By specifying a directory in the path, I'm was also able to reproduce
the ESTALE error one time, but now I always get:

unless I decrease the sleep time:

#!/bin/sh -v
rm -f dir/file1 dir/file2
ssh -n mousie rm -f dir/file1 dir/file2
echo foo > dir/file1
echo bar > dir/file2
ssh -n mousie cat dir/file1
foo
ssh -n mousie cat dir/file2
bar
sleep 1
cat dir/file1
foo
cat dir/file2
bar
ssh -n mousie 'mv dir/file1 dir/file2'
# sleep 120
sleep 1
cat dir/file2
cat: dir/file2: Stale NFS file handle
cat dir/file1
foo

In one of my tests, I got an xauth warning from ssh, which made me think
that maybe the manipulation of my .Xauthority file might affect the
results. When I reran the original tests without X11 forwarding, I got
results similar to those that I got when I specified a directory in the
path:

#!/bin/sh -v
rm -f file1 file2
ssh -x -n mousie rm -f file1 file2
echo foo > file1
echo bar > file2
ssh -x -n mousie cat file1
foo
ssh -x -n mousie cat file2
bar
sleep 1
cat file1
foo
cat file2
bar
ssh -x -n mousie 'mv file1 file2'
cat file2
cat: file2: Stale NFS file handle
cat file1
foo

Conclusion: relying on seeing an ESTALE error to retry is insufficient.
Depending on how files are manipulated, open() may successfully return a
descriptor for the wrong file and even enable the contents of that file
to be overwritten. The namei()/lookup() code is broken and that's what
needs to be fixed.

------------------------------

Message: 8
Date: Sun, 22 Jun 2003 15:53:04 +0400 (MSD)
From: Andrey Alekseyev <ui...@blackflag.ru>
Subject: Re: open() and ESTALE error
To: truc...@FreeBSD.org (Don Lewis)
Cc: freebsd...@freebsd.org
Message-ID: <2003062211...@slt.oz>
Content-Type: text/plain; charset=US-ASCII

Don,

> When a file is renamed on the server, its file handle remains valid.

Actually I was wrong in my assumption on how names are purged from the
namecache. And I didn't mean an operation with a file opened on the client.
And it actually happens that this sequence of commands will get you ENOENT
(not ESTALE) on the client because of a new lookup in #4:

1. server: echo "1111" > 1
2. client: cat 1
3. server: mv 1 2
4. client: cat 1 <--- ENOENT here

Name cache can be purged by nfs_lookup(), if the latter finds that the
capability numbers doesn't match. In this case, nfs_lookup() will send a
new "lookup" RPC request to the server. Name cache can also be purged from
getnewvnode() and vclean(). Which code does that for the above scenario
it's quite obscure to me. Yes, my knowledge is limited :)

> Here's the output of the script:
>
> #!/bin/sh -v
> rm -f file1 file2
> ssh -n mousie rm -f file1 file2
> echo foo > file1
> echo bar > file2
> ssh -n mousie cat file1
> foo
> ssh -n mousie cat file2
> bar
> tail -f file1 &
> sleep 1
> foo
> cat file1
> foo
> cat file2
> bar
> ssh -n mousie 'mv file1 tmpfile; mv file2 file1; mv tmpfile file2'
> cat file1
> bar
> cat file2
> foo
> echo baz >> file2
> sleep 1
> baz
> kill $!
> Terminated
> ssh -n mousie cat file1
> bar
> ssh -n mousie cat file2
> foo
> baz
>
> Notice that immediately after the files are swapped on the server, the
> cat commands on the client are able to immediately detect that the files
> have been interchanged and they open the correct files. The tail
> command shows that the original handle for file1 remains valid after the
> rename operations and when more data is written to file2 after the
> interchange, the data is appended to the file that was formerly file1.

By the way, what were the values of acregmin/acregmax/acdirmin/acdirmax and
also the value of vfs.nfs.access_cache_timeout in your tests?

I believe, the results of your test patterns heavily depend on the NFS
attributes cache tunables (which happen to affect all cycles of NFS
operation) and on the command execution timing as well. Moreover, I'm
suspect that all this is badly linked with the type and sequence of
operations on both the server and the client. Recall, I was about to fix
just *one* common scenario :)

With different values of acmin/acmax and access_cache_timeout, and manual
operations I was able to achieve the result you consider as "proper" above
and also, the "wrong" effect that you described below.

> And its output:
>
> #!/bin/sh -v
> rm -f file1 file2
> ssh -n mousie rm -f file1 file2
> echo foo > file1
> echo bar > file2
> ssh -n mousie cat file1
> foo
> ssh -n mousie cat file2
> bar
> sleep 1
> cat file1
> foo
> cat file2
> bar
> ssh -n mousie 'mv file1 file2'
> cat file2
> foo
> cat file1
> cat: file1: No such file or directory
>
> Even though file2 was unlinked and replaced by file1 on the server, the
> client immediately notices the change and is able to open the proper
> file.

My tests always eventually produce ESTALE for file2 here. However, I suspect
their must be configurations where I won't get ESTALE.

> Conclusion: relying on seeing an ESTALE error to retry is insufficient.
> Depending on how files are manipulated, open() may successfully return a
> descriptor for the wrong file and even enable the contents of that file
> to be overwritten. The namei()/lookup() code is broken and that's what
> needs to be fixed.

I don't think it's namei()/lookup() which is broken. I'm afraid, the name
and attribute caching logic is somewhat far from ideal. Namecache routines
seem to work fine, they just do actual parsing/lookup of a pathname. Other
functions manipulate with the cached names basing on their understanding
of the cache validity (both namecache and cached dir/file attributes).

I've also done a number of tcpdump's for different test patterns and I
believe, what happens with the cached vnode may depend on the results of
the "access" RPC request to the server.

As I said before, I was not going to fix all the NFS inefficiencies related
to heavily shared file environments. However, I still believe that
open-retry-on-ESTALE *may* help people to avoid certain erratic conditions.
At least, I think that having this functionality switchable with an
additional sysctl variable, *may* help lots of people in the black art of
tuning NFS <attribute> caching. As there are no exact descriptions on how
all of this behaves, people usually have to experiment with their own
certain environments.

Also, I agree it's not the "fix" of everything. And I didn't even mention
I want this to be integrated in the source :)

Actually, I know that it works for what I've been fixing locally and just
asked for technical comments about possible "vnode leakage" and nameidata
initialization which nobody provided yet ;-P

I appreciate *very much* all of the answers, though. Definitely, a food for
thought, but I'm a little bit tired of this issue already :)

Thanks again for your efforts.

------------------------------

Message: 9
Date: Sun, 22 Jun 2003 14:01:31 +0200 (CEST)
From: mar...@stack.nl (Marco van de Voort)
Subject: ld: memory exhausted
To: freebsd...@freebsd.org
Message-ID: <20030622120...@toad.stack.nl>
Content-Type: text/plain; charset=US-ASCII

I'm trying to link a lang/fpc program, and ld bombs out with
"ld: Memory exhausted" when it reaches my physical memory limit. Worse,
it bombed when only one third of the .a's were loaded.

I tried to link with the most optimal deadcode elimination, by simply
creating a smartlinkable compilation unit per symbol and it seems
that ld needs about 100-120x the binary size in memory, which is
a bit steep. (so binary =1 MB, ld uses about 100-150 MB)

The binary I'm trying to generate is about 8 MB. (lazarus.freepascal.org,
a port recently requested on -ports that I'm looking into)

Does some binutils wizard know how to put a break on this behaviour, (e.g.
by custom building binutils, it's no problem if that slows down ld) ?

------------------------------

Message: 10
Date: Sun, 22 Jun 2003 13:30:52 +0100
From: Scott Mitchell <scott+...@fishballoon.org>
Subject: Re: Out of Office AutoReply: LinkSys WPC54G PCCARD (fwd)
To: Soeren Straarup <xr...@x12.dk>
Cc: freebsd...@freebsd.org
Message-ID: <2003062212...@tuatara.fishballoon.org>
Content-Type: text/plain; charset=iso-8859-1

On Sat, Jun 21, 2003 at 05:12:20PM +0200, Soeren Straarup wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> Hi how is it normal to treat auto email replies like this one?
>
> Best regards Søren

Hi Søren,

Best thing to do with these messages is ignore them :-) It's just MS
Outlook being lame and sending autoreplies to messages that weren't even
addressed to the person on vacation.

Followups to -chat, please...

Scott

------------------------------

Message: 11
Date: Sun, 22 Jun 2003 11:41:05 -0500 (CDT)
From: Sean Farley <sean-f...@farley.org>
Subject: Re: Replacing GNU grep revisited
To: freebsd...@FreeBSD.ORG
Message-ID: <2003062209...@thor.farley.org>
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Sat, 21 Jun 2003, David Schultz wrote:

> dds@ has expressed some interest in compiling the FSMs for regexps
> into native code, which would make it blazingly fast. See cvs-all@.
> As a practical matter, there are only a couple of zealots who care
> what kind of license grep is under, so replacing GNU grep with
> something that's ``almost as good as GNU grep'' is a regression IMO.
> If we were talking about a kernel module or library, of course, that
> would be a different matter.

Let me clarify. As far as I have been able to find out, the big
hold-out on not considering FreeGrep was that it was too slow. I got
the time to be comparable.

Reasons to consider for switching:
1. GNU's grep -r option "is broken" according to the following post.
The only thing I have noticed is that FreeGrep has more options for
controlling how symbolic links are traversed.
http://groups.google.com/groups?hl=en&lr=lang_en&ie=UTF-8&selm=xzp7kchblor.fsf_flood.ping.uio.no%40ns.sol.net

2. GNU's grep is using libgnuregex. The speed-up by dds@ would not be
felt?

3. FreeGrep gives one less thing for zealots to complain about. :)

Sean
-----------------------
sean-f...@farley.org

------------------------------

Message: 12
Date: Sun, 22 Jun 2003 18:48:54 +0200 (CEST)
From: Dirk-Willem van Gulik <di...@webweaving.org>
Subject: prism/2 firmware upload&download question
To: freebsd...@freebsd.org
Message-ID: <20030622184404.V68614-100000@foem>
Content-Type: TEXT/PLAIN; charset=US-ASCII

Propably not quite the right list - but given the number of people working
on wireless code here:

-> In the prism/2 and 2.5 firware are so called Firmware Information
records. The types 1 .. 4 are defined (and I can handle those
now). However I occasionally see a type 0x8001 - i.e. the version
info type with the top bit set.

Anyone any guesses as to what this signals ? This record tends to be
longer; 7 words rather than 5 words in size.

Thanks,

------------------------------

Message: 13
Date: Sun, 22 Jun 2003 13:55:46 -0400 (EDT)
From: "Matthew N. Dodd" <md...@FreeBSD.ORG>
Subject: Re: prism/2 firmware upload&download question
To: Dirk-Willem van Gulik <di...@webweaving.org>
Cc: freebsd...@FreeBSD.ORG
Message-ID: <2003062213...@sasami.jurai.net>
Content-Type: TEXT/PLAIN; charset=US-ASCII

On Sun, 22 Jun 2003, Dirk-Willem van Gulik wrote:
> -> In the prism/2 and 2.5 firware are so called Firmware Information
> records. The types 1 .. 4 are defined (and I can handle those
> now). However I occasionally see a type 0x8001 - i.e. the version
> info type with the top bit set.
>
> Anyone any guesses as to what this signals ? This record tends to be
> longer; 7 words rather than 5 words in size.

I've got a number of firmware images here I was testing my firmware image
parser against and haven't run into the record type above.

Which firmware are you seeing this in?

------------------------------

_______________________________________________
freebsd...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hacke...@freebsd.org"

End of freebsd-hackers Digest, Vol 13, Issue 9
**********************************************

0 новых сообщений