freebsd-hackers Digest, Vol 366, Issue 6
freebsd-hac...@freebsd.org
freebsd...@freebsd.org
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
or, via email, send a message with subject or body 'help' to
freebsd-hac...@freebsd.org
You can reach the person managing the list at
freebsd-ha...@freebsd.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of freebsd-hackers digest..."
Today's Topics:
1. Re: Compiling kernel with gcc43 [SOLVED] (Mario Lobo)
2. Re: Compiling kernel with gcc43 [SOLVED] -ADDENDUM (Mario Lobo)
3. leak of the vnodes (Petr Salinger)
4. write(2) size is limited by INT_MAX on amd64 -- is it a bug
or a feature? (Artem Belevich)
5. Re: leak of the vnodes (Kostik Belousov)
6. Re: write(2) size is limited by INT_MAX on amd64 -- is it a
bug or a feature? (Kostik Belousov)
7. Re: leak of the vnodes (Petr Salinger)
8. howto make 8.0-RELEASE-i386-disc1.iso (Jiandong Lu)
9. Re: Fwd: mkuzip and/or geom_uzip changes? - SOLVED (Tim Judd)
10. Re: howto make 8.0-RELEASE-i386-disc1.iso (Andriy Gapon)
----------------------------------------------------------------------
Message: 1
Date: Fri, 2 Apr 2010 12:39:44 +0000
From: Mario Lobo <lo...@bsd.com.br>
Subject: Re: Compiling kernel with gcc43 [SOLVED]
To: Pegasus Mc Cleaft <k...@mthelicon.com>
Cc: freebsd...@freebsd.org, FreeBSD-...@freebsd.org
Message-ID: <20100402123...@bsd.com.br>
Content-Type: Text/Plain; charset="iso-8859-1"
On Thursday 01 April 2010 16:53:36 Pegasus Mc Cleaft wrote:
> On Thursday 01 April 2010 15:27:41 Oliver Fromme wrote:
> > Mario Lobo <lo...@bsd.com.br> wrote:
> > > [...]
> > > It's compiling right now.
> > >
> > > I'll post my findings and impressions on results and performance right
> > > after the next reboot.
> >
> > So, how is it going? Any benchmarks yet? I'm curious
> > if the new gcc version will really make a significant
> > difference.
>
> I would love to see the /etc/make.conf, /etc/src.conf and
> /etc/libmap.conf files that were used for the build. I have tried compiling
> in VBox a current kernel and world, but it usually just bombs out for me.
> I would like to give this a go as well.
>
> Peg
>
Well, to tell the truth I wasn't that thrilled with the results. I didn't
benchmark anything by my impressions were that at least disk access was a bit
slower not only during booting but it was more noticeable to me particularly
on a burning DVD session. Of course this is ultra abstract.
In all previous experiences I had in burning CD/DVD with k3b, I recollect that
during burning, the software buffer and device buffer gouges were always 100%,
with the software buffer gouge dropping down occasionally to 89/92%.
After recompiling the kernel with gcc43, the software buffer was always empty
and the device buffer rarely reached 40/50%. I think (if not mistaken) this
means that the device is asking "where is my data??" and the OS is not
providing it fast enough.
I could not get world to build with gcc43 so I gave that up. Then I moved on
to VirtualBox. I managed to have it compiled and running. After long trial and
error sessions, I could pin point what was breaking compilation and fixed it.
Here are the steps:
------------------------------------------------------------------------------
Compiling vbox/vbox-devel with gcc43
1) /usr/include/cam/cam.h needed #include <stdio.h> for FILE define,
complained by:
work/VirtualBox-3.1.51.r27657_OSE/src/VBox/Main/freebsd/HostHardwareFreeBSD.cpp:47:
/usr/include/cam/cam.h:246: error: 'FILE' has not been declared
2)/usr/ports/emulators/virtualbox-
ose/work/VirtualBox-3.1.6_OSE/src/VBox/Main/generic/NetIf-generic.cpp
needed #include <stdio.h> because of popen() (this step is ONLY for
virtualbox-ose)
3)/usr/ports/emulators/virtualbox-ose(-
devel)/work/VirtualBox-3.1.6_OSE/src/VBox/Main/freebsd/NetIf-freebsd.cpp
needed #include <stdlib.h> because of malloc()/free()
4) Config.kmk needed some tweaks:
a) line 1750 - $(APPEND) '$@' 'VBOX_GCC_mtune-generic ?= $(call
VBOX_GCC_CHECK_CC,-mtune=amdfam10 -D__FreeBSD_cc_version=0,)'
to use instructions closer to Phenom and avoid cc complains.
b) took out all references to "-fformat-extensions" and "-fno_format-
extensions"
c) Preceeded all relevant locations of "/usr/lib \" with
"/usr/local/lib/gcc43 \" so kbuild searched there for libraries first.
(except TEMPLATE_VBOXQT4GUIEXE_LIBPATH !)
d) You can use the same Config.kmk for building kmods as well
5) took out -fformat-extensions from src/sys/conf/kern.mk
And I left /etc/libmap.conf
libgcc_s.so.1 gcc43/libgcc_s.so.1
libgomp.so.1 gcc43/libgomp.so.1
libobjc.so.3 gcc43/libobjc.so.2
libssp.so.0 gcc43/libssp.so.0
libstdc++.so.6 gcc43/libstdc++.so.6
/etc/make.conf
CC=/usr/local/bin/gcc43
CXX=/usr/local/bin/g++43
CPP=/usr/local/bin/cpp43
CFLAGS+=-mssse3 -D__FreeBSD_cc_version=0
CXXFLAGS+=-D__FreeBSD_cc_version=0
CPUTYPE=amdfam10
#MAKEOPTS+= -j4
and /etc/src.conf
NO_WERROR=
WERROR=
------------------------------------------------------------------------------
New problems started to come when I tried to extend this to other ports,
breaking a lot of them, to point of making me revert everything back to what
it was. In fact I am still in this process right now, and reving a lot of
problems to rebuild kde4.
This is it for now, guys. If you find anything new, please post.
Best of luck,
--
Mario Lobo
http://www.mallavoodoo.com.br
FreeBSD since version 2.2.8 [not Pro-Audio.... YET!!] (99,7% winfoes FREE)
------------------------------
Message: 2
Date: Fri, 2 Apr 2010 13:47:43 +0000
From: Mario Lobo <lo...@bsd.com.br>
Subject: Re: Compiling kernel with gcc43 [SOLVED] -ADDENDUM
To: Pegasus Mc Cleaft <k...@mthelicon.com>
Cc: freebsd...@freebsd.org, FreeBSD-...@freebsd.org
Message-ID: <20100402134...@bsd.com.br>
Content-Type: Text/Plain; charset="iso-8859-1"
Well, to tell the truth I wasn't that thrilled with the results. I didn't
[snip]
------------------------------------------------------------------------------
Compiling vbox/vbox-devel with gcc43
1) /usr/include/cam/cam.h needed #include <stdio.h> for FILE define,
complained by:
[snip]
and /etc/src.conf
NO_WERROR=
WERROR=
------------------------------------------------------------------------------
New problems started to come when I tried to extend this to other ports,
[snip]
You have to:
1) make patch
2) apply the mods
3) make
Best of luck, again
--
Mario Lobo
http://www.mallavoodoo.com.br
FreeBSD since version 2.2.8 [not Pro-Audio.... YET!!] (99,7% winfoes FREE)
------------------------------
Message: 3
Date: Fri, 2 Apr 2010 19:45:03 +0200 (CEST)
From: Petr Salinger <Petr.S...@seznam.cz>
Subject: leak of the vnodes
To: freebsd...@freebsd.org
Message-ID: <Pine.LNX.4.62.10...@sci.felk.cvut.cz>
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
Hi,
I have the same problem as in
http://lists.freebsd.org/pipermail/freebsd-hackers/2009-August/029227.html
During "make check" of gcc-4.3 the vfs.numvnodes goes up,
after reaching default limit 100000 the machine is stuck.
kern.maxvnodes: 100000
kern.sigqueue.alloc_fail: 0
kern.sigqueue.overflow: 371
kern.sigqueue.preallocate: 1024
kern.sigqueue.max_pending_per_proc: 128
kern.minvnodes: 25000
vfs.freevnodes: 22
vfs.wantfreevnodes: 25000
vfs.numvnodes: 100005
debug.vnlru_nowhere: 811
It is not on plain FreeBSD, but the GNU/kFreeBSD
changes to the kernel are minimal.
The KTR trace of KTR_VFS from 8-stable is at
http://asdfasdf.debian.net/~salinger/ktr.gz
Thanks for any hints.
Petr
------------------------------
Message: 4
Date: Fri, 2 Apr 2010 11:53:09 -0700
From: Artem Belevich <fbsd...@src.cx>
Subject: write(2) size is limited by INT_MAX on amd64 -- is it a bug
or a feature?
To: FreeBSD Hackers <freebsd...@freebsd.org>
Message-ID:
<v2qed91d4a81004021153ka...@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Hi,
I've ran into a problem on 8-stable/amd64 today. Basically any attempt
to pass 2GB chunk of data to write(2) returns EINVAL. It looks like
we're limiting amount of data to be written to INT_MAX which looks
rather restrictive on LP64 platforms. NetBSD/OpenBSD do use SSIZE_MAX
which does seem to be the limit specified by POSIX, if I'm looking at
the correct specification here
http://www.opengroup.org/onlinepubs/000095399/functions/write.html
A bit of googling shows that this issue was also recently mentioned on
svn-src-all:
http://www.mail-archive.com/svn-s...@freebsd.org/msg18266.html
Was the INT_MAX limit in FreeBSD imposed intentionally, even on 64-bit
platforms or is it a bug that needs fixing?
Thanks,
--Artem
------------------------------
Message: 5
Date: Fri, 2 Apr 2010 22:02:39 +0300
From: Kostik Belousov <kost...@gmail.com>
Subject: Re: leak of the vnodes
To: Petr Salinger <Petr.S...@seznam.cz>
Cc: freebsd...@freebsd.org
Message-ID: <2010040219...@deviant.kiev.zoral.com.ua>
Content-Type: text/plain; charset="us-ascii"
On Fri, Apr 02, 2010 at 07:45:03PM +0200, Petr Salinger wrote:
> Hi,
>
> I have the same problem as in
> http://lists.freebsd.org/pipermail/freebsd-hackers/2009-August/029227.html
>
> During "make check" of gcc-4.3 the vfs.numvnodes goes up,
> after reaching default limit 100000 the machine is stuck.
>
> kern.maxvnodes: 100000
> kern.sigqueue.alloc_fail: 0
> kern.sigqueue.overflow: 371
> kern.sigqueue.preallocate: 1024
> kern.sigqueue.max_pending_per_proc: 128
> kern.minvnodes: 25000
> vfs.freevnodes: 22
> vfs.wantfreevnodes: 25000
> vfs.numvnodes: 100005
> debug.vnlru_nowhere: 811
>
> It is not on plain FreeBSD, but the GNU/kFreeBSD
> changes to the kernel are minimal.
>
> The KTR trace of KTR_VFS from 8-stable is at
> http://asdfasdf.debian.net/~salinger/ktr.gz
>
> Thanks for any hints.
Is machine completely stuck, or is it makes small steps once in a second ?
You could look at the "ps alx" output, or ps output from ddb. Look for
the processes in the "vnlru" state.
If my suspection is true, you have such processes. The typical cause is
the large directory hierarchy. vnlru daemon does not reclaim the free
vnodes that are sources of namecache records.
You can either increase kern.maxvnodes, the default value is very
conservative on amd64, where a lot of KVA is available. On the other
hand, increase of the value on i386 could easily cause KVA exhaustion.
Another possible workaround, if you do not need path resolutions in /proc
or lsof(1), is to set sysctl vfs.vlru_allow_cache_src=1.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20100402/64262d29/attachment-0001.pgp
------------------------------
Message: 6
Date: Fri, 2 Apr 2010 22:05:38 +0300
From: Kostik Belousov <kost...@gmail.com>
Subject: Re: write(2) size is limited by INT_MAX on amd64 -- is it a
bug or a feature?
To: Artem Belevich <fbsd...@src.cx>
Cc: FreeBSD Hackers <freebsd...@freebsd.org>
Message-ID: <2010040219...@deviant.kiev.zoral.com.ua>
Content-Type: text/plain; charset="us-ascii"
On Fri, Apr 02, 2010 at 11:53:09AM -0700, Artem Belevich wrote:
> Hi,
>
> I've ran into a problem on 8-stable/amd64 today. Basically any attempt
> to pass 2GB chunk of data to write(2) returns EINVAL. It looks like
> we're limiting amount of data to be written to INT_MAX which looks
> rather restrictive on LP64 platforms. NetBSD/OpenBSD do use SSIZE_MAX
> which does seem to be the limit specified by POSIX, if I'm looking at
> the correct specification here
> http://www.opengroup.org/onlinepubs/000095399/functions/write.html
>
> A bit of googling shows that this issue was also recently mentioned on
> svn-src-all:
> http://www.mail-archive.com/svn-s...@freebsd.org/msg18266.html
>
> Was the INT_MAX limit in FreeBSD imposed intentionally, even on 64-bit
> platforms or is it a bug that needs fixing?
I did some preliminary work for this, changing the type of uio_resid
member of struct uio in stable/8 and HEAD to ssize_t. I have half-finished
patch that takes this to completion, allowing SSIZE_MAX maximal i/o.
No ETA when it will be done.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-hackers/attachments/20100402/809a29fd/attachment-0001.pgp
------------------------------
Message: 7
Date: Fri, 2 Apr 2010 23:46:41 +0200 (CEST)
From: Petr Salinger <Petr.S...@seznam.cz>
Subject: Re: leak of the vnodes
To: Kostik Belousov <kost...@gmail.com>
Cc: freebsd...@freebsd.org
Message-ID: <Pine.LNX.4.62.10...@sci.felk.cvut.cz>
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
> You can either increase kern.maxvnodes, the default value is very
> conservative on amd64, where a lot of KVA is available. On the other
> hand, increase of the value on i386 could easily cause KVA exhaustion.
The increase helps, the system become responsive. In fact I previously
suspected scheduler, but change to 4BSD, UP, nopreemption didn't helped.
The build directory of gcc-4.3 is in a separated mount point. Even after
the build is stopped and the mount point unmounted, the vfs.numvnodes
is still to high. The temporary files (*.s) are in /tmp, but they have
been deleted, so my expectation is that vfs.numvnodes should lower
significantly.
> Another possible workaround, if you do not need path resolutions in /proc
> or lsof(1), is to set sysctl vfs.vlru_allow_cache_src=1.
I will test this.
Petr
------------------------------
Message: 8
Date: Sat, 3 Apr 2010 10:14:39 +0800 (CST)
From: Jiandong Lu <lujiand...@yahoo.com.cn>
Subject: howto make 8.0-RELEASE-i386-disc1.iso
To: freebsd...@freebsd.org
Message-ID: <95187....@web15707.mail.cnb.yahoo.com>
Content-Type: text/plain; charset=utf-8
hi,everyone.
each time I use disk1 to install my FreeBSD system.Now I want to make my own FreeBSD distro.My question is
how to make disk1?
thanks.
------------------------------
Message: 9
Date: Sat, 3 Apr 2010 02:02:36 -0600
From: Tim Judd <taj...@gmail.com>
Subject: Re: Fwd: mkuzip and/or geom_uzip changes? - SOLVED
To: John Baldwin <j...@freebsd.org>
Cc: freebsd...@freebsd.org, freebsd-...@freebsd.org
Message-ID:
<k2tade45ae91004030102m9...@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
On 4/1/10, Tim Judd <taj...@gmail.com> wrote:
> On 4/1/10, Tim Judd <taj...@gmail.com> wrote:
>> On 4/1/10, John Baldwin <j...@freebsd.org> wrote:
>>> On Wednesday 31 March 2010 6:32:09 pm Tim Judd wrote:
>>>> Hi All,
>>>>
>>>> Just starting to see if I can find other reports. You all probably
>>>> have had the "more than one pair of eyes looking at a thing is better
>>>> than my eyes alone." This is why I'm writing now, as I'm starting the
>>>> discovery.
>>>>
>>>> Let me background this a little bit. I only started looking into this
>>>> because mkuzip and it's counterpart, geom_uzip are throwing errors on
>>>> FreeBSD8 i386
>>>>
>>>>
>>>> scenario (/etc/src.conf in effect, removing *LOTS* of stuff with
>>>> knobs):
>>>> make DESTDIR=/home/small8 installworld installkernel distribution
>>>> mv /home/small8/boot /home/small8-boot/
>>>> makefs -t ffs /home/small8/usr.img /home/small8/usr/
>>>> mkuzip -o /home/small8/usr.uzip /home/small8/usr.img
>>>> [*]
>>>> chflags -R noschg /home/small8/usr/*
>>>> rm -rf /home/small8/usr/* /home/small8/usr.img
>>>> ee /home/small8/etc/rc.d/mountcritlocal
>>>> [**]
>>>> makefs -t ffs /home/small8-boot/mfsroot /home/small8/
>>>> gzip --best /home/small8-boot/mfsroot
>>>> ee /home/small8-boot/boot/loader.conf
>>>> [***]
>>>> rm /home/small8-boot/boot/kernel/*.symbols
>>>> gzip --best /home/small8-boot/boot/kernel/kernel
>>>> mkisofs -U -J -r -V "FreeBSD8" -b boot/cdboot -no-emul-boot
>>>> -iso-level 4 -o /home/small8.iso /home/small8-boot/
>>>>
>>>>
>>>> [*]: mkuzip inserts a script header that is broken. module name it's
>>>> searching for may have been renamed?
>>>> [**]: Edited mountcritlocal to mount the usr.uzip file as by using the
>>>> above script header, throws errors
>>>> [***]: added zlib and geom_uzip modules to load to the boot image, to
>>>> satisfy the script header's requirements.
>>>>
>>>> OK, the above scenario creates about a 33MB usr.uzip, and a 68MB iso.
>>>> Small enough to apparently fit into the undocumented 50 or 100MB size
>>>> limit of mfs_root module
>>>
>>> BTW, you can raise this limit by changing NKPT.
>>>
>>>> The problem:
>>>> mkuzip generates a few lines as a script in the head of the
>>>> resulting *.uzip file. Two problems...
>>>> 1) the module it queries for is geom_uzip (kldstat -m $m), but
>>>> FreeBSD8 names the geom_uzip module (i guess, internally) as g_uzip.
>>>> mkuzip's generated image will never find the module if they're not
>>>> named the same.
>>>
>>> It is g_uzip even in 7:
>>>
>>> DECLARE_GEOM_CLASS(g_uzip_class, g_uzip);
>>> MODULE_DEPEND(g_uzip, zlib, 1, 1, 1);
>>>
>>> This has probably just been broken from the start. If it used 'kldstat
>>> -n'
>>> then it might work. Well, it probably works (modulo a warning) by
>>> accident
>>> as
>>> it doesn't hurt to kldload an already-loaded module. Note though that
>>> it
>>> assumes the raw usr.img is an ISO image, not a UFS filesystem.
>>>
>>>> 2) even with geom_uzip module and it's dependency zlib loaded, i don't
>>>> get a mdconfig node '/dev/md?.uzip' to appear.
>>>>
>>>> It's been forever since I touched uzip, so I have to ask.
>>>
>>> Do you have a md0 device at all? I think you want to hack the script to
>>> do
>>> something like this:
>>>
>>> disk=`mdconfig -af /path/to/usr.img`
>>> mount -r /dev/$disk.uzip /usr
>>>
>>> --
>>> John Baldwin
>>>
>>
>>
>>
>> booted single-user
>> md0 is the mfs_root
>>
>> here is the manual attachment of an mdconfig...
>> # mdconfig -af /usr.uzip
>> WARNING: opening backing store: /usr.uzip readonly
>> md1.uzip: block size (24) should be a multiple of 512.
>> md1
>> # ls /dev/md1*
>> /dev/md1
>> #
>>
>
> Forgot the kldstat, which was obviously omitted
>
> # kldstat
> Id Refs Address Size Name
> 1 5 0xc0400000 b6e060 kernel
> 2 1 0xc0f6f000 3ffc geom_uzip.ko
> 3 2 -xc0f73000 ac20 zlib.ko
>
John, All:
Don't spend any more time on this issue as a show-stopper anymore. I
understand what was going on enough to realize that the middle line,
rather than a warning, was an outright error and the md?.uzip device
cannot be presented.
When I was trying to diagnose my cascading problems, one of the items
I did was to edit (with ee) the usr.uzip binary file. I only used the
cursor in the script header part, saved it and tried it out.
Evidentally, that screwed the file up.
Recreating the .img, converting to a .uzip is working.
I'm back on track, no need to continue to search this.
enjoy!
------------------------------
Message: 10
Date: Sat, 03 Apr 2010 11:14:49 +0300
From: Andriy Gapon <a...@icyb.net.ua>
Subject: Re: howto make 8.0-RELEASE-i386-disc1.iso
To: Jiandong Lu <lujiand...@yahoo.com.cn>
Cc: freebsd...@freebsd.org
Message-ID: <4BB6F8F9...@icyb.net.ua>
Content-Type: text/plain; charset=UTF-8
on 03/04/2010 05:14 Jiandong Lu said the following:
> hi,everyone.
> each time I use disk1 to install my FreeBSD system.Now I want to make my own FreeBSD distro.My question is
> how to make disk1?
> thanks.
Try: man 7 release
P.S. this list in not appropriate for questions like this, there is
freebsd-questions@
--
Andriy Gapon
------------------------------
End of freebsd-hackers Digest, Vol 366, Issue 6
***********************************************