freebsd-hackers-digest V5 #728
owner-freebsd-...@freebsd.org
freebsd-hackers-digest Monday, February 24 2003 Volume 05 : Number 728
In this issue:
Re: Replacement for get_user_pages() of Linux
Re: Replacement for get_user_pages() of Linux
Re: Replacement for get_user_pages() of Linux
Re: Replacement for get_user_pages() of Linux
Re: scan_ffs for UFS2
Re: C coding editor
Re: C coding editor
freebsd-hackers, LIVE FROM WALL STREET: VICC Test Results Are In...
Re: scan_ffs for UFS2
Re: arc4random() range
Re: Replacement for get_user_pages() of Linux
----------------------------------------------------------------------
Date: Sun, 23 Feb 2003 00:49:31 -0800
From: Terry Lambert <tlam...@mindspring.com>
Subject: Re: Replacement for get_user_pages() of Linux
Byunghyun Oh wrote:
> I'm porting Plex86 x86 VM, which uses get_user_pages() function at
> Linux-version kernel module to find and pin physical pages of memory
> in user space (according to its documentation). I tried many
> candidates as its replacement (PHYS_TO_VM_PAGE() macro in vm/vm_page.h
> seems most useful now), but they haven't worked at all.
>
> Any experience about porting VM-related things in Linux will be
> appreciated. :)
I've been unable to find any documentation on get_user_pages(),
and you didn't provide a link to any.
But looking at the source code, the reason for doing this is to
permit DMA directly into user pages.
I don't understand what you mean by "pin", in this context.
You are aware that FreeBSD has a unified VM and buffer cache, and
all user pages for the current process are automatically visible
in th kernel address space, with no need to call something like
get_user_pages() to establish a mapping, right?
Is the intent of this to allow the process memory to be addressed,
even though the process is not active? If so, then what you want
to do is establish a kernel mapping for the pages, and mark them
non-swappable.
Otherwise, what you want is automatic (see uiomove).
- -- Terry
------------------------------
Date: Sun, 23 Feb 2003 09:49:58 +0000
From: Christoph Hellwig <h...@infradead.org>
Subject: Re: Replacement for get_user_pages() of Linux
On Sun, Feb 23, 2003 at 12:49:31AM -0800, Terry Lambert wrote:
> I've been unable to find any documentation on get_user_pages(),
> and you didn't provide a link to any.
>
> But looking at the source code, the reason for doing this is to
> permit DMA directly into user pages.
>
> I don't understand what you mean by "pin", in this context.
get_user_pages() does the following:
(1) force all pages into physical memory if they weren't before
(2) increment the usage count on the to avoid paging them out
The latter is usually called page pinning.
> You are aware that FreeBSD has a unified VM and buffer cache, and
> all user pages for the current process are automatically visible
> in th kernel address space, with no need to call something like
> get_user_pages() to establish a mapping, right?
get_user_pages() does not establish a mapping, in Linux you don't need
a kernel mapping to perform DMA on memory.
------------------------------
Date: Sun, 23 Feb 2003 02:17:23 -0800
From: Terry Lambert <tlam...@mindspring.com>
Subject: Re: Replacement for get_user_pages() of Linux
Christoph Hellwig wrote:
> On Sun, Feb 23, 2003 at 12:49:31AM -0800, Terry Lambert wrote:
> > I've been unable to find any documentation on get_user_pages(),
> > and you didn't provide a link to any.
> >
> > But looking at the source code, the reason for doing this is to
> > permit DMA directly into user pages.
> >
> > I don't understand what you mean by "pin", in this context.
>
> get_user_pages() does the following:
> (1) force all pages into physical memory if they weren't before
> (2) increment the usage count on the to avoid paging them out
>
> The latter is usually called page pinning.
OK, you mean "make non-pageable".
The question, I guess, is "why?". Are you trying to do a delayed
operation that will complete when the process has otherwise been
swapped out?
Is this something that's going to be initiated by the process itself,
or does it have to work on a process that is not the current process
(e.g. P1 sets up the operation to happen to pages in P2, vs. P1 sets
up the operation to happen to pages in P1, and P2 sets up the operation
to happen to pages in P2)?
In other words, are you trying to pin pages that have resident mappings
at the time you want pin them, or are you trying to pin pages that may
not have resident mappings at the time you want to pin them?
> > You are aware that FreeBSD has a unified VM and buffer cache, and
> > all user pages for the current process are automatically visible
> > in th kernel address space, with no need to call something like
> > get_user_pages() to establish a mapping, right?
>
> get_user_pages() does not establish a mapping, in Linux you don't need
> a kernel mapping to perform DMA on memory.
In FreeBSD, you generally do. First off, the VM and buffer cache is
unified. That means that's there's no such thing as a buffer that
exists seperately from the VM system.
Second off, you aren't really telling us who is doing the DMA; if it's
something for which there's an existing device driver, like a disk or
whatever, then that's the way it is.
Thirdly, when you do demand paged I/O, you always do it into a mapped
page, which is then COW mapped into the process (e.g. via mmap(), or
mapped directly because it's a page in a process). In other words,
there's no such thing as an uncached page.
Mabe it would be useful for you to tell us what problem you were
trying to solve; I'm sure if you were to do that, then you would
get a lot of suggestions of how to go about solving it.
- -- Terry
------------------------------
Date: Sun, 23 Feb 2003 10:35:37 +0000
From: Christoph Hellwig <h...@infradead.org>
Subject: Re: Replacement for get_user_pages() of Linux
On Sun, Feb 23, 2003 at 02:17:23AM -0800, Terry Lambert wrote:
> OK, you mean "make non-pageable".
Well, I didn't write the initial mail :)
> The question, I guess, is "why?". Are you trying to do a delayed
> operation that will complete when the process has otherwise been
> swapped out?
well, I don't plan to do anything. The usual way get_user_pages is used on
linux is:
get_user_pages()
perform scatter gatter dma to some device on the pages
unping pages
as linux VM scanning is fully mutithreaded pages could get paged out
during the dma if you didn't pin them so this is needed.
> > get_user_pages() does not establish a mapping, in Linux you don't need
> > a kernel mapping to perform DMA on memory.
>
> In FreeBSD, you generally do. First off, the VM and buffer cache is
> unified. That means that's there's no such thing as a buffer that
> exists seperately from the VM system.
Who talks about buffers? And yes, in Linux you can have buffers that
have pages attached to it that aren't mapped into kernel virtual space,
in fact that's usual for pages used for actual filesystem data if you
have enough memory.
------------------------------
Date: Sun, 23 Feb 2003 15:40:20 +0100
From: Michael Ranner <mra...@inode.at>
Subject: Re: scan_ffs for UFS2
Am Freitag, 21. Februar 2003 22:38 schrieben Sie:
> At 10:20 PM +0100 2/19/03, Michael Ranner wrote:
>
> For what it's worth, we (FreeBSD) have a simple SuperBlock recovery
> program in /usr/src/tools/tools/find-sb. I picked up some updates
> from Dave Cross for that, and have a few more of my own. I just
> have to sit down and get them all together into a single version.
> I don't know how find-sb compares to the program you're talking
> about, but they sound kind of similar.
Scan_ffs can print the lost disklabel for use with disklabel(8).
Find-sb, that version from cvs, seems only to print info about
the superblocks of each file system and you have to rebuild the
lost disklabel for your self. I was on the search for a simple
tool for everybody, and found scan_ffs several months ago
in the OpenBSD distribution (so it seems find-sb was like
reinventing the wheel) and Robert Watson suggested in a posting
december last year to adopt scan_ffs for UFS2. IMHO we should
reuse a already written program with existing man pages, so
we have the same sounding tools for the same tasks.
I don't know what find-sb could do with your patches applied.
If it's superior to scan_ffs, we can forget scan_ffs.
- --
/\/\ichael Ranner
mra...@jawa.at - mra...@bitonline.cc - webm...@mariazell.at
- ----------------------------------------------------------------------
JAWA Management Software GmbH - http://www.jawa.at/
Liebenauer Hauptstrasse 2oo - A-8041 Graz
Tel +43 316 403274 21 - Fax +43 316 403274 10
- ----------------------------------------------------------------------
Mariazell Online - http://www.mariazell.at/
- ----------------------------------------------------------------------
- -----BEGIN GEEK CODE BLOCK-----
GIT/CS/AT dx(-) s+:(++:) a- C++ UBLVS++++$ P++>+++$ L-(+)$ E---
W+++$ N+(++) o-- K- w--()$ O-(--) M@ V-(--) PS+>++ PE(-) Y+ PGP(-)
t+ 5+ X+++(++++) R* tv++ b+(++) DI++ D-(--) G- e h--(*) r++ y?
- ------END GEEK CODE BLOCK------
------------------------------
Date: Sun, 23 Feb 2003 10:17:16 -0800
From: Wes Peters <w...@softweyr.com>
Subject: Re: C coding editor
On Friday 21 February 2003 04:21 am, Clemens Hermann wrote:
> Hi,
>
> what are your favourite editors for coding C? While vi on the first
> terminal, cc on second and runs on the third is fine for very small
> things I doubt it is the way people do it here.
Terminal? You have heard of this really cool thing called windowing
software? ;^)
I completely utterly fail to understand why some young developers attach
some sort of romance to writing code on an 80x25 screen, when all the
haxxors my age or older waited (or slaved away) for years, even
decades, to get something better and more flexible.
I've seen vim, emacs/xemacs, and kdevelop all mentioned in this thread.
I'd just like to point out that the first three have great advantages
under X and the last runs exclusively on X (at least on UNIX it does).
X is for programmers, too. Try it, you'll like it. You might even
find a use for that mouse.
- --
"Where am I, and what am I doing in this handbasket?"
Wes Peters Softweyr LLC
w...@softweyr.com http://softweyr.com/
------------------------------
Date: Sun, 23 Feb 2003 11:21:29 -0800
From: Kent Stewart <kste...@owt.com>
Subject: Re: C coding editor
On Sunday 23 February 2003 10:17 am, Wes Peters wrote:
> On Friday 21 February 2003 04:21 am, Clemens Hermann wrote:
> > Hi,
> >
> > what are your favourite editors for coding C? While vi on the first
> > terminal, cc on second and runs on the third is fine for very small
> > things I doubt it is the way people do it here.
>
> Terminal? You have heard of this really cool thing called windowing
> software? ;^)
>
> I completely utterly fail to understand why some young developers
> attach some sort of romance to writing code on an 80x25 screen, when
> all the haxxors my age or older waited (or slaved away) for years,
> even decades, to get something better and more flexible.
>
I love comments like this. We used Microsoft's developer environment to
update Unix (HPs and a Cray) for years because that was all we had
other than vi. You could click something in Microsoft, right click it
and you would see sample code, the headers you needed, and etc. The
programs were under strict code control that was acceptable to the USA
NRC.
I see the same thing more or less using the 'Crusader', 'Code Warrior',
or kdevelop environment. In a co-conversion effort using DEC Fortran
and f77 on FreeBSD, msdbg would roll over and die but kdbg would show
you the signal error. Unfortunately, it wouldn't show you why the line
was dying. Writing a one line message of parameters produced a 70 MB
file with no clue why it was dying. Msdbg would actually debug the line
and show you which pointer went out of range but it would die if you
just let it run and not tell you where. Combining the effort produced a
solution neither was capable of by themselves in a reasonable amount of
time.
It all goes under my heading of "what part of making your life easier
don't you understand".
Have a good day,
Kent
> I've seen vim, emacs/xemacs, and kdevelop all mentioned in this
> thread. I'd just like to point out that the first three have great
> advantages under X and the last runs exclusively on X (at least on
> UNIX it does). X is for programmers, too. Try it, you'll like it.
> You might even find a use for that mouse.
- --
Kent Stewart
Richland, WA
http://users.owt.com/kstewart/index.html
------------------------------
Date: Wed, 19 Feb 2003 04:54:38 GMT
From: incomin...@cs.com
Subject: freebsd-hackers, LIVE FROM WALL STREET: VICC Test Results Are In...
freebsd...@freebsd.org
<p>If you bought into our last recommendation (CIMG) early enough you had an excellent opportunity to make substantial gains (from .90 to 1.65 in just the first day). Now is your chance to do the same with our newest pick: VICC. To find out more go to <a href="http://12.148.59.67">Live From the Street</a>.</p>
<p align="center"><img border="0" src="http://12.148.59.67/bm01.gif"></p>
<p>If you no longer want to receive information from us just go to
<a href="mailto:tal...@cs.com">tal...@cs.com</a>.<p>
------------------------------
Date: Sun, 23 Feb 2003 18:54:49 -0500
From: Garance A Drosihn <dro...@rpi.edu>
Subject: Re: scan_ffs for UFS2
At 3:40 PM +0100 2/23/03, Michael Ranner wrote:
>Am Freitag, 21. Februar 2003 Garance wrote:
> >
> > I don't know how find-sb compares to the program you're
> > talking about, but they sound kind of similar.
>
>Scan_ffs can print the lost disklabel for use with disklabel(8).
>Find-sb, that version from cvs, seems only to print info about
>the superblocks of each file system and you have to rebuild the
>lost disklabel for your self. [...]
>
>I don't know what find-sb could do with your patches applied.
>If it's superior to scan_ffs, we can forget scan_ffs.
It sounds like you should go with scan_ffs, and when I have the
time to sort out my updates I'll see how they apply to that.
- --
Garance Alistair Drosehn = g...@gilead.netel.rpi.edu
Senior Systems Programmer or g...@freebsd.org
Rensselaer Polytechnic Institute or dro...@rpi.edu
------------------------------
Date: Fri, 21 Feb 2003 11:29:00 +1030
From: Greg 'groggy' Lehey <gr...@FreeBSD.org>
Subject: Re: arc4random() range
- --8bBEDOJVaa9YlTAt
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Wednesday, 19 February 2003 at 9:22:18 -0800, Wes Peters wrote:
> On Tuesday 18 February 2003 22:36, Peter Jeremy wrote:
>>
>> I see this as a major advantage of arc4random() - if I want 32-bit
>> random numbers I don't have to call random() twice and merge the
>> results. I've never understood why random() was specified to return
>> a '0' in the MSB.
>
> It probably had something to do with the PDP-11 architecture. This
> rings a bell, but I can't recall what it was. Greg Lehey might be
> able to help here, he has far better knowlege of the Good Old Days(tm)
> than I do.=20
Difficult to say. I don't think that random() was in the Seventh
Edition. They used rand() instead. Read the code and shudder:
static long randx =3D 1;
srand(x)
unsigned x;
{
randx =3D x;
}
rand()
{
return(((randx =3D randx*1103515245 + 12345)>>16) & 077777);
}
That's the entire content of Seventh Edition /usr/src/libc/gen/rand.c.
Greg
- --
See complete headers for address and phone numbers
Please note: we block mail from major spammers, notably yahoo.com.
See http://www.lemis.com/yahoospam.html for further details.
- --8bBEDOJVaa9YlTAt
Content-Type: application/pgp-signature
Content-Disposition: inline
- -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.0 (FreeBSD)
iD8DBQE+VXnUIubykFB6QiMRAk2/AKCD2dlaSVKw2X87WIt4pn/q+P4kXQCePaG8
JHbCPtSKY4TS8RfFQtbbKmY=
=FBKr
- -----END PGP SIGNATURE-----
- --8bBEDOJVaa9YlTAt--
------------------------------
Date: Mon, 24 Feb 2003 02:03:46 -0800
From: David Schultz <d...@FreeBSD.ORG>
Subject: Re: Replacement for get_user_pages() of Linux
Thus spake Byunghyun Oh <octp...@postech.ac.kr>:
> I'm porting Plex86 x86 VM, which uses get_user_pages() function at
> Linux-version kernel module to find and pin physical pages of memory
> in user space (according to its documentation). I tried many
> candidates as its replacement (PHYS_TO_VM_PAGE() macro in vm/vm_page.h
> seems most useful now), but they haven't worked at all.
>
> Any experience about porting VM-related things in Linux will be
> appreciated. :)
Glancing at the Linux source, it looks like you want vm_map_wire().
BTW, in the future, it helps if you can describe what you're
looking for, since we're not all Linux experts.
------------------------------
End of freebsd-hackers-digest V5 #728
*************************************
To Unsubscribe: send mail to majo...@FreeBSD.org
with unsubscribe freebsd-hackers-digest in the body of the message