"Rewriting kernfs and procfs" - GSoC'12

[ambarisha b]

Hi,

On Fri, Mar 9, 2012 at 1:37 AM, Elad Efrat <el...@netbsd.org> wrote:
> On 2012-03-07 6:12 PM, ambarisha b wrote:
>
>> I also found two other projects not on GSoC page really interesting.
>> "Rewriting kernfs and procfs". Can it be worked on as a GSoC project,
>> because it isn't tagged GSoC?
>
> I want to see that done, by porting the pseudofs layer from FreeBSD and
> rewriting kernfs/procfs in terms of pseudofs. If you decide to do that,
> I can probably mentor it, so let me know.

I couldn't find much documentation about pseudofs on the internet. So,
I dug a little into FreeBSD's code (src/sys/fs/pseudofs/pseudofs.h) to
inspect what pseudofs layer offers. FreeBSD's procfs has been built
over their pseudofs. I studied some parts of it. I, now, have a good
idea of how procfs is using pseudofs. But, to port the pseudofs, how
do I realize the differences between NetBSD and FreeBSD which effect
pseudofs?

The project description page [1] also talks about writing procfs in
such a way as to be able to allow external code to add/delete its
entries dynamically through an API. How would this work and what are
its implications?

Cheers
Ambarish

[1] - http://wiki.netbsd.org/projects/project/kernfs-rewrite/

[Julio Merino]

On 3/18/12 10:24 AM, ambarisha b wrote:
> Hi,
>
> On Fri, Mar 9, 2012 at 1:37 AM, Elad Efrat<el...@netbsd.org> wrote:
>> On 2012-03-07 6:12 PM, ambarisha b wrote:
>>
>>> I also found two other projects not on GSoC page really interesting.
>>> "Rewriting kernfs and procfs". Can it be worked on as a GSoC project,
>>> because it isn't tagged GSoC?

Certainly yes! In particular, this specific project was first listed as
a GSoC project; no idea when it became removed from the list.

>> I want to see that done, by porting the pseudofs layer from FreeBSD and
>> rewriting kernfs/procfs in terms of pseudofs. If you decide to do that,
>> I can probably mentor it, so let me know.
>
> I couldn't find much documentation about pseudofs on the internet. So,
> I dug a little into FreeBSD's code (src/sys/fs/pseudofs/pseudofs.h) to
> inspect what pseudofs layer offers. FreeBSD's procfs has been built
> over their pseudofs. I studied some parts of it. I, now, have a good
> idea of how procfs is using pseudofs. But, to port the pseudofs, how
> do I realize the differences between NetBSD and FreeBSD which effect
> pseudofs?

I don't know much more about pseudofs than what I see in its manual
page, but it indeed seems that taking this approach would be worthwhile.
I imagine that porting pseudofs to NetBSD should not be "too
complicated". There will probably be issues with UVM, locking and kauth
(and maybe others), but these should not be too difficult to adapt to
NetBSD-specific mechanisms.

> The project description page [1] also talks about writing procfs in
> such a way as to be able to allow external code to add/delete its
> entries dynamically through an API. How would this work and what are
> its implications?

I'd recommend to look at the Linux procfs code. They have support for
this particular functionality and you can probably get some good ideas
from there.

Personally, I'd also like to see this project done. It was at one point
an idea I wanted to work on, but then lost the time to do so and
forgotten about it completely.

Looking forward to your project proposal!

[Adam Hamsik]

Hi,

I can't talk for others but I understand it that way that there will be some
sort of a api to create/destroy dynamic entries in procfs.

struct procfs_node {
procfs_node_path;
procfs_node_permissions;
procfs_owner;
procfs_group;

procfs_callback; -> function called when this node is accessed.
};

procfs_register_node(procfs_node *node);
procfs_unregister_node(procfs_node *node);

this would allow for third party modules to create their procfs nodes and destroy
them during module unload.

>
> Cheers
> Ambarish
>
> [1] - http://wiki.netbsd.org/projects/project/kernfs-rewrite/

Regards

Adam.

[Matthew Mondor]

On Tue, 20 Mar 2012 10:35:13 +0900
Julio Merino <jm...@julipedia.org> wrote:

> Personally, I'd also like to see this project done. It was at one point
> an idea I wanted to work on, but then lost the time to do so and
> forgotten about it completely.

I was initially reticent to reply to this thread at this time, because
some details might be out of the scope of the GSoC project. But I
think that those questions are important to consider in the design of a
new procfs implementation, and the project description was very
summary, so I decided to post them anyway:

It was nice to be able to mount procfs with -o linux when I used Linux
binary compatibility. Are there other scenarios where it is required?
If not, should a new implementation simply be as compatible as possible
with Linux, such that -o linux not be necessary? Even some supposedly
portable software occasionally now expect a Linux-compatible procfs
tree.

Otherwise, I think that currently NetBSD doesn't make use of it, as
kernfs and procfs are not mounted on my systems. Is there
functionality that it should provide which
sysctl/vmstat/pmap/fstat/drvctl don't? While on Linux it's used as a
central repository for a lot of information, I regularily stumble on
ad-hoc parsers in a number of applications that query from it,
wondering why they didn't export that information via sysctl...

If it should diverge from Linux and still support -o linux, is there
a particular hierarchical direction it should respect, and suggested
file format(s), i.e. plist is an example, which applications could
parse using a supplied library? Or should the data be in a format
designed for human reading only, with sysctl used for software? I
doubt that a new implementation needs to remain compatible with the
traditional 4.4BSD procfs hierarchy, as it's not really being used by
software yet.

I once thought that it might be useful to export procfs via NFS,
but our current implementation doesn't support it. Is this something
that a new implementation should allow?

Thanks,
--
Matt

[Wolfgang Solfrank]

Hi,

> I was initially reticent to reply to this thread at this time, because
> some details might be out of the scope of the GSoC project. But I
> think that those questions are important to consider in the design of a
> new procfs implementation, and the project description was very
> summary, so I decided to post them anyway:

Given that even Linux walks away from placing non-process related data
into in /proc, slowly putting it into /sys instead, IMHO this is going
in the wrong direction.

> I once thought that it might be useful to export procfs via NFS,
> but our current implementation doesn't support it. Is this something
> that a new implementation should allow?

While I once tried that (quite long ago), it turned out that the buffering
the NFS client does somewhat conflicts the dynamic nature of /proc
and /kern. However, maybe it could work better with NFSv3 or even NFSv4?

Ciao,
Wolfgang
--
Wolf...@Solfrank.net Wolfgang Solfrank

[David Laight]

On Tue, Mar 20, 2012 at 01:45:29PM +0100, Wolfgang Solfrank wrote:
>
> While I once tried that (quite long ago), it turned out that the buffering
> the NFS client does somewhat conflicts the dynamic nature of /proc
> and /kern. However, maybe it could work better with NFSv3 or even NFSv4?

At leastsome versions of nfs stat the enclosing directory - if it's
time has changed it will invalidate the cached contents.

David

--
David Laight: da...@l8s.co.uk

[ambarisha b]

Hi,

I have some progress. I submitted my proposal on google melange. There
are some points on which I would like some feedback:

- How is my schedule? Would it be better to increase/decrease the
allotted time somewhere? I have reserved a buffer in the end incase
there is some problem somewhere.
- Would it be better to add more detail anywhere?
- Is there any other information that is needed?

I have also gone through linux procfs code and added my comments in
the proposal.

Cheers
Ambarisha