Grupos de Google ya no admite nuevas publicaciones ni suscripciones de Usenet. El contenido anterior sigue siendo visible.

ANN: pythonfs - create linux filesystems in python

Visto 0 veces
Saltar al primer mensaje no leído

David McNab

no leída,
8 dic 2003, 7:06:098/12/03
a pytho...@python.org
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

I've just built a framework which allows mountable linux filesystems to
be implemented in pure python.

It consists of a back-end to LUFS (lufs.sf.net), and is available from
http://www.freenet.org.nz/python/lufs-python

Simple to use - after adding the pythonfs code to LUFS, and
building/installing, one only needs to copy the template.py python
module and flesh out the methods in the Filesystem class. Generous
docstrings spell out exactly what is needed.

Have phun

Cheers
David

- --

leave this line intact so your email gets through my junk mail filter
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQE/1GkxUuEuev/PjrcRArSzAJ4gbHZtRN6Di3I/sD/9cCk0m/86igCeLFvi
grZoxZILdpugnNiuwb2fW5E=
=fTSI
-----END PGP SIGNATURE-----


Rene Pijlman

no leída,
8 dic 2003, 8:04:398/12/03
a
David McNab:

>a framework which allows mountable linux filesystems to be
>implemented in pure python.

Serious question: why?

--
René Pijlman

Alan Kennedy

no leída,
8 dic 2003, 8:26:028/12/03
a
[David McNab]

>>a framework which allows mountable linux filesystems to be
>>implemented in pure python.

[Rene Pijlman]
> Serious question: why?

One very serious usage that springs to mind is to implement a
versioned virtual file system, in combination with a version control
system like subversion (subversion.tigris.org).

ClearCase (the versioning system) has the concept of such virtual file
systems. You specify a view of the "Versioned Object Base" (VOB): the
view is specified using dates/labels/revision numbers/etc, and is
specified inside a "config spec" file. [1]

Once you have specified your configspec, ClearCase creates a virtual
file system, in which every file and directory actually resides inside
the ClearCase VOB, so you are actually working directly on versioned
files, even though your filesystem just looks like a plain ordinary
file system: no special tools or techniques required.

Subversion supports python fantastically well:-

o Python scripts can be executed pre- and post-commit[2]
o Python interfaces to all the important subversion libraries are
available, through SWIG[3]

It seems to me that David's work could be the last link in the chain
to creating a complete VFS-based versioning system, using python and
subversion.

And that's just one idea that occurred to me: I'm sure that there are
*plenty* of others.

Nice one David!

regards,

--
alan kennedy
------------------------------------------------------
check http headers here: http://xhaus.com/headers
email alan: http://xhaus.com/contact/alan

[1]
http://www.agsrhichome.bnl.gov/Controls/doc/ClearCaseEnv/v4.0doc/cpf_4.0/ccase_ux/ccdev/ccdev-6.html

[2]
http://svnbook.red-bean.com/html-chunk/ch05s02.html#svn-ch-5-sect-2.1

[3]
http://svnbook.red-bean.com/html-chunk/ch07s02.html#svn-ch-7-sect-2.3

David McNab

no leída,
8 dic 2003, 8:18:298/12/03
a Rene Pijlman,pytho...@python.org
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Rene Pijlman wrote:
|>a framework which allows mountable linux filesystems to be
|>implemented in pure python.
|
| Serious question: why?

Serious answer - why *not*?

Basically, the 'file/directory' tree paradigm comes in handy in a lot of
different ways. In fact, one of the original Unix claims to fame was and
is the 'everything is a file' concept.

If you look at the LUFS site (lufs.sf.net), you'll see a bunch of
experimental filesystems, including (of all things!) gnutellafs.

So once someone has decided to offer a filesystem interface for
something, it can be an immense boost to be able to implement the thing
in Python, which on average takes a fraction of the time it takes to
write similar in C.

When first learning Python, I had a dramatic experience of Python's
leverage, when I had to implement the same set of functions in Python
and C. They took about 20 minutes to implement and debug in Python, but
over 2 hours in C (even though I worked as a senior C developer for 10
years).

- --

Kind regards
David

- --

leave this line intact so your email gets through my junk mail filter
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQE/1HoWUuEuev/PjrcRAiwwAJ9+7hRZsKvZyuAj8dXZe6iyEA0d5ACeLNwb
zKw6xalCBihdocAHtZeQi3s=
=mBdk
-----END PGP SIGNATURE-----


Paul McGuire

no leída,
8 dic 2003, 9:04:558/12/03
a
Here's another idea, piggybacking on the /proc filesystem:

Use this pseudo-filesystem to create a query front-end to a database, using
a file directory application metaphor. The database could contain
information on say patients in a hospital, or lots or equipment in a
factory, or active claims in an insurance office. Then under the main
directory, each claim could have a directory of its own, with sub-files
containing a summary text description, a subdirectory of claim event
"files," and so on. You could also have directories that organize claims by
category or status, with "links" to the actual claim directories in the
associated category directories. All of this with the GUI of a simple file
browser (or a real browser!), and very flexible. If you find out that users
often want to know about claims over a certain amount, or that have remained
uncleared longer than a certain threshold period, just add another
"directory."

In a directory tree, it might look like:
/claims
/claimsByStatus
/active
/pendingEstimates
/pendingMedical
/pendingAdjuster
/pendingLegal
/closed
/contested
/claimsByAge
/claimsByAmount
/allClaimsByClaimant
/A_G
/H_M
...

and so on. In reality there are no "files" or "directories," but this
filesystem utility makes it look like there are.

Or maybe just use this as a flexible rapid prototyping or requirements
gathering technique - then once the structure stabilizes, build into an
application GUI with menus (maybe leave the filesystem around for "expert"
mode).

-- Paul

"Rene Pijlman" <reply.in.th...@my.address.is.invalid> wrote in
message news:fmt8tvcqgmduhj5e6...@4ax.com...

Tim Hoffman

no leída,
8 dic 2003, 10:30:118/12/03
a
You could access ZODB as a filesystem ;-)

Skip Montanaro

no leída,
8 dic 2003, 10:08:438/12/03
a Paul McGuire,pytho...@python.org

Paul> Here's another idea, piggybacking on the /proc filesystem:
...

Here's another idea. Provide a /proc filesystem on systems which don't
support one out of the box.

Skip

Andrew Wilkinson

no leída,
8 dic 2003, 11:56:538/12/03
a
Rene Pijlman wrote:

> David McNab:
>>a framework which allows mountable linux filesystems to be
>>implemented in pure python.
>
> Serious question: why?
>

Or how about a cvs backend to the file system?

*drools* ;-)

Terry Reedy

no leída,
8 dic 2003, 12:32:388/12/03
a

"Rene Pijlman" <reply.in.th...@my.address.is.invalid> wrote in
message news:fmt8tvcqgmduhj5e6...@4ax.com...
> David McNab:
> >a framework which allows mountable linux filesystems to be
> >implemented in pure python.
>
> Serious question: why?

If I wanted to learn about or experiment with linux filesystems, I might do
better to start with this than track down the actual Linux C sources.

TJR


Peter Hansen

no leída,
8 dic 2003, 13:49:548/12/03
a
Rene Pijlman wrote:
>
> David McNab:
> >a framework which allows mountable linux filesystems to be
> >implemented in pure python.
>
> Serious question: why?

Just thought of another possibility: might be useful for doing
unit testing where one could use the Python-based filesystem to
mock a real filesystem, but simulate certain particularly difficult
kinds of errors (corruption, files that go missing, invalid file names)
that might happen in extreme situations.

-Peter

Nick Vargish

no leída,
8 dic 2003, 23:08:228/12/03
a
Rene Pijlman <reply.in.th...@my.address.is.invalid> writes:

> Serious question: why?

How about providing a selection of ways of accessing an mp3
collection? At the top level you could have /artist, /album, /genre,
/year...

Nick

--
# sigmask || 0.2 || 20030107 || public domain || feed this to a python
print reduce(lambda x,y:x+chr(ord(y)-1),' Ojdl!Wbshjti!=obwAcboefstobudi/psh?')

Christos TZOTZIOY

no leída,
9 dic 2003, 1:29:039/12/03
a
On 08 Dec 2003 23:08:22 -0500, rumours say that Nick Vargish
<nav+...@bandersnatch.org> might have written:

>> Serious question: why?
>
>How about providing a selection of ways of accessing an mp3
>collection? At the top level you could have /artist, /album, /genre,
>/year...

And suppose the mp3 files are spread over many computers in the local
network... perhaps the vfs could substitute a file of momentary silence
for every song not available at the moment, so that you never have to
change the playlist...
--
TZOTZIOY, I speak England very best,
Ils sont fous ces Redmontains! --Harddix

Mike Rovner

no leída,
9 dic 2003, 13:51:339/12/03
a pytho...@python.org
Nick Vargish wrote:

> Rene Pijlman <reply.in.th...@my.address.is.invalid> writes:
>> Serious question: why?
>
> How about providing a selection of ways of accessing an mp3
> collection? At the top level you could have /artist, /album, /genre,
> /year...

It reminds me of Microsoft's promise to combine RDBMS and file system in a
new Longhorn OS. Several years ago that idea was highly popular but nothing
came out.

That approach may be very interesting to allow user to use some plug-ins for
the file system
(access, security, notification, etc.).

Mike


Tim Roberts

no leída,
10 dic 2003, 2:36:5210/12/03
a

Yes, but his pythonfs product is extremely Linux-centric, where the /proc
filesystem is ALWAYS available.
--
- Tim Roberts, ti...@probo.com
Providenza & Boekelheide, Inc.

Paul Boddie

no leída,
10 dic 2003, 6:52:3610/12/03
a
"Mike Rovner" <mi...@nospam.com> wrote in message news:<mailman.2.10709958...@python.org>...

>
> It reminds me of Microsoft's promise to combine RDBMS and file system in a
> new Longhorn OS. Several years ago that idea was highly popular but nothing
> came out.

Longhorn isn't supposed to be out before 2006, unless you live in
Malaysia (apparently). Is this a message from the future? ;-)

Actually, I can imagine that it isn't quite equivalent to the
much-hyped Longhorn features and the GNOME Storage stuff, since a
filesystem typically provides a fairly rigid naming interface,
although I'd admit that the same kinds of naming issues superficially
exist with Web applications, and you can always disregard the
hierarchical "assumptions" there and interpret the query string in
your own way. One thing I dislike about various Web frameworks is that
they don't really let you disregard the directory-like nature of the
namespace, forcing you into mod_rewrite and other magic to "name
things your own way".

> That approach may be very interesting to allow user to use some plug-ins for
> the file system (access, security, notification, etc.).

What about doing the kinds of things that KDE's IOSlaves and GNOME's
VFS do, but at a lower layer in the system (and in a sane language, of
course)? Python lets you do HTTP, FTP and loads of other protocols
fairly trivially - imagine hooking that up to a filesystem interface.

Paul

David M. Wilson

no leída,
10 dic 2003, 6:55:1110/12/03
a
"Mike Rovner" <mi...@nospam.com> wrote...

> It reminds me of Microsoft's promise to combine RDBMS and file system in a
> new Longhorn OS. Several years ago that idea was highly popular but nothing
> came out.

> That approach may be very interesting to allow user to use some plug-ins for
> the file system
> (access, security, notification, etc.).

Quite a few people have attempted work in this area, but as with
Microsoft, the plan to combine a traditional database and a
traditional filesystem never seems to come to fruition.

If you are interested in this sort of stuff, the only really serious
attempt I have seen in the recent years is the work of Hans Reiser, of
reiserfs fame. Take a read through the pages at:

http://www.namesys.com/


And possibly pick the thread up over on one of the database or
filesystem news groups. ;)


David.

John J. Lee

no leída,
10 dic 2003, 9:09:4510/12/03
a
Peter Hansen <pe...@engcorp.com> writes:

Better to clobber open() than to risk crashing your kernel, though!

Though maybe rebinding open won't remain legal forever...


John

Skip Montanaro

no leída,
10 dic 2003, 11:56:0110/12/03
a Tim Roberts,pytho...@python.org

Paul> Here's another idea, piggybacking on the /proc filesystem:

>> Here's another idea. Provide a /proc filesystem on systems which don't


>> support one out of the box.

Tim> Yes, but his pythonfs product is extremely Linux-centric, where the
Tim> /proc filesystem is ALWAYS available.

Whoops, yeah...

Skip

Mike Rovner

no leída,
10 dic 2003, 15:45:2110/12/03
a pytho...@python.org
Paul Boddie wrote:
> "Mike Rovner" <mi...@nospam.com> wrote in message
> news:<mailman.2.10709958...@python.org>...
>>
>> It reminds me of Microsoft's promise to combine RDBMS and file
>> system in a new Longhorn OS. Several years ago that idea was highly
>> popular but nothing came out.

You misquoted me a bit ;)
I ment (expanding the previous author) it can be used to provide "different
views
to the same data" (database) in contrast to the "same view on the different
data" aka filesystem.

> Longhorn isn't supposed to be out before 2006, unless you live in
> Malaysia (apparently). Is this a message from the future? ;-)

I red 2005. So I live on the same planet at least. :)

> Actually, I can imagine that it isn't quite equivalent to the
> much-hyped Longhorn features and the GNOME Storage stuff, since a
> filesystem typically provides a fairly rigid naming interface,

No. The whole idea was to invert that. You use familiar approach (and api)
to get to dynamic data (as in /proc).

>> That approach may be very interesting to allow user to use some
>> plug-ins for the file system (access, security, notification, etc.).
>

> course)? Python lets you do HTTP, FTP and loads of other protocols
> fairly trivially - imagine hooking that up to a filesystem interface.

Right, the whole universe. Don't fix up on my list.

Mike


Mike Rovner

no leída,
10 dic 2003, 15:54:2010/12/03
a pytho...@python.org
David M. Wilson wrote:
> Microsoft, the plan to combine a traditional database and a
> traditional filesystem never seems to come to fruition.

AFAIK Microdoft never come out with revolutionary ideas,
it makes mass-production of already proved concepts and excel in it.
The idea of implementing filesystem interface on top of proved database
pluses like transaction isolation, integrity, changes propagation and
backup/restore
mechanisms may come to something usefull due to them.

> If you are interested in this sort of stuff, the only really serious

Not really but thank you for the link.

Mike


Nick Vargish

no leída,
10 dic 2003, 23:45:3510/12/03
a
pa...@boddie.net (Paul Boddie) writes:

> course)? Python lets you do HTTP, FTP and loads of other protocols
> fairly trivially - imagine hooking that up to a filesystem interface.

Accessing an ftp site by mounting it... I like it. Another obvious
application would be accessing archive files by mounting them (wasn't
that one of the original features planned for hurd?).

JanC

no leída,
11 dic 2003, 0:00:2411/12/03
a
Nick Vargish <nav+...@bandersnatch.org> schreef:

>> Python lets you do HTTP, FTP and loads of other protocols
>> fairly trivially - imagine hooking that up to a filesystem interface.
>
> Accessing an ftp site by mounting it... I like it. Another obvious
> application would be accessing archive files by mounting them (wasn't
> that one of the original features planned for hurd?).

Pythonfs is based on LUFS, which includes a.o. FTPFS, SSHFS & GnutellaFS as
examples...

--
JanC

"Be strict when sending and tolerant when receiving."
RFC 1958 - Architectural Principles of the Internet - section 3.9

Bengt Richter

no leída,
12 dic 2003, 1:10:4112/12/03
a
On Tue, 09 Dec 2003 01:06:09 +1300, David McNab <da...@rebirthing.co.nz> wrote:

>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>Hi,
>
>I've just built a framework which allows mountable linux filesystems to
>be implemented in pure python.
>
>It consists of a back-end to LUFS (lufs.sf.net), and is available from
>http://www.freenet.org.nz/python/lufs-python
>
>Simple to use - after adding the pythonfs code to LUFS, and
>building/installing, one only needs to copy the template.py python
>module and flesh out the methods in the Filesystem class. Generous
>docstrings spell out exactly what is needed.
>

Thanks for the link. Maybe it can help if I ever get to trying some of the ideas in

http://groups.google.com/groups?selm=begle0%241fs%240%40216.39.172.122

I think using a vfs as an interface to windows/widget hierarchy would be particularly interesting.
And if the package noticed the difference between opening a widget-as-file in binary vs text mode,
one could have efficient marshalled packed data formats a la cpickle to define and initialize
GUI structures, as well as text interfaces for interactive noodling.

If it could be built on top of a low level video HAL a la frame buffer or maybe simulated via directx
or such, it could be interesting. If there was system support so that different processes could open
differently owned GUI subtrees from a shared "desktop" root, that you could switch between like F-keys
on linux between console windows, it might also lead to a different way of handling user login/access
control, and ... ramble, ramble ;-)

It would also make an interesting interface for output of graphic formats, whether graphs and plots,
or thin wrappers around pdf/postscript/svg/latex/windows-metafile/etc static formats, not to mention
smil and multimedia etc.

Regards,
Bengt Richter

0 mensajes nuevos