Hi,
I've just built a framework which allows mountable linux filesystems to
be implemented in pure python.
It consists of a back-end to LUFS (lufs.sf.net), and is available from
http://www.freenet.org.nz/python/lufs-python
Simple to use - after adding the pythonfs code to LUFS, and
building/installing, one only needs to copy the template.py python
module and flesh out the methods in the Filesystem class. Generous
docstrings spell out exactly what is needed.
Have phun
Cheers
David
- --
leave this line intact so your email gets through my junk mail filter
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQE/1GkxUuEuev/PjrcRArSzAJ4gbHZtRN6Di3I/sD/9cCk0m/86igCeLFvi
grZoxZILdpugnNiuwb2fW5E=
=fTSI
-----END PGP SIGNATURE-----
Serious question: why?
--
René Pijlman
[Rene Pijlman]
> Serious question: why?
One very serious usage that springs to mind is to implement a
versioned virtual file system, in combination with a version control
system like subversion (subversion.tigris.org).
ClearCase (the versioning system) has the concept of such virtual file
systems. You specify a view of the "Versioned Object Base" (VOB): the
view is specified using dates/labels/revision numbers/etc, and is
specified inside a "config spec" file. [1]
Once you have specified your configspec, ClearCase creates a virtual
file system, in which every file and directory actually resides inside
the ClearCase VOB, so you are actually working directly on versioned
files, even though your filesystem just looks like a plain ordinary
file system: no special tools or techniques required.
Subversion supports python fantastically well:-
o Python scripts can be executed pre- and post-commit[2]
o Python interfaces to all the important subversion libraries are
available, through SWIG[3]
It seems to me that David's work could be the last link in the chain
to creating a complete VFS-based versioning system, using python and
subversion.
And that's just one idea that occurred to me: I'm sure that there are
*plenty* of others.
Nice one David!
regards,
--
alan kennedy
------------------------------------------------------
check http headers here: http://xhaus.com/headers
email alan: http://xhaus.com/contact/alan
[1]
http://www.agsrhichome.bnl.gov/Controls/doc/ClearCaseEnv/v4.0doc/cpf_4.0/ccase_ux/ccdev/ccdev-6.html
[2]
http://svnbook.red-bean.com/html-chunk/ch05s02.html#svn-ch-5-sect-2.1
[3]
http://svnbook.red-bean.com/html-chunk/ch07s02.html#svn-ch-7-sect-2.3
Rene Pijlman wrote:
|>a framework which allows mountable linux filesystems to be
|>implemented in pure python.
|
| Serious question: why?
Serious answer - why *not*?
Basically, the 'file/directory' tree paradigm comes in handy in a lot of
different ways. In fact, one of the original Unix claims to fame was and
is the 'everything is a file' concept.
If you look at the LUFS site (lufs.sf.net), you'll see a bunch of
experimental filesystems, including (of all things!) gnutellafs.
So once someone has decided to offer a filesystem interface for
something, it can be an immense boost to be able to implement the thing
in Python, which on average takes a fraction of the time it takes to
write similar in C.
When first learning Python, I had a dramatic experience of Python's
leverage, when I had to implement the same set of functions in Python
and C. They took about 20 minutes to implement and debug in Python, but
over 2 hours in C (even though I worked as a senior C developer for 10
years).
- --
Kind regards
David
- --
leave this line intact so your email gets through my junk mail filter
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQE/1HoWUuEuev/PjrcRAiwwAJ9+7hRZsKvZyuAj8dXZe6iyEA0d5ACeLNwb
zKw6xalCBihdocAHtZeQi3s=
=mBdk
-----END PGP SIGNATURE-----
Use this pseudo-filesystem to create a query front-end to a database, using
a file directory application metaphor. The database could contain
information on say patients in a hospital, or lots or equipment in a
factory, or active claims in an insurance office. Then under the main
directory, each claim could have a directory of its own, with sub-files
containing a summary text description, a subdirectory of claim event
"files," and so on. You could also have directories that organize claims by
category or status, with "links" to the actual claim directories in the
associated category directories. All of this with the GUI of a simple file
browser (or a real browser!), and very flexible. If you find out that users
often want to know about claims over a certain amount, or that have remained
uncleared longer than a certain threshold period, just add another
"directory."
In a directory tree, it might look like:
/claims
/claimsByStatus
/active
/pendingEstimates
/pendingMedical
/pendingAdjuster
/pendingLegal
/closed
/contested
/claimsByAge
/claimsByAmount
/allClaimsByClaimant
/A_G
/H_M
...
and so on. In reality there are no "files" or "directories," but this
filesystem utility makes it look like there are.
Or maybe just use this as a flexible rapid prototyping or requirements
gathering technique - then once the structure stabilizes, build into an
application GUI with menus (maybe leave the filesystem around for "expert"
mode).
-- Paul
"Rene Pijlman" <reply.in.th...@my.address.is.invalid> wrote in
message news:fmt8tvcqgmduhj5e6...@4ax.com...
Here's another idea. Provide a /proc filesystem on systems which don't
support one out of the box.
Skip
> David McNab:
>>a framework which allows mountable linux filesystems to be
>>implemented in pure python.
>
> Serious question: why?
>
Or how about a cvs backend to the file system?
*drools* ;-)
If I wanted to learn about or experiment with linux filesystems, I might do
better to start with this than track down the actual Linux C sources.
TJR
Just thought of another possibility: might be useful for doing
unit testing where one could use the Python-based filesystem to
mock a real filesystem, but simulate certain particularly difficult
kinds of errors (corruption, files that go missing, invalid file names)
that might happen in extreme situations.
-Peter
> Serious question: why?
How about providing a selection of ways of accessing an mp3
collection? At the top level you could have /artist, /album, /genre,
/year...
Nick
--
# sigmask || 0.2 || 20030107 || public domain || feed this to a python
print reduce(lambda x,y:x+chr(ord(y)-1),' Ojdl!Wbshjti!=obwAcboefstobudi/psh?')
>> Serious question: why?
>
>How about providing a selection of ways of accessing an mp3
>collection? At the top level you could have /artist, /album, /genre,
>/year...
And suppose the mp3 files are spread over many computers in the local
network... perhaps the vfs could substitute a file of momentary silence
for every song not available at the moment, so that you never have to
change the playlist...
--
TZOTZIOY, I speak England very best,
Ils sont fous ces Redmontains! --Harddix
> Rene Pijlman <reply.in.th...@my.address.is.invalid> writes:
>> Serious question: why?
>
> How about providing a selection of ways of accessing an mp3
> collection? At the top level you could have /artist, /album, /genre,
> /year...
It reminds me of Microsoft's promise to combine RDBMS and file system in a
new Longhorn OS. Several years ago that idea was highly popular but nothing
came out.
That approach may be very interesting to allow user to use some plug-ins for
the file system
(access, security, notification, etc.).
Mike
Yes, but his pythonfs product is extremely Linux-centric, where the /proc
filesystem is ALWAYS available.
--
- Tim Roberts, ti...@probo.com
Providenza & Boekelheide, Inc.
Longhorn isn't supposed to be out before 2006, unless you live in
Malaysia (apparently). Is this a message from the future? ;-)
Actually, I can imagine that it isn't quite equivalent to the
much-hyped Longhorn features and the GNOME Storage stuff, since a
filesystem typically provides a fairly rigid naming interface,
although I'd admit that the same kinds of naming issues superficially
exist with Web applications, and you can always disregard the
hierarchical "assumptions" there and interpret the query string in
your own way. One thing I dislike about various Web frameworks is that
they don't really let you disregard the directory-like nature of the
namespace, forcing you into mod_rewrite and other magic to "name
things your own way".
> That approach may be very interesting to allow user to use some plug-ins for
> the file system (access, security, notification, etc.).
What about doing the kinds of things that KDE's IOSlaves and GNOME's
VFS do, but at a lower layer in the system (and in a sane language, of
course)? Python lets you do HTTP, FTP and loads of other protocols
fairly trivially - imagine hooking that up to a filesystem interface.
Paul
> It reminds me of Microsoft's promise to combine RDBMS and file system in a
> new Longhorn OS. Several years ago that idea was highly popular but nothing
> came out.
> That approach may be very interesting to allow user to use some plug-ins for
> the file system
> (access, security, notification, etc.).
Quite a few people have attempted work in this area, but as with
Microsoft, the plan to combine a traditional database and a
traditional filesystem never seems to come to fruition.
If you are interested in this sort of stuff, the only really serious
attempt I have seen in the recent years is the work of Hans Reiser, of
reiserfs fame. Take a read through the pages at:
And possibly pick the thread up over on one of the database or
filesystem news groups. ;)
David.
Better to clobber open() than to risk crashing your kernel, though!
Though maybe rebinding open won't remain legal forever...
John
>> Here's another idea. Provide a /proc filesystem on systems which don't
>> support one out of the box.
Tim> Yes, but his pythonfs product is extremely Linux-centric, where the
Tim> /proc filesystem is ALWAYS available.
Whoops, yeah...
Skip
You misquoted me a bit ;)
I ment (expanding the previous author) it can be used to provide "different
views
to the same data" (database) in contrast to the "same view on the different
data" aka filesystem.
> Longhorn isn't supposed to be out before 2006, unless you live in
> Malaysia (apparently). Is this a message from the future? ;-)
I red 2005. So I live on the same planet at least. :)
> Actually, I can imagine that it isn't quite equivalent to the
> much-hyped Longhorn features and the GNOME Storage stuff, since a
> filesystem typically provides a fairly rigid naming interface,
No. The whole idea was to invert that. You use familiar approach (and api)
to get to dynamic data (as in /proc).
>> That approach may be very interesting to allow user to use some
>> plug-ins for the file system (access, security, notification, etc.).
>
> course)? Python lets you do HTTP, FTP and loads of other protocols
> fairly trivially - imagine hooking that up to a filesystem interface.
Right, the whole universe. Don't fix up on my list.
Mike
AFAIK Microdoft never come out with revolutionary ideas,
it makes mass-production of already proved concepts and excel in it.
The idea of implementing filesystem interface on top of proved database
pluses like transaction isolation, integrity, changes propagation and
backup/restore
mechanisms may come to something usefull due to them.
> If you are interested in this sort of stuff, the only really serious
Not really but thank you for the link.
Mike
> course)? Python lets you do HTTP, FTP and loads of other protocols
> fairly trivially - imagine hooking that up to a filesystem interface.
Accessing an ftp site by mounting it... I like it. Another obvious
application would be accessing archive files by mounting them (wasn't
that one of the original features planned for hurd?).
>> Python lets you do HTTP, FTP and loads of other protocols
>> fairly trivially - imagine hooking that up to a filesystem interface.
>
> Accessing an ftp site by mounting it... I like it. Another obvious
> application would be accessing archive files by mounting them (wasn't
> that one of the original features planned for hurd?).
Pythonfs is based on LUFS, which includes a.o. FTPFS, SSHFS & GnutellaFS as
examples...
--
JanC
"Be strict when sending and tolerant when receiving."
RFC 1958 - Architectural Principles of the Internet - section 3.9
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>Hi,
>
>I've just built a framework which allows mountable linux filesystems to
>be implemented in pure python.
>
>It consists of a back-end to LUFS (lufs.sf.net), and is available from
>http://www.freenet.org.nz/python/lufs-python
>
>Simple to use - after adding the pythonfs code to LUFS, and
>building/installing, one only needs to copy the template.py python
>module and flesh out the methods in the Filesystem class. Generous
>docstrings spell out exactly what is needed.
>
Thanks for the link. Maybe it can help if I ever get to trying some of the ideas in
http://groups.google.com/groups?selm=begle0%241fs%240%40216.39.172.122
I think using a vfs as an interface to windows/widget hierarchy would be particularly interesting.
And if the package noticed the difference between opening a widget-as-file in binary vs text mode,
one could have efficient marshalled packed data formats a la cpickle to define and initialize
GUI structures, as well as text interfaces for interactive noodling.
If it could be built on top of a low level video HAL a la frame buffer or maybe simulated via directx
or such, it could be interesting. If there was system support so that different processes could open
differently owned GUI subtrees from a shared "desktop" root, that you could switch between like F-keys
on linux between console windows, it might also lead to a different way of handling user login/access
control, and ... ramble, ramble ;-)
It would also make an interesting interface for output of graphic formats, whether graphs and plots,
or thin wrappers around pdf/postscript/svg/latex/windows-metafile/etc static formats, not to mention
smil and multimedia etc.
Regards,
Bengt Richter