Does python have the capability for driver development ?
MalC0de
I want to know does python have any capability for using Ring0 and
kernel functions for driver and device development stuff .
if there's such a feature it is very good, and if there something for
this kind that you know please refer me to some reference and show me
some snippet .
thanks
- Malc0de
Diez B. Roggisch
No, it can't do such things. At least it isn't embedded in the kernel -
in theory that might be possible, but given the timing-constraints and
concurrency-requirements, it's not really feasible.
Diez
Nick Craig-Wood
You can write FUSE (file systems in userspace) drivers in python I believe.
Not the same as running in ring0 but in most senses a kernel driver...
--
Nick Craig-Wood <ni...@craig-wood.com> -- http://www.craig-wood.com/nick
Martin P. Hellwig
Python is interpreted, so the first requirement would be that the
interpreter (the python VM to be more precise) would run in the kernel
or that there is a way for the interpreter to delegate operations to
kernel restricted operations. Most notably access to the memory location
of the hardware you want to write a driver for and possibly also a way
to pass through a callback function for triggered interrupt coming from
the hardware.
So technically speaking it shouldn't be impossible. And there is perhaps
something to say for being able to write drivers in a readable/easy
programming language however I am afraid that if such a beast would be
made that it would remain an academical exercise only due to performance
constraints.
Though I would love to play around with a system where the kernel is
essentially only a python interpreter, with full raw access to the hardware.
But creating such a thing requires more talent and understanding than
currently (and probably ever) in my possession.
--
MPH
http://blog.dcuktec.com
'If consumed, best digested with added seasoning to own preference.'
David Lyon
> hello there, I've a question :
> I want to know does python have any capability for using Ring0 and
> kernel functions for driver and device development stuff .
> if there's such a feature it is very good, and if there something for
> this kind that you know please refer me to some reference and show me
> some snippet .
What operating system are you talking about?
Most device drivers run at ring 3 (or lower) and not zero. This way if
there
is a driver crash the whole operating system doesn't freeze.
Python is generally considered a high-level language. If you want
to play around with drivers.. usb serial.. etc do it at a python
level through the existing device drivers.
imho the performance of interpreted python isn't compatible with
writing block-mode device drivers (hard-disks) and so forth.
What hardware do you have that you need to write a device driver
for ? Isn't there a device driver available already? or do you
mean just a device controller?
David
Martin P. Hellwig
> <cut>
> What about user level device drivers? Think the Python VM could
> communicate with the driver through the user space API. Is there a
> Python module for that?
>
>
Sure why not?
Look for example to libusb, which provides a userspace environment and
pyusb which uses that and provides an interface to Python.
--
MPH
MalC0de
you know, there's A kit name DDK available at microsoft's website for
developing device drivers under C / C++ environment, now i'm working
with this kind of toolkit, but when I started programming with Python,
I saw this is very good and easy to develop application in all aspects
of a programmer or customer .
I love python cuze I can write every application i want, it's easy and
it's very robust, But i thought if there's any available toolkit like
Microsoft DDK which can program driver under windows but based on
python it's very good and well, I don't know it's the lack of python
or interpreter concept but cuze of the extensibility of python if such
a good feature can be added to python it will be good enough.
actually I want to write device drivers in Ring0 mode or kernel
mode ...
please introduce me some resource for system programming, I know that
python is very well at system programming level.
thanks
- Malc0de
Marcus Wanner
Marcus
Rodrigo S Wanderley
> Python is interpreted, so the first requirement would be that the
> interpreter (the python VM to be more precise) would run in the kernel
> or that there is a way for the interpreter to delegate operations to
> kernel restricted operations. Most notably access to the memory
> location of the hardware you want to write a driver for and possibly
> also a way to pass through a callback function for triggered
> interrupt coming from the hardware.
>
> So technically speaking it shouldn't be impossible. And there is
> perhaps something to say for being able to write drivers in a
> readable/easy programming language however I am afraid that if such a
> beast would be made that it would remain an academical exercise only
> due to performance constraints.
What about user level device drivers? Think the Python VM could
communicate with the driver through the user space API. Is there a
Python module for that?
--
Rodrigo S. Wanderley <rwand...@rsw.digi.com.br>
-- Blog: http://rsw.digi.com.br
Diez B. Roggisch
No. That's why it is called "userspace". The kernel just hooks into a
running program.
Diez
Michel Claveau - MVP
> Python is interpreted
No. Python is compiled (--> .pyc)
But the term "to compile" is not always unambiguous...
And the notion of "compiler" is not attached to Python (the language), but is attached to the implementation.
@+
MCI
Martin P. Hellwig
<cut>
>> Look for example to libusb, which provides a userspace environment and
>> pyusb which uses that and provides an interface to Python.
>>
> iicr pyusb uses a c interface to libusb, not python...
>
According to them they use ctypes indeed. Sorry if I was misleading in
my explanation.
Martin P. Hellwig
Well the pyc, which I thought was the Python bytecode, is then
interpreted by the VM.
Christian Heimes
> Well the pyc, which I thought was the Python bytecode, is then
> interpreted by the VM.
Python is often referred as byte-code interpreted language. Most modern
languages are interpreted languages. The list [1] is rather long.
Technically speaking even native code is interpreted by the micro code
of most CPUs.
[1] http://en.wikipedia.org/wiki/Interpreted_language
[2] http://en.wikipedia.org/wiki/Microcode
Francesco Bochicchio
Once upon a time there where lisp machines, whith processors designed
to fastly execute lisp code ... I worked with one of them for 2
years.
I wonder: has anybody thought of making a python-machine, or at least
a processor able to directly execute high-level bytecode (LLVM-like?).
I think the main feature of such a machine should be hyper-fast hash
lookup. Then dynamic memory management hardware ...
Ciao
-----
FB
Chris Rebert
Yes, it was considered a decade ago:
http://mail.python.org/pipermail/python-list/1999-June/004451.html
Cheers,
Chris
--
http://blog.rebertia.com
Dave Angel
> <div class="moz-text-flowed" style="font-family: -moz-fixed">Michel
>> Hi!
>>
>>> Python is interpreted
>>
>> No. Python is compiled (--> .pyc)
>> But the term "to compile" is not always unambiguous...
>> And the notion of "compiler" is not attached to Python (the
>> language), but is attached to the implementation.
>>
>> @+
>>
>> MCI
>
> Well the pyc, which I thought was the Python bytecode, is then
> interpreted by the VM.
>
includes a one-way transformation from human-readable source text into
something that can be more efficiently interpreted by other code, or by
hardware. The compiler really doesn't care whether the machine it's
targeting is real or virtual.
The CPython implementation of Python compiles the source text into a
bytecode file, with extension .pyc. That certainly is a compilation
step. Followed (much) later by an interpreted one.
To pick a specific implementation of C++, Microsoft C++ compiles C++
source text into an "executable file," with extension .exe (I'm
ignoring little details, like the linker). That's a compilation step.
Then the exe file is (later) interpreted by the microcode on the Pentium
chip.
As far as I know, nobody has yet built a microcode implementation of a
Python VM (Virtual Machine). Nor have I seen one for the Java VM.
However, in the early 80's there was a microcode implementation of the
P-system VM. It was never a commercial success, but it existed. And
there have been at least three Forth machines, where the hardware itself
was designed to support the language's VM. No microcode at all.
DaveA
Ben Finney
> Hi!
>
> > Python is interpreted
>
> No.
Yes. The same Python code is both interpreted and compiled so as to run
it.
> Python is compiled (--> .pyc)
The Python bytecode (the contents of the compiled ‘foo.pyc’ file) is
then interpreted by the run-time Python interpreter, to actually run the
program.
--
\ “Value your freedom or you will lose it, teaches history. |
`\ “Don't bother us with politics,” respond those who don't want |
_o__) to learn.” —Richard Stallman, 2002 |
Ben Finney
Martin P. Hellwig
<cut definition/interpretation of compiling>
Ah yes, we thread on the territory of word definition and difference in
interpretation. Any argument is doomed to fail if not agreed or at least
taken in perspective of the terminology used by users.
I could be (well it is quite likely) wrong in my interpretation of the
terminology, but here goes it anyway:
Machine Code:
Whatever the machine executes, it could be that the CPU uses an
abstraction of microcode to do this but from the perspective of the
user, this is all done in the same 'black box'
Compiling:
Translate words/symbols/mnemonics to machine code, which than can be
either loaded, linked and executed by an OS or read and executed by the
BIOS.
Interpreted:
Instructions which can be fed to a previous compiled program that is
able to dynamically change its execution and flow without the need to
recompile itself.
Dave Angel
> <div class="moz-text-flowed" style="font-family: -moz-fixed">Dave
> <cut definition/interpretation of compiling>
> Ah yes, we thread on the territory of word definition and difference
> in interpretation. Any argument is doomed to fail if not agreed or at
> least taken in perspective of the terminology used by users.
>
> I could be (well it is quite likely) wrong in my interpretation of the
> terminology, but here goes it anyway:
>
> Machine Code:
> Whatever the machine executes, it could be that the CPU uses an
> abstraction of microcode to do this but from the perspective of the
> user, this is all done in the same 'black box'
>
> Compiling:
> Translate words/symbols/mnemonics to machine code, which than can be
> either loaded, linked and executed by an OS or read and executed by
> the BIOS.
>
> Interpreted:
> Instructions which can be fed to a previous compiled program that is
> able to dynamically change its execution and flow without the need to
> recompile itself.
>
and his biases, he will include more or less in his "black box." In the
old days, microcode was not "on-chip" but stored separately in control
memory. And on many machines, it was changeable at will.
To many users these days, the entire system including software is a
black box, which gradually breaks down (gets slow, runs out of space,
crashes a lot) and must be replaced. They don't distinguish operating
system from application, real memory from virtual, or viruses from
bugs. But of course those users wouldn't be participating in this
discussion.
My background includes specifying hardware instruction sets and
architecture. And writing microcode for multiple machines. And writing
parts of compilers, interpreters, assemblers, and so on. And
microcoding interpreters. And hooking into compilers to modify how they
would generate code. And hooking into runtimes to test running code in
realtime.
So I tend to have very flexible definition of compiler and interpreter.
Probably the only reason I jumped in here was the unmentioned bias that
somehow a compiler is superior to an interpreter.
I think I'd better extend my definition of compilation. It's a step
that's statically taken over a series of instructions (not necessarily
text source), that transforms it into a form closer to the targeted
enviromment, real or virtual. Most C++ compilers have two compilers
operating serially, the first to turn the source code into an
intermediate form (like byte code), and the second to generate what is
commonly called "machine code." The second part of course is duplicated
for each different target processor.
So Java is compiled into byte code, and the typical java VM then
compiles that piecewise into machine code (so called JIT compiling, for
just in time).
BTW, interpreters don't have to be written in a compiled language.
Anyway, I don't object to your definitions. We all have different
perspectives.
DaveA
Ben Finney
> Machine Code:
> Whatever the machine executes, it could be that the CPU uses an
> abstraction of microcode to do this but from the perspective of the
> user, this is all done in the same 'black box'
This requires, of course, defining what is the machine. Python bytecode
targets a virtual machine that is implemented differently for each
hardware platform.
> Compiling:
> Translate words/symbols/mnemonics to machine code, which than can be
> either loaded, linked and executed by an OS or read and executed by
> the BIOS.
Related to the above point, the “machine code” can just as easily be
codes for a virtual machine specification. This is the case for the
bytecode instructions Python gets compiled to.
> Interpreted:
> Instructions which can be fed to a previous compiled program that is
> able to dynamically change its execution and flow without the need to
> recompile itself.
This doesn't make much sense to me, I must say.
I'd say, instead, that a program is interpreted if its instruction are
dynamically translated to underlying platform instructions at execution
time. This is the case for the bytecode instructions interpreted by the
Python virtual machine.
--
\ “Often, the surest way to convey misinformation is to tell the |
`\ strict truth.” —Mark Twain, _Following the Equator_ |
_o__) |
Ben Finney
Grant Edwards
> Michel Claveau - MVP wrote:
>
>>> Python is interpreted
>>
>> No. Python is compiled (--> .pyc)
>> But the term "to compile" is not always unambiguous...
>> And the notion of "compiler" is not attached to Python (the
>> language), but is attached to the implementation.
>
It is, but that's just one particular implementation you're
talking about (though by far the most widely used one).
> is then interpreted by the VM.
Yup. Just like .exe files are interpreted by the microcode in
the processor that implements the IA32 VM. It would be quite
possible to put a Python VM into hardware. Alternatevly, you
can compiler Python into Java bytecode and run that "directly"
on hardware.
--
Grant Edwards grante Yow! It don't mean a
at THING if you ain't got
visi.com that SWING!!
Grant Edwards
> As far as I know, nobody has yet built a microcode implementation of a
> Python VM (Virtual Machine). Nor have I seen one for the Java VM.
Didn't Sun or somebody do one 10-12 years ago? Or did I
misinterpret some press release or something? Off to google...
> However, in the early 80's there was a microcode implementation of the
> P-system VM.
Ah yes. I remember sitting at an Intel MDS-80 "blue box" CP/M
system entering assembly language by hand for a simple Pascal
-> P-code compiler. IIRC, I typed it from a listing in the
"Byte Big Book of Pascal". That machine was pretty high-tech,
since I could save my work on an 8" floppy rather than a spool
of paper tape. The floppy disks didn't leave big oil stains in
your backpack!
I got the compiler working, but I don't remember ever having a
VM and run-time system.
> It was never a commercial success, but it existed. And there
> have been at least three Forth machines, where the hardware
> itself was designed to support the language's VM. No
> microcode at all.
--
Grant Edwards grante Yow! When you get your
at PH.D. will you get able to
visi.com work at BURGER KING?
Martin P. Hellwig
> "Martin P. Hellwig" <martin....@dcuktec.org> writes:
>
>> Machine Code:
>> Whatever the machine executes, it could be that the CPU uses an
>> abstraction of microcode to do this but from the perspective of the
>> user, this is all done in the same 'black box'
>
> This requires, of course, defining what is the machine. Python bytecode
> targets a virtual machine that is implemented differently for each
> hardware platform.
>
I would further define 'black box' as the hardware a kernel programmer
writes to.
<cut>
>> Interpreted:
>> Instructions which can be fed to a previous compiled program that is
>> able to dynamically change its execution and flow without the need to
>> recompile itself.
>
> This doesn't make much sense to me, I must say.
>
> I'd say, instead, that a program is interpreted if its instruction are
> dynamically translated to underlying platform instructions at execution
> time. This is the case for the bytecode instructions interpreted by the
> Python virtual machine.
>
Yes that is indeed a much better description, I'll steal that from you :-)
MRAB
[snip]
> As far as I know, nobody has yet built a microcode implementation of a
> Python VM (Virtual Machine). Nor have I seen one for the Java VM.
> was designed to support the language's VM. No microcode at all.
>
MRAB
execution time. What you're describing is more like JIT.
Michael Torrie
> please introduce me some resource for system programming, I know that
> python is very well at system programming level.
> thanks
Although it is possible (as others have said) to embed Python the
interpreter into a driver, no one has done that that I know of. You'd
have to write the driver in C or C++, though, and then provide embed a
python interpreter and then provide a python interface (wrapper code
written in C++) that would expose binary primitives and structures to
python that the OS and driver use. Sounds like quite an undertaking. I
embedded a python interpreter in a gina dll once (to write login code in
python). That's a form of system programming, but not what you're
asking about.
Someone years ago someone embedded the perl interpreter into a Linux
driver allowing some kinds of device drivers to be written in perl. It
was made more as a curiosity though.
Python is not well-suited to system programming. It takes extra work
(via the ctypes library or other wrappers) to interface with C structs,
plus calls all have to be marshalled to and from the native C APIs.
Might be cool, though.
System programming in Win32 is very complicated and messy. What little
system programming I've done in Linux was a dream compared. Definitely
you'll have to master win32 system programming in C and C++ before you
try to embed python in anything. I'm sure there are a number of books
on the subject. That's where I'd start.
Ethan Furman
> "Martin P. Hellwig" <martin....@dcuktec.org> writes:
>
>
>>Machine Code:
>>Whatever the machine executes, it could be that the CPU uses an
>>abstraction of microcode to do this but from the perspective of the
>>user, this is all done in the same 'black box'
>
>
> This requires, of course, defining what is the machine. Python bytecode
> targets a virtual machine that is implemented differently for each
> hardware platform.
>
>
>>Compiling:
>>Translate words/symbols/mnemonics to machine code, which than can be
>>either loaded, linked and executed by an OS or read and executed by
>>the BIOS.
>
>
> codes for a virtual machine specification. This is the case for the
> bytecode instructions Python gets compiled to.
>
>
>>Interpreted:
>>Instructions which can be fed to a previous compiled program that is
>>able to dynamically change its execution and flow without the need to
>>recompile itself.
>
>
> This doesn't make much sense to me, I must say.
>
> I'd say, instead, that a program is interpreted if its instruction are
> dynamically translated to underlying platform instructions at execution
> time. This is the case for the bytecode instructions interpreted by the
> Python virtual machine.
>
I would have said compiled is executed by hardware, interpreted is
executed by software -- but I like your definition better. :)
~Ethan~
Hendrik van Rooyen
> As far as I know, nobody has yet built a microcode implementation of a
> Python VM (Virtual Machine). Nor have I seen one for the Java VM.
Atmel has made an ARM that has support for Java Bytecode.
AT91SAM9X512 and friends (smaller versions)
- Hendrik
greg
> I wonder: has anybody thought of making a python-machine, or at least
> a processor able to directly execute high-level bytecode (LLVM-like?).
In some of my idle moments I've speculated on what such
a machine might be like. One of my ideas for potential
future projects is to flesh out the design and maybe even
build an FPGA prototype.
It would purely be for fun, though. It's unlikely that
such a processor would be able to compete speed-wise
with a general-purpose CPU running a Python interpreter
with critical libraries written in C.
That's what killed things like the Lisp machine. Their
developers couldn't keep up with the huge resources that
people like Intel and Motorola had to throw at CPU
development, so eventually a general-purpose CPU could
run Lisp faster than a Lisp machine.
--
Greg
sturlamolden
> actually I mean driver programming under Windows operating system, if
> you know, there's A kit name DDK available at microsoft's website for
> developing device drivers under C / C++ environment,
Actually, Microsoft has replaced DDK with a new kit called WDK for
Vista. The funny thing about WDK is that there are a lot of illegal C
and C++ in the headers, which even Microsoft's own compiler refuse to
accept. So good luck on getting anything to compile.
Steven D'Aprano
> That's what killed things like the Lisp machine. Their developers
> couldn't keep up with the huge resources that people like Intel and
> Motorola had to throw at CPU development, so eventually a
> general-purpose CPU could run Lisp faster than a Lisp machine.
When you say "eventually", I think you mean "decades ago". I recall a
collaboration between Apple and Texas Instruments to build a Macintosh
with a Lisp Machine in the late 1980s. From the one box, you could run
two computers simultaneously, with two operating systems, one running the
Motorola 68020 and the other a Lisp Machine processor.
I don't think it sold very well -- by memory, benchmarks showed that for
half (or less) of the price, you could run Lisp in software on a vanilla
Mac and the software would be faster than running it on the Lisp Machine.
TI also had at least one Nubus card for the Mac running a Lisp Machine:
http://en.wikipedia.org/wiki/TI_Explorer
On a similar note, there were Forth machines also available for the Apple
Macintosh. Unlike Lisp, I think they suffered from the general lack of
popularity of Forth rather than lack of speed.
--
Steven
John Nagle
> hello there, I've a question :
> I want to know does python have any capability for using Ring0 and
> kernel functions for driver and device development stuff .
> if there's such a feature it is very good, and if there something for
> this kind that you know please refer me to some reference and show me
> some snippet .
>
>
> - Malc0de
With the CPython interpretive system, it's not likely to work. But
using ShedSkin, which generates hard machine code, it might be possible.
The main problem is that Shed Skin uses a garbage-collected environment,
which few kernels have.
Under QNX, which is a a real message-passing operating system with
all drivers in user space, it should be possible to write a driver in
Python. There's Python for QNX. It would probably be too slow to
be useful, though.
I've actually written a handler for Baudot Teletype machines in
Python. See
https://sourceforge.net/projects/baudotrss/
Those machines are so slow (45.45 baud) that Python isn't the bottleneck.
John Nagle