Thoughts on singularity and other single address space oses
paolino
http://mikehearn.wordpress.com/2008/01/26/singularity-part-2/ ) about
the ms research os singularity. It sounds all good and well. High
performance, microkernel design, software isolated processes, etc...
However we know that programs under singularity are msil bytecode
binaries and no direct access to memory or i/o is permitted.
In the article I have mentioned, the user Josh Haberman says: "You
have to buy into the MSIL’s JIT, its garbage collector, its threading
model, its memory visibility model. Forget competing innovations in
JIT-land, forget popular software like libmad that optimizes its inner
loops with assembly, forget game-changing programming paradigms like
lock-free data structures that require using CPU-specific instructions
like compare-and-swap."
I think he has a point but I don't know how much important are the
things he is saying.
For example, without direct access to memory through pointers you
cannot use all the fancy "protection" softwares that try to stop
reverse engineering.
But, how much application software needs to access memory directly?
Could singularity really block a lot of software?
Rod Pemberton
news:2251eddc-aeff-4b8c...@m16g2000yqc.googlegroups.com...
>
> [MS singularity]
Sorry, I haven't read your link and I have no comments in regards to MS'
"singularity" at this time.
> But, how much application software needs to access memory directly?
From assembly (or machine) language's perspective? Everything... No
exceptions. The reason is that all HLL's (high level languages) are
implemented in assembly and all assembly languages use addresses to
reference data on the stack and in memory. This fact eludes many who view a
HLL as an abstraction entirely independent from assembly.
FYI, assembly (or machine) language is what HLL (high level languages) are
compiled to. In assembly, you can't access variables in memory or on a
stack without direct access to memory. You can't move data into registers
to perform computations without direct access to memory. And, you can't
perform any mathematics, binary operations, logic, etc. without data
somewhere: memory, register, or stack.
> But, how much application software needs to access memory directly?
Your question doesn't specify the quantity of memory, or a method of access
to memory. Both of those affect your question.
For an example regarding quantity of accessable memory, if the quantity of
memory the application has direct access to is limited, e.g., via various
cpu protection methods, then the application can access that limited memory
block directly. But, it can't access other blocks of memory, nor can it
access the entirety of memory.
For an example of direct memory access even without pointer access, you
could look at a language that has no memory pointers, e.g., Java. Java must
still be able to access variables in memory. To access a variable in memory
using assembly, you'll need it's address. An address is the low-level
assembly language equivalent of a high-level language's pointer. Without an
address, you can't locate the variable in memory, or access the variable's
contents in memory. So, even though Java language has no pointers or
addresses, they are still used to implement Java's functionality in
assembly. This pointer functionality is hidden from the programmer. This
pointer functionality is also restricted in Java to only "known" objects.
This means the "pointer" can't point outside a "known" object, or to
unallocated memory, or out-of-bounds memory, or unassigned memory locations,
or be modified by user arithmetic, or be assigned a value by the user, etc.
Java apparently calls this a "reference", e.g., array reference or object
reference. A reference is a just constant address.
HTH,
Rod Pemberton
paolino
> From assembly (or machine) language's perspective? Everything... No
no I mean from the software industry perspective
a lot of application software can be built without pointers, we have
legions of programmers that do their work with java, c#, php, all
languages where you are encouraged to not use pointers
so what I'm asking is "in what cases do you absolutely need to access
specific memory locations?"
for example I have written some code ( some years ago ) that accessed
directly the frame buffer of the video card to display characters and
colors on the screen....I remember the memory segment was B800
in that case it was vital to have direct access to a specific memory
location/region....but if you are writing a simple application
software you can declare variables, then assign values to them,
etc.... without caring where in memory they're really located....
a virtual machine can handle the second case without a fuss, but to
isolate processes it is necessary that direct access in memory through
pointers is blocked
Rod Pemberton
> so what I'm asking is "in what cases do you absolutely need to access
> specific memory locations?"
Oh... Basically, never - for a modern OS.
The only time you need to specifiy a specific memory location is to access
something that is memory mapped. Memory mapped devices and data are located
at specific physical memory addresses. For a modern OS, it's likely that an
application should never need to access a specific location. But, it
depends on the OS design. The modern OS provides API's and device drivers
which should hide all memory mapped devices and data from the application.
If so, the application only needs to access the known objects in the
application's memory space. Since everything in the application space is
known, it should be accessable by name in the source code. So, the ability
to assign specific values to pointers shouldn't be needed. However, it
could be useful in a few circumstances to have assignable pointers:
self-disassembly or code- or data-walking, etc. I.e., where the application
is accessing itself as raw data instead via it's known objects. Depending
on the language, it might be possible to do this with array syntax instead
of pointers. But, in general and for the most part, there should be no need
for assignable pointers since there are no memory mapped devices in the
application space. For a "simple" OS like DOS, there is much memory mapped
stuff an application could need access to: memory mapped screen, memory
mapped device registers, accessing BIOS data area (BDA) in memory, interrupt
vector table (IVT) in memory, etc.
Rod Pemberton
BGB / cr88192
<--
-->
having to write all the SW in a .NET language would block a lot of
software...
almost just as bad as if it all had to be written in Java...
the likely result is that an OS like this could not likely have a general
purpose appeal, and would likely be restricted to being little more than a
research OS.
granted, a hybrid OS could be a usable strategy.
BGB / cr88192
On Jan 1, 9:58 am, "Rod Pemberton" <do_not_h...@nohavenot.cmm> wrote:
> From assembly (or machine) language's perspective? Everything... No
<--
no I mean from the software industry perspective
a lot of application software can be built without pointers, we have
legions of programmers that do their work with java, c#, php, all
languages where you are encouraged to not use pointers
so what I'm asking is "in what cases do you absolutely need to access
specific memory locations?"
for example I have written some code ( some years ago ) that accessed
directly the frame buffer of the video card to display characters and
colors on the screen....I remember the memory segment was B800
in that case it was vital to have direct access to a specific memory
location/region....but if you are writing a simple application
software you can declare variables, then assign values to them,
etc.... without caring where in memory they're really located....
a virtual machine can handle the second case without a fuss, but to
isolate processes it is necessary that direct access in memory through
pointers is blocked
-->
access to HW is not really the issue, the OS does this, but in general
software does not (and generally can't on modern OS's, given each process
already has its own isolated address space).
however, regarding the process as a big flat memory space executing a known
ISA does have many uses.
for example, one can implement features that the particular language
implementor may not have originally considered, such as adding features like
"eval" or "apply" to C. if direct access of the app to itself were not
possible, it would be similarly not possible to add features like eval or
apply.
basically, in some ways, these VM languages, like Java or C#, are little
toys vs C or C++.
they essentially elevate the VM designer to a level far above that of the
programmer using the VM, where the VM can implement things (because it is
usually themselves written in C or C++) that are essentially not possible to
implement from within the VM. one has to instead rely on the good graces of
the VM designer to implement every conceivable core VM feature (or that the
VM has not left out something where the VM designer was afraid someone could
cut themselves on...).
this is why, for example, in my case I am working on a VM architecture where
the client code essentially has the same level of power as the implementor,
because, essentially, the VM can access itself (the facilities created in
implementing the VM are, by natural extension, also available to client
code).
granted, this is not mandatory, and I do have an x86 interpreter (allowing,
among other things, the creation of sandboxed virtual processes).
but, in general, I am designing my VM stuff for my own uses (IOW: to
increase the power and options I have available), not for some "other"
people felt to be too stupid for their own good (AKA: the JVM strategy...).
granted, C# is not as bad as Java, but as a general rule, C# still does not
give the needed level of power, and it is still the case that for
fundamentally new facilities, explicit VM-implementor involvement may be
needed...
this is not the case with native code (such as x86), since in this case, the
low-level representation is very basic and highly open-ended.
similarly, restrictions can be imposed on native x86 code to allow it to be
statically provable as well (and, likewise, it can be sandboxed, ...).
...
Rod Pemberton
>
> Hi, I was reading an interesting article ( this one
> http://mikehearn.wordpress.com/2008/01/26/singularity-part-2/ ) about
> the ms research os singularity. It sounds all good and well. High
> performance, microkernel design, software isolated processes, etc...
>
Ok, I got around to reading it.
"Programs in Singularity _are_ isolated from each other, but the isolation
is done entirely in software, using type theory instead of silicon. They can
do this, because programs in Singularity are all implemented in a C#
derivative called Sing#"
The problem is that they can't prevent assembly code from executing. There
is no method to force the user to use Sing#.
Elsewhere, I said:
RP> The reason is that all HLL's (high level languages) are
RP> implemented in assembly and all assembly languages use addresses to
RP> reference data on the stack and in memory. This fact eludes many who
RP> view a HLL as an abstraction entirely independent from assembly.
From the article, I think they haven't realized that fact and this fact: it
is almost impossible to prevent execution of assembly (or machine) code.
They didn't design the assembly language they are using. They can't control
what the assembly language is capable of.
You can read more from me on why assembly code _almost_ can't be blocked
from execution from parts of my other related posts:
http://groups.google.com/group/alt.os.development/msg/5ff9ba2c15a9cfad?hl=en
http://groups.google.com/group/alt.os.development/msg/aedee21fd277a009?hl=en
The closest they can come to prevent uncontrolled execution, IMO, is to
implement something similar to what I describe in the first link. Since
they aren't implementing *any* privileges according to the article, the
methods of bypassing the secured toolchains thereby allowing unwanted
assembly to execute are even more true, IMO.
At present, I don't understand how they can "pull this off" without hardware
support implementing security. It seems to me all they've done is eliminate
toolchains that will emit instructions that could cause problems. That
doesn't prevent such code from being created and executed via other methods.
IMO, this is similar to the problems I see with Google's nativeclient:
http://groups.google.com/group/comp.lang.asm.x86/msg/e05b166d78bf2561?hl=en
Google nativeclient
http://code.google.com/p/nativeclient/
Rod Pemberton
Maxim S. Shatskih
Dead-born research project.
--
Maxim S. Shatskih
Windows DDK MVP
ma...@storagecraft.com
http://www.storagecraft.com
paolino
So the thing seems possible after all, but device drivers pose a
problem because they need access to specific memory regions where
devices are mapped.
The good thing is that application software makes use of named objects
to access memory locations. But some of you have a point, because
there's a lot of software in C/C++ etc... that makes use of pointers.
Also the problem that I was thinking about is the behaviour of
singularity. Practically they compile the bytecode when you install
the program, but who can stop a malicious software from escaping the
sandbox and modify the native code executable that singularity create
on the disk for each installed program?
This is a major problem. I know that another OS ( Taos/Elate ) used
the same strategy, but they had binaries in bytecode all time and only
at load time they were compiled but a jit compiler.
However Elate was not developed with software isolation in mind, the
bytecode served the purpose of heterogeneous clusters where you need
to migrate a particular software component from a pentium to a powerpc
for example and obviously you need a bytecode binary that is compiled
on the specific machine where it is executing at the moment.
Also there's a need for self modifying code after all and this need
cannot be addressed by singularity. Afterall it seems singularity is a
dead end from a practical point of view.
Alexei A. Frounze
<ma...@storagecraft.com.no.spam> wrote:
> >the ms research os singularity.
>
> Dead-born research project.
Unless it's made to somehow support existing Windows apps (using
virtualization, probably), it won't replace Windows in the masses.
And then if it does, I guess, the driver model will change once again
rendering many devices incompatible and useless.
Alex
paolino
> Unless it's made to somehow support existing Windows apps (using
> virtualization, probably), it won't replace Windows in the masses.
> And then if it does, I guess, the driver model will change once again
> rendering many devices incompatible and useless.
>
> Alex
is it feasible? From what I know, virtualization makes use of the same
protection mechanisms used to separate ring 0 from ring 3 ( in fact
virtualizers run in ring -1 ).
Instead singularity simply doesn't make use of ring protection, so
they can build a very fast microkernel. Using MMU and rings, etc...
would make the os very slow compared to monolithic ones.
Alexei A. Frounze
> On Jan 2, 9:10 pm, "Alexei A. Frounze" <alexfrun...@gmail.com> wrote:
>
> > Unless it's made to somehow support existing Windows apps (using
> > virtualization, probably), it won't replace Windows in the masses.
> > And then if it does, I guess, the driver model will change once again
> > rendering many devices incompatible and useless.
>
> > Alex
>
> is it feasible?
Why not?
> From what I know, virtualization makes use of the same
> protection mechanisms used to separate ring 0 from ring 3 ( in fact
> virtualizers run in ring -1 ).
I don't see a problem in effectively running 2 different OSes side by
side in 2 VMs. Within a VM you are not required to use multiple rings
and multiple address spaces.
> Instead singularity simply doesn't make use of ring protection, so
> they can build a very fast microkernel. Using MMU and rings, etc...
> would make the os very slow compared to monolithic ones.
Only the compatibility "layer/VM" would need to use these things.
Alex
BGB / cr88192
"Alexei A. Frounze" <alexf...@gmail.com> wrote in message
news:59d922a4-0a25-4c07...@d20g2000yqh.googlegroups.com...
> On Jan 3, 3:57 am, paolino <skyp...@gmail.com> wrote:
> > On Jan 2, 9:10 pm, "Alexei A. Frounze" <alexfrun...@gmail.com> wrote:
> >
> > > Unless it's made to somehow support existing Windows apps (using
> > > virtualization, probably), it won't replace Windows in the masses.
> > > And then if it does, I guess, the driver model will change once again
> > > rendering many devices incompatible and useless.
> >
> > > Alex
> >
> > is it feasible?
>
> Why not?
>
>> From what I know, virtualization makes use of the same
>> protection mechanisms used to separate ring 0 from ring 3 ( in fact
>> virtualizers run in ring -1 ).
>
> I don't see a problem in effectively running 2 different OSes side by
> side in 2 VMs. Within a VM you are not required to use multiple rings
> and multiple address spaces.
>
yep.
>> Instead singularity simply doesn't make use of ring protection, so
>> they can build a very fast microkernel. Using MMU and rings, etc...
>> would make the os very slow compared to monolithic ones.
>
> Only the compatibility "layer/VM" would need to use these things.
>
if even this...
what is to stop, say, an OS from running x86 apps on a non-x86 CPU by making
use of the same sorts of trickery often used to JIT bytecode:
build a virtual model of the process image, and dynamically translate code
fragments into native code.
my experience with both compiler writing and writing an x86 interpreter
would not make it seem like the problem would be "that" hard (the only real
issue being mostly that self-modifying code could mean needing to flush some
of the "traces" and then re-translate them if needed).
I don't mean static recompilation (that would SUCK to implement), rather,
one would operate partway between an interpreter and a JIT, establishing
"traces" which would start wherever we happen to land following a jump.
the sequence of instructions would then be disassembled up to the next
branch (or other "edge criteria"), then this sequence could be rewritten
into a flow-graph and mapped to actual machine registers, and at the end the
state is "flushed" (commiting the machine registers back to the faked
CPU/process image), this is then rewritten into a sequence of native
instructions, made visible in a "jump LUT", and then executed.
when done, the sequence would trigger a re-jump (through the LUT), and
possibly trigger the process again for the next fragment (if there is no
translated code present).
or, even failing this, the code can be interpreted (my interpreter works
already by caching decoded instructions, which actually makes the thing
perform "acceptably" well, or at least as far as interpreters go...).
granted, I am not assuming 1:1 performance here, but maybe 50-80% would be
not all "that" difficult?...
I am getting performance from an interpreter (well, an ~ 60x slowdown, but >
100x is common for interpreters), which would otherwise be acceptable if one
were not that demanding of raw performance (I could probably run Doom or
Quake in this, if I faked a whole system, otherwise I would have to port it
and I am not going to bother...).
on average, it may actually run better than this, since much of the actual
work (API's, ...) simply redirect to the "kernel" (AKA: native code), and so
the slowdown could be reduced so long as nothing "intensive" is done in the
interpreter.
for example, in this interpreter, things like "malloc" and "free" are simply
system calls, with the actual malloc/free machinery working in native code
with a fake process, ...
and, hell, other people probably wont be doing their instruction decoding
and some of their interpreter machinery via "strings-driven logic" either...
(IOW: I was lazy, and built some of the logic code out of
finite-state-machines driven by character strings...).
but, anyways, given how well DosBox and QEMU perform, it should be possible,
since by default neither is using fancy CPU features AFAIK (or, at least on
Windows, QEMU can use CPU virtualization on Linux...).
also, Apple pulled it off with PPC->x86 and no one started a riot over
horrible performance...
granted, there are many issues more than this which would matter to really
have much reason to use an OS.
if, for example, the OS sucked and the virtualization were no better than
DosBox or QEMU (the whole "OS in a window" experience...), well then, that
wouldn't work out well.
or such...
Maxim S. Shatskih
> use of the same sorts of trickery often used to JIT bytecode:
> build a virtual model of the process image, and dynamically translate code
> fragments into native code.
Yes, but IL bytecode is by far more suitable as an input for such things then the real x86 code.
Think on IL bytecode as about the compiler's intermediate P-code. More so - MS's compilers for x64 really uses this thing (called Whole Program Optimization, the C compiler generates IL and then the linker translates this IL to x64).
Rod Pemberton
news:hhs6l9$fn3$1...@news.albasani.net...
>
> what is to stop, say, an OS from running x86 apps on a non-x86 CPU by
making
> use of the same sorts of trickery often used to JIT bytecode:
> build a virtual model of the process image, and dynamically translate code
> fragments into native code.
>
I've probably mentioned this previously... DEC Alpha's did that using
FX!32. They ran x86 code on non-x86 CPU: DEC Alpha. App.'s ran a bit slow
the first few times. But, as code was translated, it ran at near native
speed. There are a few good papers about FX!32 on CiteSeer. It'd be great
if someone did x86 to x86 this way, say 16-bit to 32-bit. It could rewrite
your BIOS as 32-bit, or rewrite DOS - any DOS - as 32-bit.
http://en.wikipedia.org/wiki/FX!32
RP
paolino
protection to a virtual machine, why we are creating a single address
space with no memory protection in the first place?
After all singularity has the only advantage of having very good
performance because it bypasses all the nasty things of modern cpus
about paging, virtual addressing, memory protection, etc.....
Instead of running singularity inside a virtual machine, it is best to
implement an os on top of l4 microkernel.
BGB / cr88192
"Maxim S. Shatskih" <ma...@storagecraft.com.no.spam> wrote in message
news:hhscg2$2cbp$1...@news.mtu.ru...
> what is to stop, say, an OS from running x86 apps on a non-x86 CPU by
> making
> use of the same sorts of trickery often used to JIT bytecode:
> build a virtual model of the process image, and dynamically translate code
> fragments into native code.
<--
Yes, but IL bytecode is by far more suitable as an input for such things
then the real x86 code.
Think on IL bytecode as about the compiler's intermediate P-code. More so -
MS's compilers for x64 really uses this thing (called Whole Program
Optimization, the C compiler generates IL and then the linker translates
this IL to x64).
-->
granted...
I am aware of the differences.
I am also aware of how with simulating x86, one sits around having to fake
eflags behavior after pretty much every operation, ...
one can so better translation with IL, and one can translate in advance with
IL.
this does not mean x86 can't be translated, even if it is a bit nastier to
do so, and one has to worry about such issues as the virtual process image,
relations between real and simulated code addresses, the possiblity of
self-modifying code (and the consequent nastiness), ...
but, this much is not so much the issue, as is how well it can be made to
work...
BGB / cr88192
no, I think you miss the point...
one would likely use an x86 VM for legacy app support and some basic level
of HW independence...
consider for example, if one had an OS which was binary compatible with,
say, Win32, but ran on lots of different HW (PPC? ARM? ...).
this would have a lot more appeal than, say, an OS which couldn't run much
of anything... (except maybe .NET stuff...).
Alexei A. Frounze
> "BGB / cr88192" <cr88...@hotmail.com> wrote in messagenews:hhs6l9$fn3$1...@news.albasani.net...
Actually, 16-bit to 32-bit x86 conversion isn't that hard. In most
cases you could just stick a prefix or two in front of the original
instruction and execute it. It gets slightly more complicated with
real-mode segmentation though.
As for rewriting BIOS and DOS as 32-bit, I guess, that'll only work
for VMs, 'cause the real-time emulator/translator will need to work on
top of some OS/kernel anyway. And in the case of BIOS, one would need
to have a special (plain) version of the BIOS since the one from the
physical machine would be hard to stick into a VM due to hardware
dependencies.
Alex
BGB / cr88192
On Jan 4, 2:06 am, "Rod Pemberton" <do_not_h...@nohavenot.cmm> wrote:
> "BGB / cr88192" <cr88...@hotmail.com> wrote in
> messagenews:hhs6l9$fn3$1...@news.albasani.net...
>
>
>
> > what is to stop, say, an OS from running x86 apps on a non-x86 CPU by
> making
> > use of the same sorts of trickery often used to JIT bytecode:
> > build a virtual model of the process image, and dynamically translate
> > code
> > fragments into native code.
>
> I've probably mentioned this previously... DEC Alpha's did that using
> FX!32. They ran x86 code on non-x86 CPU: DEC Alpha. App.'s ran a bit slow
> the first few times. But, as code was translated, it ran at near native
> speed. There are a few good papers about FX!32 on CiteSeer. It'd be great
> if someone did x86 to x86 this way, say 16-bit to 32-bit. It could rewrite
> your BIOS as 32-bit, or rewrite DOS - any DOS - as 32-bit.
>
> http://en.wikipedia.org/wiki/FX!32
>
> RP
<--
Actually, 16-bit to 32-bit x86 conversion isn't that hard. In most
cases you could just stick a prefix or two in front of the original
instruction and execute it. It gets slightly more complicated with
real-mode segmentation though.
-->
PM segmentation is not easy either, since one would likely be going from a
segmented address space (as in PM16) to a flat address space (PM32).
this would end up adding the need for a (possibly variable) segment base,
which on 32-bit x86 would likely entail the use of additional registers.
now, the mapping is not quite so easy, as there is no longer a 1:1 mapping
between registers, ...
if done per-instruction, this could also get fairly expensive (since a
specific instruction contains a lot of details which may need to be faked,
and one is confined in that both the state before and after this particular
instruction need to be consistent).
this is why I had mentioned doing translation in terms of "traces":
in this case, we treat the simulated registers as if they were simply
variables, which may be "cached" in actual machine registers;
and, at the end, the final state of any dirty registers or variables is
written back.
at this point, one can actually allow the model to break down some:
between the start and end of a trace, it only matters that the "externally
visible" state be preserved, but what actually happens here is up to the
translator;
so, the x86 instructions can be regarded, not as decrees of specific events,
but as collections of state-transitions, and the codegen can then go about
mapping these transitions to the native instructions.
at the end of the trace, one jumps to the next trace, ...
I guess, one can note, what is my definition of a trace:
there are no jumps into a trace, the only valid entry point is the start;
there are no jumps out of a trace, the only exit point is at the end;
there are no events within a trace which either violate deterministic
behavior or the above 2 rules;
...
any instruction which violates these rules then constitutes the end of a
trace, although certain instructions could be treated as "special". these
would not violate the above rules, but would instead become part of the
terminal-state of the trace.
an example of this:
some code is done, a 'cmp' is done, and then a conditional jump is used.
under a strict interpretation, the trace would end (writing back any changed
registers, eflags, ...);
then the conditional jump would be, essentially, interpreted, and the
control re-enters a trace at one of the targets.
the special condition case could be that of actually inlining the jump into
the trace, so at the end, it may transfer control to possibly 2 other
traces, and instead of simulating the eflags resulting from the 'cmp', could
possibly instead set eflags "as it would have needed to have been" to get to
each jump target (or, possibly, look a little further and infer that it need
not bother with simulating eflags...).
<--
As for rewriting BIOS and DOS as 32-bit, I guess, that'll only work
for VMs, 'cause the real-time emulator/translator will need to work on
top of some OS/kernel anyway. And in the case of BIOS, one would need
to have a special (plain) version of the BIOS since the one from the
physical machine would be hard to stick into a VM due to hardware
dependencies.
-->
yep, this is common practice...
granted, of course, for faked x86, one need not simulate the whole CPU, or
as I have found, even simulate it particularly faithfully.
an example of this is only simulating "ring-3", with the "kernel" being raw
native code, implementing processes by having a fake CPU for each process,
or even gross violations of the underlying memory model (say, using memory
spans rather than pages, ...).
for example, does an interpreted user app need to know whether or not the
LDT, GDT, TSS, Page Tables, ... actually exist?...
note: I used spans rather than pages because spans are fast and simple
(pages seem simple enough in concept, but my partial versions of a
page-based MMU give doubts as to how efficient they would be in an
interpreted context). also the spans are easier to work with (since in
native land they are raw flat memory buffers).
granted, an emulator which can run DOS or the BIOS, or a pre-existing 32-bit
OS, needs to be a good deal more faithful.
ammusingly though, spans would likely still work well, just with the page
tables being an address-translation mechanism built on top (albeit with the
ugly edge case of a memory operation crossing a page boundary, which would
likely need to be detected and handled, ...).
in this case though, the spans would be for physical addresses, rather than
VAs.
or such...
paolino
> no, I think you miss the point...
>
> one would likely use an x86 VM for legacy app support and some basic level
> of HW independence...
> consider for example, if one had an OS which was binary compatible with,
> say, Win32, but ran on lots of different HW (PPC? ARM? ...).
>
an x86 VM to run x86 code on other platforms? I think singularity main
goal is to build a superfast os that bypasses all the slow and complex
things modern processors have to speedup the software.
If you want to use a vm you must run it on top of an os capable of
supporting rings and memory protection right?
So you can run windows or linux as host, then a vm on top of them,
inside the vm singularity? What is the point in doing this? Yeah,
singularity could run on top of a lot of hardware and could run side
to side with legacy oses and applications.
This can work as a first step to complete migration to singularity.
But how many developers will do the transition? .NET is not the
onnipresent God ms had in mind and a lot of programmers continue to
build their software with c/c++
Singularity could suffer the same fate and become a niche OS. A
complete migration to singularity requires a massive redevelopment of
a ton of software. I know that using pointers today is pointless, but
the problem is real software out there that still makes use of
pointers and non safe languages as c.
BGB / cr88192
On Jan 4, 6:13 pm, "BGB / cr88192" <cr88...@hotmail.com> wrote:
> no, I think you miss the point...
>
> one would likely use an x86 VM for legacy app support and some basic level
> of HW independence...
> consider for example, if one had an OS which was binary compatible with,
> say, Win32, but ran on lots of different HW (PPC? ARM? ...).
>
<--
an x86 VM to run x86 code on other platforms? I think singularity main
goal is to build a superfast os that bypasses all the slow and complex
things modern processors have to speedup the software.
If you want to use a vm you must run it on top of an os capable of
supporting rings and memory protection right?
-->
no, a VM does not need rings and memory protection...
a VM is "just software" FWIW.
having an x86 based CPU, or access to low-level features, can help in a
faster VM, but are not actually necessary for this.
even plain interpreters are a workable solution (even if not a particularly
fast one).
<--
So you can run windows or linux as host, then a vm on top of them,
inside the vm singularity? What is the point in doing this? Yeah,
singularity could run on top of a lot of hardware and could run side
to side with legacy oses and applications.
-->
you don't need all of Windows or Linux FWIW. think more instead of the
possibility of a "compatibility shim", like for example what exists with
Wine.
a small part of the OS is simulated, namely the part directly visible to
apps, but then outside this narrow scope, all is redirected to the real OS.
<--
This can work as a first step to complete migration to singularity.
But how many developers will do the transition? .NET is not the
onnipresent God ms had in mind and a lot of programmers continue to
build their software with c/c++
-->
for a VM of this sort, C or C++ does not matter.
C and C++ can work on .NET, it is just not "native" to .NET (unlike C#), and
there is a little extra ugliness in making this all work.
<--
Singularity could suffer the same fate and become a niche OS. A
complete migration to singularity requires a massive redevelopment of
a ton of software. I know that using pointers today is pointless, but
the problem is real software out there that still makes use of
pointers and non safe languages as c.
-->
yep.
some level of virtualization would likely be needed in such an OS for it to
have any real hope of catching on.
a C compiler for .NET would further ease integration, and if done well could
allow people to continue to develop cross-platform (say, code which can run
on .NET, or native on another OS).
as is, this is a problem, since C++.NET is not strictly the same as either C
or C++.
but, assuming something like C.NET, the issue of course is that it would
have to provide an experience "fairly close" to that of native C, such that
people can still write code which functions in both cases, and without
needing to throw any extra funky syntax into the mix (at least for most
code), unlike as with C++.NET.
making C code "safe" and retaining standards conformance is the difficulty.
one possibility is that it allows and valid C code to be written, but could
warn when it can't be proven safe:
"warning: foo.c: 231: pointer arithmetic used in an 'unsafe' manner".
then the OS may be allowed to reject the program, or require special
verification (errm... UAC...).
basically, it is not pointers in themselves which violate safety, rather, it
is using them in ways which can't be statically proven. it is possible to
use pointers in ways which are provable, and via relatively minor
adaptations in coding style, only that strict enforcement (making it an
error) would violate the C standard (since standard conformance requires
that all conformant code work, but not all conformant code can be statically
proven safe).
hence is the issue...
it would be much like JIT with x86 machine code:
one could pull off a much better JIT if they could prove that x86 conformed
to certain rules;
this issue is in itself an issue.
I once attempted in-part to define such a restricted "provable" subset of
x86, but have not implemented.
the main issue, of course, is to implement a compiler which could produce
it, since one can realize fairly quickly that generic compiler output
doesn't exactly conform to the sorts of rules and restrictions needed for
formal "proof".
but, alas, not all is free:
these same sort of provability restrictions also impair many of the
"interesting" uses of C and x86.
imagine a C where, about as soon as one did anything "creative", it would
turn into a compiler warning, and if one pushes it harder, a verification
failure...
however, even with this, not all provable code is necessarily "safe" code,
since there may still be weaknesses and flaws deeper within the VM which
could still allow exploitation (for example, if a public API exposes access
to a few calls which, really, should be off-limits to untrusted code, but
for whatever reason were left unprotected).
this is itself a problem I have tried to address, but this is a slightly
more difficult problem (AKA: how to do these kinds of security validation
without impeding performace or making things more difficult to use).
the idea would be how to try to design a security system which not only
grants or blocks access to things, but as well could detect when security
has been breached (which I suspect could only really be detected deeper into
the system via "inconsistencies"). this would imply, not some simple
"layered" security model, but infact something a bit more complicated (and
which could fault on "inconsistencies likely involving several different
parts of the system"), which is a cause for worry (both because it would be
problematic, and likely also slow...).
"layers" are simple and fast, but in some ways, not terribly strong.
partial idea, by analogy:
we have probably all seen keys, and can note that certain keys fit into
certain doors but not others;
a key which does not fit a door, even if one "thinks" it is the right key,
it will not fit.
so, there can be a "key" representing a set of known security rights, and as
a key passes through each "lock", it may in turn be modified by the
properties of the lock (to produce a new key which may fit through other
locks);
the "locks" are in turn created via a sort of "proof" process, which could
analyze definitions of the basic requirements of each lock (required rights,
disallowed rights, ...), and in turn calculate a set of possible keys which
could fit, and at the same time the modifications which would be applied to
said keys, ...
...
the validation process would likely be a set of bit-masking operations,
probably AND and XOR, and checking that the value is 0. the key modification
would be AND, OR, and XOR.
so, the basic idea would be that, even if one finds a "hole" somewhere (an
unsecured API or a badly declared lock, for example), trying to use it will
produce an ill-formed key, which would likely blow up at a subsequent lock.
simple example:
control ends up in kernel space, but the current key happens to have, say,
'user_guest' set (because it didn't go through a lock which explicitly
inverted this "privledge"), it hits a lock, which is set not to either mask
or invert this "privledge" (say the lock had nothing set for this right, and
was set to be 'strict'), the result: the post-op key is not 0, and so the
lock-check fails.
also possibly useful:
it the logic for building the locks wanders around to figure out the logic
for the locks (AKA: the "key shape"), possibly it would also be able to
detect possible logic errors and ill-formed locks as well.
nevermind all the more glaring problems with this idea...
or such...