I designed it, and it works well enough for C, and with some hammering I was
able to make it work on both x86 and x86-64...
but, I have been facing difficulty trying to force other languages into
working with the same lower compiler, since as noted, it is just not very
flexible.
however, recently I had an idea:
even though I don't have a flexible lower-end, I do have something that is
reasonably flexible:
a C compiler (which in turn, produces output I can run through my existing
lower end).
so, my idea was then to take an older version of my frontend compiler (from
the time when it was simpler and faster... errm... mostly because it was not
based on DOM...), and basically force it in between the upper and lower
compiler stages.
this way, I rework my upper-end to essentially produce C, which is fed into
this newer (or, technically, older...) compiler, which in turns produces the
IL for my lower-end.
however, to better fill the role (ok, mostly to reduce performance
overhead), I considered making a few design changes/restrictions:
I have removed the preprocessor, as for now I will assume that any used
input will be "PP clean"...
I have considered deprecating typedef (the reason is that typedef adds a
good deal of complexity to the parser, as well as likely making it a good
deal slower);
I have also dropped/deprecated some other misc language features (enum, a
lot of the C99 additions, ...);
I may deprecate support for "inline" struct/union declarations (all structs
are declared toplevel, and may be referenced, but no struct bodies may
appear as part of a variable of function declaration);
all variable declarations will be at the top of the function;
functions may not be nested;
...
as well, there will be some new language extensions:
qualified names will be added (I am using an essentially C++ style syntax,
namely: "Foo::Bar::baz");
intrinsics will likely be added for things like exception handling and
(managed) object manipulation (unmanaged classes will be likely treated like
structs at this level);
...
note: qnames will not add features like namespaces/... at this level rather,
names will need to be fully qualified, and thus resolved, before they get
here (this compiler will assume a single big-flat namespace, but containing
qnames...).
similarly, this stage will not handle templates/generics, which will also be
assumed to have been expanded before reaching this stage, and (probably)
similar goes for inlines, ...
...
but, otherwise, things will likely be mostly like C...
(I have actually renamed it to 'BMC' for my uses, or "BGB Middle C").
note that there is an another C frontend, and so it will be in effect a C->C
compiler, but will have the option of essentially "lowering" other
difficult-to-compile features/extensions, such as inner functions and/or
closures, exception handling, OO extensions, ...
as well as simplifying the task of getting other, non-C frontends, to be
able to compile...
(note: FWIW preprocessing could be done manually/externally, and typedefs
faked via macros, if needed...).
so, the questions here are:
do people think using C as the basis for an IL is a good idea?...
are some of my considered alterations (such as deprecating typedef)
reasonable?...
...
or, hypothetically, would anyone else target a C subset with these
alterations?...
--
BGB: Hobbyist Programmer (Specialty: 3D, Compilers, VMs)
Site: http://cr88192.dyndns.org/
> so, the questions here are:
> do people think using C as the basis for an IL is a good idea?...
Speaking as someone who never designed a compiler, nor has any intention
of doing so:
It has been a popular choice for a few decades, but my impression is
that it is going out of vogue, where people either
- target the bytecodes of a popular VM such as Java, .NET, or Flash (I
leave Python out because, AFAIK, its bytecodes are not a popular
target for compilers for other languages)
- design and implement their own bytecode, with the intention of making
it popular in the long run. Few implementers can choose this route with
any chance of success, but there are quite a few contenders that IMHO
have a reasonable chance: Parrot (for Perl6), Jarvik (on Android), and
at least two contenders for Ruby
- write a custom parser and code generator for their language
- target a real intermediate language such as LLVM
An argument for no longer using C as intermediate language is that C's
data structures do not fit modern dynamic languages well.
An argument for not using an intermediate language at all is that
describing compilation as a series of source transformations may make
theoretical sense, but in practice slows down the resulting compiler too
much.
> are some of my considered alterations (such as deprecating typedef)
> reasonable?...
I think it could make it unnecessary hard to read the intermediate code
(and writers targeting your language would spend quite a time doing that
during the development of their compiler frontend)
> or, hypothetically, would anyone else target a C subset with these
> alterations?...
I think the better question would be
"Would anyone else target _my_ C subset?"
To be blunt: in what you write, I do not see many arguments for
answering that with 'yes'.
Liberally licensed compilers implementing a superset of what you propose
are available. How would targeting your C subset instead of any of these
benefit any compiler writer?
- better code generation?
- more supported platforms?
- faster compilation?
- compilation in less memory?
- fewer bugs?
- better guarantees of future availability of the compiler?
- ...?
The only things that might swing this to "yes" for some languages could
be your remark
> intrinsics will likely be added for things like exception handling and
> (managed) object manipulatio
Reinder
I don't like this route, as my design philosophy and target domain differs
notably from most VMs...
I do have partial (import) support for JBC though, and this may improve
later (mostly because compiling JBC into a C-like form would get around some
of my prior road-blocks...).
I had considered MSIL/CIL for input as well, but JBC is essentially the
"testing ground" for what I would need to get CIL working, and so if JBC
can't be made to work, not much point in trying with CIL...
> - design and implement their own bytecode, with the intention of making
> it popular in the long run. Few implementers can choose this route with
> any chance of success, but there are quite a few contenders that IMHO
> have a reasonable chance: Parrot (for Perl6), Jarvik (on Android), and
> at least two contenders for Ruby
similar to the above...
I personally have come to dislike the "one bytecode to rule them all"
mindset of most VMs...
> - write a custom parser and code generator for their language
this is what I did originally...
my codegen is still used, FWIW...
> - target a real intermediate language such as LLVM
>
I have considered this, but in the past LLVM did not do exactly what I
wanted.
it seems like more recently people have been addressing these issues, so it
may well become the case that LLVM fulfills my requirements...
(I may eventually use it as my backend codegen, but am not doing this at the
present moment...).
> An argument for no longer using C as intermediate language is that C's
> data structures do not fit modern dynamic languages well.
>
this is why I have compiler extensions...
well, that and most of the input languages I am compiling (C++, C#, and
Java), are C family, and thus map adequately to the C model.
ECMAScript should compile to a C subset a whole lot better than it compiles
to JavaByteCode (I have tried... it is not a good fit...).
the main point of C though is that it is the basic set of what is needed to
build "efficient" code (dynamic-language feautres can be addressed
differently, for example, via a different IL, or a different codegen if
needed...).
I don't preclude using different codegens, no more than I preclude using
multiple ILs...
> An argument for not using an intermediate language at all is that
> describing compilation as a series of source transformations may make
> theoretical sense, but in practice slows down the resulting compiler too
> much.
>
yes, I have been running into this problem...
some effort has been put into trying to optimize things some, but it is
difficult...
part of the issue though is that, for the most part, C is my frameworks main
input language, and so a lot of time goes into plowing through masses of
data originating in headers...
>> are some of my considered alterations (such as deprecating typedef)
>> reasonable?...
>
> I think it could make it unnecessary hard to read the intermediate code
> (and writers targeting your language would spend quite a time doing that
> during the development of their compiler frontend)
>
potentially...
but typedef does make the parser slower (as it is very often needed to check
if the next token is one of the known/registered types...). this is also why
I decided on leaving out the preprocessor...
>> or, hypothetically, would anyone else target a C subset with these
>> alterations?...
>
> I think the better question would be
>
> "Would anyone else target _my_ C subset?"
>
> To be blunt: in what you write, I do not see many arguments for
> answering that with 'yes'.
>
> Liberally licensed compilers implementing a superset of what you propose
> are available. How would targeting your C subset instead of any of these
> benefit any compiler writer?
>
(but, if it is a superset it defeats the point of being a subset... the
point of a subset is to trim off features... I can just leave it as a full C
compiler as well, since this is what it was before, I just figured I would
trim off a few "unnecessary" features, and maybe get more performance...).
after all, I would think the whole C99 featureset would be overkill for an
IL, and if really needed, the C frontend could be targetted, rather than the
C-derived IL...
> - better code generation?
> - more supported platforms?
> - faster compilation?
> - compilation in less memory?
> - fewer bugs?
> - better guarantees of future availability of the compiler?
> - ...?
>
> The only things that might swing this to "yes" for some languages could
> be your remark
>
AFAIK, the only real "direct" competitors I have are:
TinyC;
LLVM/CLang.
I support x86 and x86-64, but LLVM supports more.
JIT'ed code performance tends to be "comparable" to GCC (with default
options...).
Mono and the JVM could be competitors, but as I see it they are operating in
a different target domain, and under a different set of operating
principles, and so as I see it are not direct competitors...
>> intrinsics will likely be added for things like exception handling and
>> (managed) object manipulatio
>
would it also help if I mention that I am also implementing a good portion
of the JVM and .NET core featuresets?...
similarly, I have runtime support for both Prototype OO and Class/Instance
OO (SI+MI+Interfaces...).
the runtime also does Garbage Collection and other things, but these are not
intrinsics (not much need to make them intrinsic as I see it...).
> Reinder
> this way, I rework my upper-end to essentially produce C, which is fed
> into this newer (or, technically, older...) compiler, which in turns
> produces the IL for my lower-end.
Surely if the middle part is in C code, you don't have to bother with the
rest of it; just use an off-the-shelf C compiler.
I think that was most of the point of having C as a target language, in that
much of the compiler work can be off-loaded, and the project becomes
'portable'.
> as well, there will be some new language extensions:
> qualified names will be added (I am using an essentially C++ style syntax,
> namely: "Foo::Bar::baz");
I tried something like this, but as Foo$Bar$baz (for a C compiler accepting
$ in names, and when I also tried, unsuccessfully, to generate C code).
> note that there is an another C frontend, and so it will be in effect a
> C->C compiler, but will have the option of essentially "lowering" other
> difficult-to-compile features/extensions, such as inner functions and/or
> closures, exception handling, OO extensions, ...
> do people think using C as the basis for an IL is a good idea?...
> are some of my considered alterations (such as deprecating typedef)
> reasonable?...
If you are then also creating a compiler for that intermediate C, my thought
was you might as well target a lower-level, asm-friendly intermediate
language. Then it can be made to do anything without needing to create a new
C.
> or, hypothetically, would anyone else target a C subset with these
> alterations?...
(See comp.compilers, around 9-Feb-2009, "Is Assembler Language
essential...". I mentioned some problems in generating C instead of asm,
although everyone else seems to be targetting C quite happily.)
Mainly the problem was struggling to fit a new language into a form that C
is happy with. You say you need to modify the intermediate C to fit the
front end language better; why not combine these into single project: modify
that intermediate C to /be/ the front end language, or are they too
different?
--
Bart
Interpreters are slow (in general and IMO...).
> - target a real intermediate language such as LLVM
Even real intermediate languages, e.g., ANDF, can fail to gain acceptance
for non-technical reasons. Why would you recommend LLVM to him? Yes, LLVM
does what may be needed. But, has it really become a "de facto" standard
like Java?
> An argument for no longer using C as intermediate language is that C's
> data structures do not fit modern dynamic languages well.
If many of them, i.e., "dynamic languages", use C as their backend already,
how can this be?... e.g., Ruby.
> An argument for not using an intermediate language at all is that
> describing compilation as a series of source transformations may make
> theoretical sense, but in practice slows down the resulting compiler too
> much.
An argument, not a great one - but valid, for as many intermediate languages
as possible is that much of the information the programmer needs is lost wit
hout intermediate output. How does one determine where the compiler is
erroneous or why code doesn't compile without the intermediate layers? If
you take a compiler, say OpenWatcom, that directly emits binary, how do you
tell if there was an error in the assembly? There is no emitted assembly
code by the compiler.
> describing compilation as a series of source transformations may make
> theoretical sense, but in practice slows down the resulting compiler too
> much.
If not by these methods, how does one convert a language to binary or
assembly?... Binary or assembly is far more primitive than the HLL.
Doesn't that automatically imply that a transformation or series of such
will be required?
> (and writers targeting your language would spend quite a time doing that
> during the development of their compiler frontend)
I agree. That's especially true if they don't understand C... But, on the
other hand, they'll have to learn _something_ for _any_ choice of IL...
> > or, hypothetically, would anyone else target a C subset with these
> > alterations?...
>
> I think the better question would be
>
> "Would anyone else target _my_ C subset?"
I think the better question for him is:
"Can a high level C with extensions target _my_ C subset"
If not, then there will be problems implementing the advanced features of
other languages _with_ Algol heritage. Without Algol heritage, should one
even try to target C? I don't believe so.
> Liberally licensed compilers implementing a superset of what you propose
> are available. How would targeting your C subset instead of any of these
> benefit any compiler writer?
>
> - better code generation?
> - more supported platforms?
> - faster compilation?
> - compilation in less memory?
> - fewer bugs?
> - better guarantees of future availability of the compiler?
> - ...?
Simplified C could allow portability of more complicated applications to
embedded environments or be useful in OS environments where bootstrapping
utilities is useful. I.e., the C-to-C translation could preserve
functionality while allowing a simpler C compiler to compile the code.
Rod Pemberton
bitfields? floats? unions? multiple integer and char types? qualifiers?
volatile? const? void? void*?
I'd get rid of all those too... :-)
> so, the questions here are:
> do people think using C as the basis for an IL is a good idea?...
Many modern languages already do, don't they?
If you do, you'll be restricted to the functionality of that subset of C,
and to the Algol-like language concepts. You'll be restricted to the Unix
everything-as-a-file concept, etc.
Although I was using a FORTH-like IL (not anymore...), as were you - IIRC, I
really wouldn't want to try convert FORTH or LISP to C... I don't see them
as being similar enough to convert them optimally. I'd consider C as an IL
to be fine for FORTRAN, Pascal, or BASIC, etc. Of course, you'd need to
keep the floats for FORTRAN, at least.
C has the advantage that it can be implemented for the most part using very
basic ? concepts, such as addresses and pointers. However, C can be made
far more portable with slight extensions.
> are some of my considered alterations (such as deprecating typedef)
> reasonable?...
I don't know. I know that it's about the only way to extend C. If the
other language is very extendable, you might have some problems. The two
situations where I commonly use typedef is with structs to self-reference
the struct and to hide the code complexity of function pointers.
> or, hypothetically, would anyone else target a C subset with these
> alterations?...
Possibly. A high level C with extensions, like GCC, to a very simple
reduced C would be very useful, IMO. I know there is Necula's CIL (not to
be confused with MS CIL), but I haven't seen much else. There are a number
of ancient C compilers, like Small C, which could compile the output of the
C-to-C conversion. I think it would be good for OS development and embedded
development etc. It really depends on how many advanced features and
extensions you could convert into simple C. The idea of implementing a very
simple C language was popular back when C compilers weren't common or
widespread.
Rod Pemberton
Argh... I forgot to finish the following paragraph:
> C has the advantage that it can be implemented for the most part using
very
> basic ? concepts, such as addresses and pointers. However, C can be made
> far more portable with slight extensions.
C has the advantage that it can be implemented for the most part using very
basic computer science concepts, such as addresses and pointers. However, C
can be made far more portable with slight extensions. Examples of this is
Objective-C, and OpenStep. According t the Wikipedia page for OpenStep,
they changed five important things from NEXTSTEP that improved C's
portability. Other examples might be Unified Parallel C and Cg (C for
Graphics).
Rod Pemberton
this is not always an option...
the main issue is in cases where one can't use an off-the-shelf C compiler,
such as when compiling at runtime.
otherwise, why not?...
> I think that was most of the point of having C as a target language, in
> that much of the compiler work can be off-loaded, and the project becomes
> 'portable'.
>
yes, and it allows multiple backends...
this is another major reason I was considering C.
the main alternative is GIMPLE, which is essentially a C subset as well
(just it prunes down the language a little more than what I was
considering...).
producing GIMPLE would also give the choice of multiple backends (though,
for my uses, only one would work: LLVM).
the reason I had not been using LLVM to begin with is that it has not been
(until fairly recently) the case that anyone had been addressing certain
issues I consider to be of reasonable importance to my project...
(or, in short, LLVM did not do what I wanted...).
I think other people have figured out it needed those features as well, and
so people are getting around to it now...
>> as well, there will be some new language extensions:
>> qualified names will be added (I am using an essentially C++ style
>> syntax, namely: "Foo::Bar::baz");
>
> I tried something like this, but as Foo$Bar$baz (for a C compiler
> accepting $ in names, and when I also tried, unsuccessfully, to generate C
> code).
>
I considered '$' (and a few other possible unused characters), but decided
on '::' in my case...
sadly, qnames is one of those features that will need to be properly
supported, and so will not be a "portable" feature (as in, for anyone using
it I can't guerantee any other compilers will support it...).
(in my case, this will also partly serve to indicate to my backend that it
needs to use name mangling...).
void foo(); //likely to be compiled with an unmangled name.
void ::bar(); //will use a mangled name with global scope (this or use a
keyword...).
void Foo::bar(); //will use a qualified mangled name...
below this, RPNIL (my "actual" IL) uses '/' as the qname syntax, so the
compiler will just sort of translate '::' to '/' when producing output...
note that this prefix will not be included in the final mangled name...
to be able to produce portable C would likely require using name mangling in
the IL, which is an idea I dislike (I would much rather use non-portable
extensions...).
>> note that there is an another C frontend, and so it will be in effect a
>> C->C compiler, but will have the option of essentially "lowering" other
>> difficult-to-compile features/extensions, such as inner functions and/or
>> closures, exception handling, OO extensions, ...
>
>> do people think using C as the basis for an IL is a good idea?...
>> are some of my considered alterations (such as deprecating typedef)
>> reasonable?...
>
> If you are then also creating a compiler for that intermediate C, my
> thought was you might as well target a lower-level, asm-friendly
> intermediate language. Then it can be made to do anything without needing
> to create a new C.
>
I already have the compiler at hand... (AKA: all the code already
exists...).
as noted elsewhere, this is being derived from my original C frontend (used
from 2007-2008...).
a lower-level IL exists, but my problem is its inflexibility (or, in short,
I am having difficulty ramming all these "high level" features into it...).
in the years from 2007-2009, the low-level codegen has gotten fairly
brittle...
(actually, this particular C frontend was retired originally for being
fairly brittle, but it was at least fairly fast, and it still has a good
deal more flex left than my IL...).
but, I guess it is sad... the whole lower end of my compiler is essentially
built out of a "house of cards...". it gets increasingly difficult to do
anything without the whole thing breaking apart...
>> or, hypothetically, would anyone else target a C subset with these
>> alterations?...
>
> (See comp.compilers, around 9-Feb-2009, "Is Assembler Language
> essential...". I mentioned some problems in generating C instead of asm,
> although everyone else seems to be targetting C quite happily.)
>
yep...
my project also accepts ASM, and actually has a good amount of utility
features defined at the ASM/Linker level.
it is actually through linker-level features that one of the interfaces to
my object system is implemented (as well as TLS, exception handling, and a
few other things...).
so, to interface with my codebase, one does not need to target some
particular IL, just conform with the C ABI for the arch in question, and for
a little higher performance, be able to interface with the ASM-level ABI...
> Mainly the problem was struggling to fit a new language into a form that C
> is happy with. You say you need to modify the intermediate C to fit the
> front end language better; why not combine these into single project:
> modify that intermediate C to /be/ the front end language, or are they too
> different?
>
actually, I have 2 C compilers:
a frontend C compiler, and this C compiler...
the frontend compiler will accept a decent portion of C99 (and a good number
of compiler extensions), and so is the "full featured" C.
this is the older/original frontentd, and will be reused as an IL...
this is why I figured I could leave off the preprocessor:
if I really need to preprocess code, I can use the one from the frontend C
compiler (essentially a direct descendant of the one I am leaving off, and
unlike the main compiler, actually got faster...).
FWIW, I don't need to have to worry about yet-another preprocessor, when I
can reuse one which exists in another library within the same subproject
(AKA: the main C frontend...).
leaving it off by default, however, allows the code to be compiled faster
(so, the PP will be "manual", rather than "automatic", as leaving off the PP
can easily save a good number of milliseconds...).
(another variant of the PP is used on the assembler, but this is left on
even if not often needed, as usually the ASM is small enough that the PP
does not contribute significantly to compile times...).
(the assembler's PP is essentially a descendant of the one used in the main
frontend, but some features developed in the ASM's PP were propagated back
to the main PP, ...).
or such...
> --
> Bart
bitfields, yeah, probably, since a bitfield can be implemented via masks and
shifts easily enough...
'floats' are too damn useful, and can't be effectively done without them.
unions are useful as well.
multiple integer types: one can't safely get rid of them while still keeping
ABI conformance...
it is the same reason as to why one can't get rid of cdecl...
also, char and short and similar are useful due to their reduced size.
qualifiers contain useful info which is needed to effectively generate code.
volatile is useful, as it indicates how to (safely) make use of a variable
(otherwise, to be safe, requires treating all variables as volatile, which
eliminates some kinds of optimizations).
const, maybe, since it mostly matters to the upper end, but may also have
relevance to the optimizer.
void contains useful semantic info (and once one gets past having to worry
about it, all pointers are effectively void anyways, and their type ceases
to matter in general...).
or, in short, all of these features (except maybe bitfields) are needed in
the IL to compile C...
>> so, the questions here are:
>> do people think using C as the basis for an IL is a good idea?...
>
> Many modern languages already do, don't they?
>
yeah, gcc does...
pretty much everything in gcc turns into GIMPLE, which is essentially a C
subset+superset...
it is like, rip out all but the most basic expressions, but add intrinsics
for OpenMP and all sorts of other things...
> If you do, you'll be restricted to the functionality of that subset of C,
> and to the Algol-like language concepts. You'll be restricted to the Unix
> everything-as-a-file concept, etc.
>
not really...
I am not removing features of much semantic relevance...
it is fairly easy to, say, migrate an inner function to the toplevel, move
all the declarations to the top of the function, ...
after all, I already do this when compiling to RPNIL...
> Although I was using a FORTH-like IL (not anymore...), as were you - IIRC,
> I
> really wouldn't want to try convert FORTH or LISP to C... I don't see
> them
> as being similar enough to convert them optimally. I'd consider C as an
> IL
> to be fine for FORTRAN, Pascal, or BASIC, etc. Of course, you'd need to
> keep the floats for FORTRAN, at least.
>
Scheme and LISP can be "reasonably" effectively compiled to C, but ideally
these sort of languages would need a more specialized backend (namely, one
which knows about CPS, ...).
> C has the advantage that it can be implemented for the most part using
> very
> basic ? concepts, such as addresses and pointers. However, C can be made
> far more portable with slight extensions.
>
yep.
>> are some of my considered alterations (such as deprecating typedef)
>> reasonable?...
>
> I don't know. I know that it's about the only way to extend C. If the
> other language is very extendable, you might have some problems. The two
> situations where I commonly use typedef is with structs to self-reference
> the struct and to hide the code complexity of function pointers.
>
absent typedef, the idea would be to make all declarations manually, such
as:
struct Foo_s { ... };
int UseFoo(struct Foo_s *foo, int x)
{
...
}
(from the way typedef works, everything that typedef does can be done
manually, and more so, what typedef does is typically done in the upper-end
of the compiler anyways...).
of course, for non-C input languages, typedef would be a lot more useful (as
well as the preprocessor, ...).
so, I guess it may be best to leave them in...
I guess another possibility, which would be more of a tradeoff, would be to
allow typedef to be disabled in cases where it is known not to be needed,
but otherwise it remains in the compiler.
>> or, hypothetically, would anyone else target a C subset with these
>> alterations?...
>
> Possibly. A high level C with extensions, like GCC, to a very simple
> reduced C would be very useful, IMO. I know there is Necula's CIL (not to
> be confused with MS CIL), but I haven't seen much else. There are a
> number
> of ancient C compilers, like Small C, which could compile the output of
> the
> C-to-C conversion. I think it would be good for OS development and
> embedded
> development etc. It really depends on how many advanced features and
> extensions you could convert into simple C. The idea of implementing a
> very
> simple C language was popular back when C compilers weren't common or
> widespread.
>
yep...
I am running in a case where existing C compilers aren't really available...
as I mentioned elsewhere, it was not until recently that LLVM gained a few
features which I felt were fairly important.
now LLVM has them, so it is a lot more of an issue.
using C as an IL, however, does make the prospect of using LLVM more likely,
since C is one of the supported input languages to LLVM, it would mean there
would be less work to produce C which LLVM would accept...
or such...
>
> Rod Pemberton
>
>
yep...
FWIW, UPC did offer a few ideas which I partly "borrowed"/"adapted" for my
effort...
Cg, yes, ok...
I use GLSL instead...
GLSL had offered inspiration before, mostly for the idea of "physics
shaders", but FWIW I have never really done much with this idea (partly
because it would create a dependency between my physics engine and my
compiler...).
>
> Rod Pemberton
>
>
>> bitfields? floats? unions? multiple integer and char types?
>> qualifiers?
>> volatile? const? void? void*?
>>
>> I'd get rid of all those too... :-)
>>
>
> bitfields, yeah, probably, since a bitfield can be implemented via masks
> and shifts easily enough...
>
> 'floats' are too damn useful, and can't be effectively done without them.
> unions are useful as well.
Not indispensable:
static union {int i; double x; char c;} u;
u.i=100; printf("u.i %d\n",u.i);
u.x=200.0; printf("u.x %f\n",u.x);
u.c='Z'; printf("u.c %c\n",u.c);
can be replaced with:
char u[8];
*((int*)u)=100; printf("u.i %d\n",*((int*)u));
*((double*)u)=200.0; printf("u.x %f\n",*((double*)u));
*((char*)u)='Z'; printf("u.c %c\n",*((char*)u));
The same pattern may be used in many places. Structs could also be removed
completely, except they are useful because they are the only arbitrary-size
datatype passed by value.
Another problem would be debugging the final code and trying to recover the
original source lines as the intermediate C may not be too readable.
--
Bart
What exactly is C specific in this point, what doesn't go for say Pascal,
Modula-2,Ada etc?
IME, many Pascal variants seem a little weak WRT pointers (at least vs C).
C has pointers as essentially one of the core language features, whereas
Pascal/... tend to regard them as second-class citizens.
as for portability, yeah, Pascal is typically reasonably portable, but on
the downside, typically rather compiler specific.
code in C for one compiler will usually work without much issue in another
compiler, whereas one is lucky if code from one Pascal compiler will work in
another...
Ada is standardized, so there should be less of an issue, apart from the
relatively limited usage of Ada.
...
Then how about graphics? (requires floats in many places)
Would you keep the double? It's even worse than a float.. Why waste the
space when you don't need it and kill the performance of non-trivial
operations such as division which are a lot slower on doubles than on
floats?
Interoperability with everything (only 64bit integers will kill everything
including file IO, and anything smaller will kill math) and then there are
memory space considerations (just because people have several gigs of ram
doesn't mean they want you to waste it)
No volatile? How about threading then?
No const? Doesn't matter for the code as much, but you'd put more load on
the optimizer..
But throwing out bitfields and unions wouldn't hurt too much, you can
emulate both using other features..
And void, well, just return zero I guess
Not too sure about the void pointer, I can't really think of a place where
it's absolutely required. And don't give me "UB!" because UB doesn't make it
required, and since it's a new language anyway you could just define the
behaviour to get rid of the UB.
yeah...
however, it can be noted that, as far as implementation goes, structs and
unions are only trivially different, and so, if one keeps structs, they may
as well keep unions (which come almost "for free" if one supports structs).
however, this sort of large-scale feature removal is what I would do in a
lower-level IL (such as, one flattened into TAC or SSA form), whereas this
will still be a higher-level IL (as in, no TAC or SSA).
similar, this will not be the "bottom-level" IL either, as I will likely
continue having another IL below this (namely, one much closer to the final
codegen...).
however, I will note that even my partial low-level IL continued supporting
structs and unions (as does GNU's GIMPLE, and many other ILs...).
> --
> Bart
I've no idea what you are talking about. Are you sure you are not confusing
like e.g. a 16-bits real mode Turbo Pascal with a protected mode linear
memorymodel C compiler?
Some dialects (and by that I mean the ISO standard) even have more
possibilities than C, like the ability to allocate (with new() which is like
Malloc) different sizes for "pointer to union" depending on which union
variant is chosen.
> as for portability, yeah, Pascal is typically reasonably portable, but on
> the downside, typically rather compiler specific.
FPC and Delphi are highly compatible to eachother, and together form most of
the Pascal use.
> code in C for one compiler will usually work without much issue in another
> compiler, whereas one is lucky if code from one Pascal compiler will work in
> another...
Actually, Pascal works better in that department, because e.g. FPC
implements a way larger set of the "other" one (Delphi in this case).
Differences in language are even more negiable (more in the realm of details
of the COM interfacing)
well, I haven't done much Pascal coding, but what I remember was long ago
using FreePascal (maybe late 90s).
I forget the details, only that I think I remember the pointer-system at the
time being fairly limited, as in only really having pointers to basic types,
not having C-style pointer arithmetic, ...
then again, at the time I also remembered being unable to use if/goto
everywhere, since this was before I eventually discovered that heavy use of
if/goto was not a good style... (my memory is that goto and labels were not
working...).
I also remember poor english in the compiler messages, sometimes mashed-up
with german, ...
...
but, then again, my memory may be wrong...
> Some dialects (and by that I mean the ISO standard) even have more
> possibilities than C, like the ability to allocate (with new() which is
> like
> Malloc) different sizes for "pointer to union" depending on which union
> variant is chosen.
>
ok.
well, I would have to look into it...
>> as for portability, yeah, Pascal is typically reasonably portable, but on
>> the downside, typically rather compiler specific.
>
> FPC and Delphi are highly compatible to eachother, and together form most
> of
> the Pascal use.
>
well, I thought I remembered there being a lot more options last I looked,
but admittedly, I haven't looked much at Pascal in years (it just never
really comming up as particularly relevant...).
>> code in C for one compiler will usually work without much issue in
>> another
>> compiler, whereas one is lucky if code from one Pascal compiler will work
>> in
>> another...
>
> Actually, Pascal works better in that department, because e.g. FPC
> implements a way larger set of the "other" one (Delphi in this case).
> Differences in language are even more negiable (more in the realm of
> details
> of the COM interfacing)
>
yes, ok...
>
>
That would have to be fairly early then, it has been at least ok since +/-
1996.
> then again, at the time I also remembered being unable to use if/goto
> everywhere, since this was before I eventually discovered that heavy use of
> if/goto was not a good style... (my memory is that goto and labels were not
> working...).
That can be true relative to ISO Pascal which is more permissive (and needs
special stackframes to handle it), though I doubt that C's setlongjmp supports
that.
> I also remember poor english in the compiler messages, sometimes mashed-up
> with german, ...
Could be. I've seen C compilers with Russian errormsgs evne.
ok...
well, as noted, my memory is stale, and I had messed with several compilers
at the time (though I don't remember what the others were...).
I guess whichever version was on DOS that I would have probably had
available maybe about 1997/1998 or so... (it could have easily been an older
release for all I know...).
>> then again, at the time I also remembered being unable to use if/goto
>> everywhere, since this was before I eventually discovered that heavy use
>> of
>> if/goto was not a good style... (my memory is that goto and labels were
>> not
>> working...).
>
> That can be true relative to ISO Pascal which is more permissive (and
> needs
> special stackframes to handle it), though I doubt that C's setlongjmp
> supports
> that.
>
I remember more now:
I used to use goto all over the place in C.
but, I also remember that around the time I was in a transition between
BASIC and C...
functions were used sparingly, and if/goto was the main control-flow
mechanism.
C had goto, but I had remembered trying to use it in Pascal and not having
it working.
I also remember I sort of learned ASM at the time, and I had often made use
of the ASM output from the compiler to help me better understand how things
behaved, ...
(at this point there is some garbage in my memory, mislocated information
about compilers compiling to bytecode and having been looking at bytecode,
some information about fileformats, bunches of other fragmentary and
unsorted stuff, asm fragments, ...).
actually, it is probably funny, but it was not until about 2007 or so, that
I noticed I could do generic [reg+reg*scale+disp] addressing in ASM...
I had for the most part thought what I remembered:
[EBX+disp]
[EBP+disp]
[ESI+disp]
[EDI+disp]
I noticed this while writing my assembler...
it may seem funny, but it turned out my ASM style have been heavily
"flavored" by the technical restrictions of 16-bit real-mode, and I had not
noticed that these restrictions were no longer in place...
[BX+disp], [BP+disp], [SI+disp], [DI+disp]
sometimes though, memory messes up, like a few years ago implementing some
file-handling code, and having it not work. I then went and looked at it,
and resorted to finding info. the problem was that in my memory, several
fields in one of the structures had gotten out of order (over the some-odd
years or so since I had done much with the format...).
(going too much further back, namely back into my elem-school years, and my
memory becomes rather fragmented, random chunks of events mashed up with
random pieces of information, ... almost surreal in a way...).
>> I also remember poor english in the compiler messages, sometimes
>> mashed-up
>> with german, ...
>
> Could be. I've seen C compilers with Russian errormsgs evne.
>
yep...
So?
Is it important enough language feature to keep or implement in his IL? I'd
say "no." Floats/doubles are not a high use feature. He might keep because
they are needed to compile floating point.
E.g., by choosing all characters to be "unsigned char," he can eliminate
"signed char" from the IL. There will be a few situations where code
correction is needed to compensate for different range checks, but they can
be transformed.
> Would you keep the double? It's even worse than a float..
No.
> Interoperability with everything
I think that's part of the point. He can't get a 100% interoperability for
multiple languages with a generic IL. He'd need a highly specialized IL.
So, what does he need for a very _useful_ IL? He definately wants to keep
control-flow, procedures, integer arithmetic, characters, etc. If some of
the features he needs can be converted to simpler forms, then remove the
"compound" features...
> No volatile? How about threading then?
> No const? Doesn't matter for the code as much, but you'd put more load on
> the optimizer..
C existed without these. These could be considered specialized
functionality - which, in my mind, justifies considering them for removal
from an IL. Does an IL based on C really need "const" once the
type-checking on the language has been done?
> But throwing out bitfields and unions wouldn't hurt too much, you can
> emulate both using other features..
Ditto for const, array declarations, void, void*, typedefs...
> And void, well, just return zero I guess
> Not too sure about the void pointer, I can't really think of a place where
> it's absolutely required.
C existed without these too.
Rod Pemberton
The implementation of arrays and strings and pointers are different. Pascal
is severely lacking in low-level features, and was lacking pointers.
Basically, you're trapped to programming in a limited box that you can't
escape from. I'm not familiar with Modula-2 or Ada. I'm familiar with a
variant of PL/1 which was very Pascal-ish.
- C requires all objects to be at different addresses.
- C requires all C objects to map onto a contiquous sequence of C
characters.
- C implements an "offset operator" which indexes contiguous sequences of C
characters.
- C doesn't implement arrays.
- C does implement an array declaration. That array declaration is
effectively converted into a pointer that can be used with the offset
operator in order to simulate arrays.
- C doesn't implement strings.
- C does simulate "strings", using an all bit-zero character to terminate a
simulated "array"
- etc.
In other words, you only need a contiquous sequence of C characters and an
address, to implement the (non-specialized) features of C, including
"strings" and "arrays". Just due to the way strings are implemented in
Pascal and PL/1 etc., I'd say C fits better with the underlying computing
platform. However, C has numerous other low-level features that most
high-level languages lack. These too add to it's usefulness as an IL.
Rod Pemberton
Well it means that if you chuck out floats, you don't have a screen. Except
through a pixel API. Any other API is going to involve floats at some point.
I also really can't agree on floats not being a high use feature.. maybe not
in software you write, but games are loaded with floats. Especially (no
surprises here) the graphics part. No floats = no games. One could decide
not to care about that, but that would certainly make the IL non-generic.
> E.g., by choosing all characters to be "unsigned char," he can eliminate
> "signed char" from the IL. There will be a few situations where code
> correction is needed to compensate for different range checks, but they
> can
> be transformed.
signed chars never made sense anyway..
>> Would you keep the double? It's even worse than a float..
>
> No.
Good. No really, I mean it. Having doubles but not floats would be a far
bigger WTF than having neither.
>
>> Interoperability with everything
>
> I think that's part of the point. He can't get a 100% interoperability
> for
> multiple languages with a generic IL. He'd need a highly specialized IL.
> So, what does he need for a very _useful_ IL? He definately wants to keep
> control-flow, procedures, integer arithmetic, characters, etc. If some
> of
> the features he needs can be converted to simpler forms, then remove the
> "compound" features...
So how can you have something useful with only 1 type of integer? You
couldn't make any external function calls with arguments with an IL like
that, which would essentially isolate it to something useless. Unless you
cheat by, say, hardcoding the arguments in the function name and selecting
the right name at runtime (nasty hack)
>
>> No volatile? How about threading then?
>> No const? Doesn't matter for the code as much, but you'd put more load on
>> the optimizer..
>
> C existed without these. These could be considered specialized
> functionality - which, in my mind, justifies considering them for removal
> from an IL. Does an IL based on C really need "const" once the
> type-checking on the language has been done?
The world existed without threading.. that world is gone.
Const is not so much about type checking as it is about letting a caller
know that the called code won't change something so that no copy is needed,
not vital, but important (not as important as floats of course)
>
>> But throwing out bitfields and unions wouldn't hurt too much, you can
>> emulate both using other features..
>
> Ditto for const, array declarations, void, void*, typedefs...
You can't emulate const unless you count "just leaving it out and doing
expensive analyses to insert it back in where possible" as a valid
alternative
Doesn't seem like a good plan to me..
IME, they are a fairly high use feature...
poor float support would kill the performance of all my huge masses of
numerical code...
> E.g., by choosing all characters to be "unsigned char," he can eliminate
> "signed char" from the IL. There will be a few situations where code
> correction is needed to compensate for different range checks, but they
> can
> be transformed.
>
but, there is no real "advantage" to eliminating, for example, signed
char...
more so, I will stress that this is a "high level" IL, not a "low level"
IL...
stripping the language down into a minimalistic core is for a low-level IL,
not a high-level IL.
I started designing such a low-level IL recently as well (originally
considering the name BXTIL but then considering IR/DKA-0 instead...). this
IL would essentially serve as a simplistic wrapper over the code-generator
(and is in TAC form, presumably post-SSA, ...).
however, these are 2 different domains, representing 2 different stages of
the process.
BMC -> SSA -> IR/DKA -> ASM
but, BMC and IR/DKA would be essentially somewhat different, with BMC not
needing minimalism, and IR/DKA not needing a whole lot of anything (IR/DKA
would only have a loosely C-like syntax...).
but, initially, it will be:
BMC -> RPNIL -> ASM
since IR/DKA would likely take a decent amount of time to implement (as it
would be essentially a restructuring and significant rewrite of the
low-level codegen...). it would be, of course, far more minimalistic...
BMC -> LLVM
is also a possible route...
>> Would you keep the double? It's even worse than a float..
>
> No.
>
it may seem odd, but SSE/SSE2 is currently a fairly central feature of my
compiler, as in, errm... code is not likely to work right on processors
without it...
>> Interoperability with everything
>
> I think that's part of the point. He can't get a 100% interoperability
> for
> multiple languages with a generic IL. He'd need a highly specialized IL.
> So, what does he need for a very _useful_ IL? He definately wants to keep
> control-flow, procedures, integer arithmetic, characters, etc. If some
> of
> the features he needs can be converted to simpler forms, then remove the
> "compound" features...
>
by having good integration with 'cdecl' and 'SysV' at the ABI level, as well
as good C integration and acceptable C++ integration, one effectively has
interoperability with a good portion of "everything"...
nearly anything else can be patched up with "bridges"...
>> No volatile? How about threading then?
>> No const? Doesn't matter for the code as much, but you'd put more load on
>> the optimizer..
>
> C existed without these. These could be considered specialized
> functionality - which, in my mind, justifies considering them for removal
> from an IL. Does an IL based on C really need "const" once the
> type-checking on the language has been done?
>
const allows optimizations such as constant propagation, ... otherwise, any
such propagation has to be done in the frontend.
>> But throwing out bitfields and unions wouldn't hurt too much, you can
>> emulate both using other features..
>
> Ditto for const, array declarations, void, void*, typedefs...
>
array declarations are useful for allocating storage, and for some kinds of
sanity checks.
void may have useful semantics, and so is useful to keep in a higher-level
IL.
my main reason for deprecating typedef would be to make the parser faster
and allow for context-independent parsing.
as-is, to parse C damn near every name token ends up having to be checked to
see if it might be a type, whereas eliminating typedef could largely
eliminate this check.
however, a reasonable option may be to have a kind of typedef-disable
feature:
__pragma disable_typedef;
the compiler, upon seeing this, will disable typedef support...
but, then, thinking about it, not using typedef should have a similar
performance boosting effect, so this pragma is unnecessary (it would have a
better fit as a security feature than an optimization feature, and actually
disable using typedef, rather than type lookup...).
so, the simplest option is just to not use typedef, and pretend it is
invalid (aka: the original point of deprecating it...).
as a slight cost (of no typedefs), one would end up seeing:
'struct <whatwhatever>' and 'union <whatever>' all over the damn place, but
this may be acceptable...
(all other types would be their associated fully-expanded primitive types).
sadly, this would require syntax extensions for things like function pointer
types.
__proto int foo(int x);
__proto foo *bar()
{
...
}
this being because this is a lot closer to how my compiler internally
handles function pointers, and otherwise it would be too much effort to
regenerate the "proper" syntax, or it would involve fairly heavy use of
typedef...
(I guess the opposite extreme is conceivable in a C-based IL: namely making
every type essentially a typedef, which would allow its own set of
simplifications and optimizations...).
>> And void, well, just return zero I guess
>> Not too sure about the void pointer, I can't really think of a place
>> where
>> it's absolutely required.
>
> C existed without these too.
>
just because earlier C did not have something is not a good reason not to
have it IMO...
many things useful in an IL did not originally exist in C...
I will have many features which "pure" C will not have...
>
> Rod Pemberton
>
>
agreed...
similarly, vs float other alternatives, such as fixed point, are fairly
lame...
more so, why bother, when the CPU does most of the work?...
of course, if the option is to just not use x87, this is more reasonable, as
I can then do most of the floating point via SSE (this is actually what I
do, although it could be better, for example, if I could manage to pull off
"auto vectorization" or similar...).
none the less, x87 does have a few uses, namely:
it does sin, cos, tan, ... which otherwise would have to be simulated (and
thus be slower...);
well, that and it is a good way to pull of a few other calculations which
are at-present problematic with SSE (either awkward or not very efficient).
for example, vector dot product (on SSE2) is better done via x87 IME, ...
none the less though, these cases can be handled by special code fragments,
and so for the most part the codegen can pretend that x87 does not exist...
>> E.g., by choosing all characters to be "unsigned char," he can eliminate
>> "signed char" from the IL. There will be a few situations where code
>> correction is needed to compensate for different range checks, but they
>> can
>> be transformed.
>
> signed chars never made sense anyway..
>
they do have a use as small signed integers, but granted 'char' does not
make as much sense.
they can instead be called 'signed bytes' (as done in C# and friends...).
>>> Would you keep the double? It's even worse than a float..
>>
>> No.
>
> Good. No really, I mean it. Having doubles but not floats would be a far
> bigger WTF than having neither.
>
well, one might go this route for some special-purpose uses, such as in
certain dynamic languages, ...
I actually have several additional float types:
hfloat, which are 16 bit floats and handled via a load/store hack with 32
bit floats;
float128, which before were sort of faked, but then I started implementing a
more "proper" SW implementation, but never really cared enough to get this
fully working;
as well as float80/"long double".
the original float128 "faking" was later re-made as long-double support,
which is basically an 80-bit float being passed around in a 128-bit SSE
register, then when operations are performed, loaded into the FPU,
calculated, and stored back (I also moved the float80 to the lower end of
the register after noticing that float80 and float128, though having the
same "bits", are not value compatible...).
so, presently, float128 is broke, and before it was just a long-double in
the upper end of the register...
if needed, new "faking" logic could be implemented, mostly by using a few
shifts and logic-ops to convert between them, and then maybe get around to
properly implementing, well, at least the core arithmetic operations...
>>
>>> Interoperability with everything
>>
>> I think that's part of the point. He can't get a 100% interoperability
>> for
>> multiple languages with a generic IL. He'd need a highly specialized IL.
>> So, what does he need for a very _useful_ IL? He definately wants to
>> keep
>> control-flow, procedures, integer arithmetic, characters, etc. If some
>> of
>> the features he needs can be converted to simpler forms, then remove the
>> "compound" features...
>
> So how can you have something useful with only 1 type of integer? You
> couldn't make any external function calls with arguments with an IL like
> that, which would essentially isolate it to something useless. Unless you
> cheat by, say, hardcoding the arguments in the function name and selecting
> the right name at runtime (nasty hack)
>
agreed...
one needs at least 2 integer types even to be able to perform arithmetic
effectively...
the additional types offer a lot of value as well (smaller types for reduced
storage and other uses, bigger types when one wants big values and is
willing to pay the cost...).
for example, I have 128 bit integers, and they have a few more uses than I
had originally expected for them. bigger would be inconvinient though, as
there would be no suitible registers.
then again, there is a trick I came up with for structs and matrices:
internally, the full type is used, but the register is actually a GPR
holding a pointer to the value
in this way, a register 'can' be used, only the actual value is
memory-backed...
another (almost possible) trick would be to used some of the SSE registers
as "fake GPRs", but the problem here is that it would be difficult to do
this efficiently (getting stuff into and out of SSE registers is difficult
to do efficiently), although it can be done better if one imagines them as
pairs of 64-bit registers (where there are operations to load/store only the
upper and lower half of the register...).
this could give up to about 16 additional 64-bit "registers" (x86, 32 on
x86-64), at the cost that they would conflict with vector and floating-point
use of the registers (although... it may be possible to multiplex this usage
with float and double operations, such that one can have 8 doubles and 8
64-bit values in "registers", each sharing their respective half of the 8
SSE registers...).
I guess I could do this, as thinking about it, at present whenever one of
these regs goes unused, or holds a float or double, possible register space
is wasted...
(on x86-64, this could mean having, potentially, 48 64-bit integer registers
available...).
one likely use for them is holding either overflow from the GPRs, or for
pointers to certain (non-struct) large memory-mapped datatypes (matrices,
bignums, bigvectors, ...).
on x86, they would also be a good "alternative" for long-long (which
presently allocate full SSE registers...).
>>
>>> No volatile? How about threading then?
>>> No const? Doesn't matter for the code as much, but you'd put more load
>>> on
>>> the optimizer..
>>
>> C existed without these. These could be considered specialized
>> functionality - which, in my mind, justifies considering them for removal
>> from an IL. Does an IL based on C really need "const" once the
>> type-checking on the language has been done?
>
> The world existed without threading.. that world is gone.
> Const is not so much about type checking as it is about letting a caller
> know that the called code won't change something so that no copy is
> needed, not vital, but important (not as important as floats of course)
>
yep...
it is a useful hint to the optimizer...
>>
>>> But throwing out bitfields and unions wouldn't hurt too much, you can
>>> emulate both using other features..
>>
>> Ditto for const, array declarations, void, void*, typedefs...
>
> You can't emulate const unless you count "just leaving it out and doing
> expensive analyses to insert it back in where possible" as a valid
> alternative
> Doesn't seem like a good plan to me..
>
yep.
Whoever mentioned doing away with floats can't have been serious. Floats are
a fundamental machine type.
> of course, if the option is to just not use x87, this is more reasonable,
> as
Even here, you want to hide the implementation of floats from the higher
level language, which shouldn't care whether there is an FPU or not. So any
IL should have them. And the overheads are minimal: you need to set aside 4
or 8 bytes (or 10 or 12), and provide arithmetic on them.
> one needs at least 2 integer types even to be able to perform arithmetic
> effectively...
> the additional types offer a lot of value as well (smaller types for
> reduced storage and other uses, bigger types when one wants big values and
> is willing to pay the cost...).
Depends on the top-level language. Some may want just a generic integer (or
even numeric) type, others may want a whole family of sizes, signed and
unsigned. The IL should at least reflect the machine types available. On x86
that would be 8, 16, 32-bits, and also 64-bits (even on 32-bit cpu).
> for example, I have 128 bit integers, and they have a few more uses than I
> had originally expected for them. bigger would be inconvinient though, as
> there would be no suitible registers.
I doubt you really want those 128 bits as /numbers/, probably just some data
or collection of fields that needs 16-bytes in total. If there really was a
need for integers that big, then you'd probably want more than 128 bits.
--
Bart
yep.
>> of course, if the option is to just not use x87, this is more reasonable,
>> as
>
> Even here, you want to hide the implementation of floats from the higher
> level language, which shouldn't care whether there is an FPU or not. So
> any IL should have them. And the overheads are minimal: you need to set
> aside 4 or 8 bytes (or 10 or 12), and provide arithmetic on them.
>
granted, yes, I am mostly just left thinking of implementation details
here...
long-double generally needs 16-bytes (for efficiently loading/storing SSE
regs), but generally uses 12-bytes (on x86) for arguments (this is what gcc
uses, MSVC does not support long-double via what info I have...). on x86-64,
a full 16-bytes is typically used.
>
>> one needs at least 2 integer types even to be able to perform arithmetic
>> effectively...
>> the additional types offer a lot of value as well (smaller types for
>> reduced storage and other uses, bigger types when one wants big values
>> and is willing to pay the cost...).
>
> Depends on the top-level language. Some may want just a generic integer
> (or even numeric) type, others may want a whole family of sizes, signed
> and unsigned. The IL should at least reflect the machine types available.
> On x86 that would be 8, 16, 32-bits, and also 64-bits (even on 32-bit
> cpu).
>
yep.
>> for example, I have 128 bit integers, and they have a few more uses than
>> I had originally expected for them. bigger would be inconvinient though,
>> as there would be no suitible registers.
>
> I doubt you really want those 128 bits as /numbers/, probably just some
> data or collection of fields that needs 16-bytes in total. If there really
> was a need for integers that big, then you'd probably want more than 128
> bits.
>
they are rare, but happen sometimes...
one place I use them (indirectly) is in "wide pointers", which are basically
an intended mechanism for DSM (Distributed Shared Memory) and persistent
storage.
the reason they are so large is that this allows more-or-less asynchronous
addressing and an almost arbitrarily large "cluster", whereas a smaller
address (say, 64 bits), would likely require a smaller cluster and
centralized address assignment (or limiting each DSM/store segment to 4GB or
less...).
implementation-wise, it is sort of a "hybrid" between linear and segmented
addressing. the idea is currently that the space itself is linear, but
divided up into more or less evenly-sized sgements.
but, thus far, I have not had enough need for this feature to really bother
finishing implementing it (sort of like float128...), more it is just there
should I have need for such a huge space.
(basically, the handling, pointer arithmetic support, ... is in place, just
what is lacking is the code to actually do something when one tries to
access memory via these pointers...).
> --
> Bart
I've no idea what you mean here. Even bitpacking is part of the nearly every
dialect's syntax (though there is no requirement to implement it).
In the past (the long forgotten standards, 16-bit segmented models and the
virtual machine experiments) had other rules, but every 32-bit Pascal I ever
used (read the Borland dialects used by 99.9% of the users) implements the C
subset, except some weirdness as the ? operator and post and preincrement as
expression (a similar feature as statement exists). And of course the
preprocessor is not mandatory.
Over things like ? and ++ one could discuss if they are baroque remnant of
ancients architectures or not, but lets skip that discussing by stating
whatever your opinion on them is, they are by no means crucial for lowlevel
programming.
C effectively has no string type, and its library emulation of one is 1:1
convertable to Pascal. No difference there.
As far as I know Pascal even has been slightly upper hand, performancewise,
because it didn't have pointer aliasing rules to hinder the compiler (which
afaik you can only can escape off in C since C99)
> Basically, you're trapped to programming in a limited box that you can't
> escape from.
I wonder how this is possible
> I'm not familiar with Modula-2 or Ada. I'm familiar with a
> variant of PL/1 which was very Pascal-ish.
Pascallish like C# is Cish? IOW in syntax or in language?
> - C requires all objects to be at different addresses.
Union?
> - C requires all C objects to map onto a contiquous sequence of C
> characters.
> - C implements an "offset operator" which indexes contiguous sequences of C
> characters.
I don't get these, could you explain what you mean here, and what the
relevance for lowlevel is? I think you mean that most constructs are either
static structures or pointers to them.
> - C doesn't implement arrays.
> - C does implement an array declaration. That array declaration is
> effectively converted into a pointer that can be used with the offset
> operator in order to simulate arrays.
While it saves maybe on the compiler in a 4k environment, I'm more
interested in lowlevel TARGET than HOST. So while this may true, could you
please explain why having a normal array in addition to a pointer is a bad
thing?
> - C doesn't implement strings.
> - C does simulate "strings", using an all bit-zero character to terminate a
> simulated "array"
(a decision regretted by many security consultant and frustrated
programmers)
> - etc.
>
> In other words, you only need a contiquous sequence of C characters and an
> address, to implement the (non-specialized) features of C, including
> "strings" and "arrays". Just due to the way strings are implemented in
> Pascal and PL/1 etc., I'd say C fits better with the underlying computing
> platform. However, C has numerous other low-level features that most
> high-level languages lack. These too add to it's usefulness as an IL.
You are on the wrong track here. First, there is no C string construct that
I'm aware of that can't be matched in Pascal. There is a pointer to char,
you can increment it, check for zero, subtract them, add them. Stronger
even, Delphi/FPC come with a strings unit with the common null-terminated
char* routines.
IOW it is roughly like C++, you can simply do the pointer trick if you feel
masochistic or (more rarely) see a need.
Note even that the Delphi higher level string type is fully compatible (as
in compiletime cast, no conversion whatsoever) with char *, as long as you
don't try to reallocate it.
only for "recent" Pascals (read: past 10 years), which could almost more
correctly be called Delphi's than Pascal's...
what I remember of Pascal at least, was from a compiler I was using in the
late 90s, and with my reference material being books from the 80s...
(at the time, I had also partly learned Fortran77, but noted that F77 style
code didn't apparently work in GNU Fortran...).
similarly, I originally learned ASM from an 80s era book, which talked of
nothing newer than the 8088 and 8086.
but, alas, the book I had from the 80s gave no real mention of pointers...
I think it did give mention though of "record-based magnetic disk storage"
or somesuch...
> Over things like ? and ++ one could discuss if they are baroque remnant of
> ancients architectures or not, but lets skip that discussing by stating
> whatever your opinion on them is, they are by no means crucial for
> lowlevel
> programming.
>
++ is convinient, but yes, not crucial...
> C effectively has no string type, and its library emulation of one is 1:1
> convertable to Pascal. No difference there.
>
C doesn't need a string type...
ok, ok, granted for other reasons much of my compiler stuff internally uses
a string type, just it happens to nicely map to 'char *'...
actually, by default my compiler stuff also varies a little from what C is
"officially" supposed to do:
I don't really support locales and code-pages...
pretty much everything is UTF-8...
> As far as I know Pascal even has been slightly upper hand,
> performancewise,
> because it didn't have pointer aliasing rules to hinder the compiler
> (which
> afaik you can only can escape off in C since C99)
>
not sure what is meant here...
>> Basically, you're trapped to programming in a limited box that you can't
>> escape from.
>
> I wonder how this is possible
>
>> I'm not familiar with Modula-2 or Ada. I'm familiar with a
>> variant of PL/1 which was very Pascal-ish.
>
> Pascallish like C# is Cish? IOW in syntax or in language?
>
hmm... Lua is Pascal...
C# is C'ish, or at least a lot more than Java is, in both syntax and
semantics...
after all, C# has structs and pointers for those who want them...
>> - C requires all objects to be at different addresses.
>
> Union?
>
as noted, this is more "in theory" than in practice...
in theory, each address is a different object (and no too objects hold the
same address).
in practice, this is not strictly the case, as there are many special case
"pseudo-objects" which may show up at compile time which may not follow this
rule exactly, among many other cases which would neither be expected at the
C or ASM levels...
it can also be noted that I end up using SSE for all sorts of things, many
of which are not strictly supported by the processor (and a lot of code goes
into simulating a higher level of "orthogonality" than actually exists...).
(consider the ninjitsu magic needed to allow pointer-addressing to work via
a register that is actually just the upper-half of an SSE register, one may
soon notice that more than a few useful opcodes do not exist, and a good
amount of codegen contortion is needed...).
(I may end up deciding against general addressing via low/high XMM regs, as
the overhead is likely to be much worse than via GPRs, even in the face of
the high register pressure on x86...).
>> - C requires all C objects to map onto a contiquous sequence of C
>> characters.
>> - C implements an "offset operator" which indexes contiguous sequences of
>> C
>> characters.
>
> I don't get these, could you explain what you mean here, and what the
> relevance for lowlevel is? I think you mean that most constructs are
> either
> static structures or pointers to them.
>
I think the point is that, C bases its fundamental operating model on a
byte-mapped address space, which many other languages do not adhere to (as
such, C is much closer to the Von Neumann ideal than are many other
languages...).
>> - C doesn't implement arrays.
>> - C does implement an array declaration. That array declaration is
>> effectively converted into a pointer that can be used with the offset
>> operator in order to simulate arrays.
>
> While it saves maybe on the compiler in a 4k environment, I'm more
> interested in lowlevel TARGET than HOST. So while this may true, could you
> please explain why having a normal array in addition to a pointer is a bad
> thing?
>
I am not sure the intent of this point exactly...
for C99 at least one needs an actual array type (in addition to pointer
types), it just happens that arrays can be converted to pointers fairly
easily...
>> - C doesn't implement strings.
>> - C does simulate "strings", using an all bit-zero character to terminate
>> a
>> simulated "array"
>
> (a decision regretted by many security consultant and frustrated
> programmers)
>
it is not that bad IMO...
note, however, that FWIW one can add their own "string" type via library
code...
I typically do so, and typically regard strings as immutable...
I guess eliminating buffer overflow could require bounds-checking and
similar...
but, for whatever reason, many newer languages think it a good idea to waste
lots of memory with unicode strings (vs UTF-8 strings...).
>> - etc.
>>
>> In other words, you only need a contiquous sequence of C characters and
>> an
>> address, to implement the (non-specialized) features of C, including
>> "strings" and "arrays". Just due to the way strings are implemented in
>> Pascal and PL/1 etc., I'd say C fits better with the underlying computing
>> platform. However, C has numerous other low-level features that most
>> high-level languages lack. These too add to it's usefulness as an IL.
>
> You are on the wrong track here. First, there is no C string construct
> that
> I'm aware of that can't be matched in Pascal. There is a pointer to char,
> you can increment it, check for zero, subtract them, add them. Stronger
> even, Delphi/FPC come with a strings unit with the common null-terminated
> char* routines.
>
> IOW it is roughly like C++, you can simply do the pointer trick if you
> feel
> masochistic or (more rarely) see a need.
>
> Note even that the Delphi higher level string type is fully compatible (as
> in compiletime cast, no conversion whatsoever) with char *, as long as you
> don't try to reallocate it.
>
not much comment...
although, one can still debate if Delphi is really Pascal, or more like
Pascal++...
No, there are more, pretty much every non 16-bit versions does. And Delphi
is also 15 years old.
And x86 16-bits C's had similar limitations as e.g. TP.
> what I remember of Pascal at least, was from a compiler I was using in the
> late 90s, and with my reference material being books from the 80s...
Which compiler? Afaik the only other option is GNU Pascal and that supports
arithmetic too (wouldn't be surprised if it did since the eighties)
> (at the time, I had also partly learned Fortran77, but noted that F77 style
> code didn't apparently work in GNU Fortran...).
Point is, it is archaic. I might as wel make similar statements about
C because I dabbled a bit in C51 or a 8-bit microchip variant, and whine
about the fact that in PIC's C you can't have arrays larger than 256 bytes.
> I think it did give mention though of "record-based magnetic disk storage"
> or somesuch...
Speaking about archaic.....
>> C effectively has no string type, and its library emulation of one is 1:1
>> convertable to Pascal. No difference there.
>
> C doesn't need a string type...
Well, I don't agree. At least not anymore since the era that compiler's have
enough memory.
> C# is C'ish, or at least a lot more than Java is, in both syntax and
> semantics...
> after all, C# has structs and pointers for those who want them...
Its pointers are rudimentary) Its semantics are rather Delphi than C. No
wonder, since it was designed by Delphi's author. It is roughly Delphi
syntax with curly braces and a few C operators.
> in theory, each address is a different object (and no too objects hold the
> same address).
This goes nearly for any static compiled imperative language. Including
Pascal, Modula2, Ada etc. It is not really an unique C selling point.
> it can also be noted that I end up using SSE for all sorts of things, many
> of which are not strictly supported by the processor (and a lot of code goes
> into simulating a higher level of "orthogonality" than actually exists...).
The main use of SSE in not specialised code is to make numeric benchmarks go
faster (think shootout) and to improve the basic memory move.
>>> - C requires all C objects to map onto a contiquous sequence of C
>>> characters.
>>> - C implements an "offset operator" which indexes contiguous sequences of
>>> C
>>> characters.
>>
>> I don't get these, could you explain what you mean here, and what the
>> relevance for lowlevel is? I think you mean that most constructs are
>> either
>> static structures or pointers to them.
>>
>
> I think the point is that, C bases its fundamental operating model on a
> byte-mapped address space, which many other languages do not adhere to (as
> such, C is much closer to the Von Neumann ideal than are many other
> languages...).
All languages that I named do it in the same way. Stop reading eighties
books :-)
>> While it saves maybe on the compiler in a 4k environment, I'm more
>> interested in lowlevel TARGET than HOST. So while this may true, could you
>> please explain why having a normal array in addition to a pointer is a bad
>> thing?
>>
>
> I am not sure the intent of this point exactly...
> for C99 at least one needs an actual array type (in addition to pointer
> types), it just happens that arrays can be converted to pointers fairly
> easily...
Contrary to popular perception, a static array does not have a pointer in
the language sense. There is no room allocated for the precalculated object,
you cannot change it.
C is indeed a bit obscure about the difference between pointers and arrays,
but I never really considered it a feature. I always attributed that to a
result of some minor memory saving in the original compiler or to its funky
preprocessor architecture.
> note, however, that FWIW one can add their own "string" type via library
> code...
No you can't. A stringtype and emulating it are two different things. The
compiler can't check and enforce your string code.
> I guess eliminating buffer overflow could require bounds-checking and
> similar...
Hard to do on dynamically allocated types without explicit compiler support.
> but, for whatever reason, many newer languages think it a good idea to waste
> lots of memory with unicode strings (vs UTF-8 strings...).
That is a hard and emotional subject. I had a lot of opinions about that,
but the more I dove into it, the more confused one gets.
> although, one can still debate if Delphi is really Pascal, or more like
> Pascal++...
It's both, just like C++ also includes C.
Define rudimentary? You can do as much artihmetic with them as you want, you
can type-pun with them, and you can cast arbitrary integers to pointers..
yeah, far pointers, ...
far pointers weren't really limited, just awkward...
only now I forget some of the rules of far-pointer usage, but it doesn't so
much matter...
>> what I remember of Pascal at least, was from a compiler I was using in
>> the
>> late 90s, and with my reference material being books from the 80s...
>
> Which compiler? Afaik the only other option is GNU Pascal and that
> supports
> arithmetic too (wouldn't be surprised if it did since the eighties)
>
I remember I used FreePascal / FPK Pascal back then, and a few others, only
I don't remember which it was...
>> (at the time, I had also partly learned Fortran77, but noted that F77
>> style
>> code didn't apparently work in GNU Fortran...).
>
> Point is, it is archaic. I might as wel make similar statements about
> C because I dabbled a bit in C51 or a 8-bit microchip variant, and whine
> about the fact that in PIC's C you can't have arrays larger than 256
> bytes.
>
I think the issue is that the language GNU Fortran accepts is too much
newer, and so F77 is no longer accepted...
I guess, newer fortran's dispensed with the line numbers and putting
everything at particular indentations, and so F77 with its line numbers and
fixed-indentation was not accepted as such.
probably it does something different than jumping around based on line
number as well, but I have not cared enough about Fortran to really bother
looking (apart from, when I have run across it, newer Fortran code tends to
look a lot more Pascal'ish...).
>> I think it did give mention though of "record-based magnetic disk
>> storage"
>> or somesuch...
>
> Speaking about archaic.....
>
yep, I guess this is what higher-end computers were using at the time...
(PC's then, had floppy drives, and a FAT-style filesystem, or at least in
the late 80s...).
then again, I forget what part of the 80s the book was from, as I no longer
have the book I think...
some of my books are from the 60s...
yeah, some of these I ended up getting because they were library discards...
>>> C effectively has no string type, and its library emulation of one is
>>> 1:1
>>> convertable to Pascal. No difference there.
>>
>> C doesn't need a string type...
>
> Well, I don't agree. At least not anymore since the era that compiler's
> have
> enough memory.
>
but, it has 'char *', which can do strings fairly nicely (nevermind UTF-8
and wchar_t strings...).
in my compiler, I made wchar_t a builtin type (in most cases, aliased to
unsigned short, FWIW).
this is what is used in the (incomplete) Java and C# frontends for 'char'
(and where C's char maps to 'signed byte'...).
annoyingly at times, my signature system was originally designed relative to
the C typesystem, and the change to a more language neutral form (AKA:
integers types are based on size, ...) left some anomolies (such as, apart
from making the frontend mildly CPU-specific, I can either not support LP64
on x86-64, or force LP64 on x86, both of which would raise issue...).
this is because 'sizeof(long)' depends on arch, and previously the sig
handling code for 'long' had made it variable-sized, but now I have
designated it as being a fixed 64 bits, which otherwise makes a problem for
x86 (where much code on x86 may end up assuming a 32-bit long...).
of course, the new IL could help this, as this little piece of information
could be kept in the 'BMC' compiler, which could more safely be allowed to
know which arch it is targetting, and thus make relative adjustments.
however, this would mean that on x86:
void foo(int x);
and:
void foo(long x);
would have the same signature, which 'could' pose a problem if this
distinction is used when overloading functions in the C++ frontend (granted,
probably no one would do this anyways...).
this will not matter for C# and Java, which define long as always being 64
bits.
FWIW:
I may use a "modified" typesystem in BMC, namely it will still accept C's
typenames, but may also accept another set of builtin type names:
__int8
__uint8
__int16
__uint16
__int32
__uint32
__int64
__uint64
__int128
__uint128
...
__float16
__float32
__float64
__float80
__float128
...
as well as misc:
__char8
__char16
the reason then would be to reduce ambiguity, for example:
the C frontend may emit its long as 'long';
the C# frontend may emit its long as '__int64'.
...
the reason being that, when the input is comming from multiple possible
languages, it may get confusing as to which exact languages' conventions are
being followed in the IL (the IL is C based, but need not be C-specific,
although being "in keeping with C" is helpful, as otherwise I can be almost
certain I will only ever be someone to have an implementation of it...).
>> C# is C'ish, or at least a lot more than Java is, in both syntax and
>> semantics...
>> after all, C# has structs and pointers for those who want them...
>
> Its pointers are rudimentary) Its semantics are rather Delphi than C. No
> wonder, since it was designed by Delphi's author. It is roughly Delphi
> syntax with curly braces and a few C operators.
>
odd, I had thought C# had grabbed C's pointer system as-is, but granted I
had not looked at this aspect much...
in my implementation though, since the parser and a lot of other things are
shared between languages, I will probably support full C-style pointers (as
is, my implementation is 'unsafe' by default, so the unsafe keyword is
presently sort of a no-op...).
apart from a few minor differences, the object system seemed about like the
one from Java.
...
so, you are saying, they are close enough so one could probably just do an
alternative parser and compile it along with these other C-family
languages?...
maybe interesting, but I don't know as much if this is something I would
probably do (enough other things to do, FWIW...).
>> in theory, each address is a different object (and no too objects hold
>> the
>> same address).
>
> This goes nearly for any static compiled imperative language. Including
> Pascal, Modula2, Ada etc. It is not really an unique C selling point.
>
yes, ok.
>> it can also be noted that I end up using SSE for all sorts of things,
>> many
>> of which are not strictly supported by the processor (and a lot of code
>> goes
>> into simulating a higher level of "orthogonality" than actually
>> exists...).
>
> The main use of SSE in not specialised code is to make numeric benchmarks
> go
> faster (think shootout) and to improve the basic memory move.
>
yep...
I use them for memory copying as well...
but they are used for many other tasks:
floating point;
vectors and quats;
128 bit integers and floats;
128 bit pointers;
'long long' (x86);
...
recently, I have added SSE half-register support, which could be useful
mainly for long-long (x86), and possible register spillover (x86 and
x86-64...).
in the long-long case, it would likely be primarily beneficial as LL will
only take about 1/2 the register space (presently, LL uses an entire SSE
register). I am a little less certain though, as certain LL operations may
be made less efficient (since the LL may be in the upper-half of the reg,
and otherwise has to "cooperate" with whatever other value it may be sharing
the register with, meaning I can't just do full-register operations in these
cases...).
I had considered the prospect of using them for pointers (basically, so
certain memory-based types could refrain from using up the precious GPRs on
x86), but then realize the instruction sequences for doing so may not be
cheap...
it may or may not be justified vs, say, the relative cost of performing a
matmult, but I am not sure...
the alternative is, of course, to just pass the pointer to the matrix/... in
a GPR.
this could mean, however, performing matrix operations in a slightly less
efficient 2-address form, mostly because 3-address matrix ops with the
pointers held in GPRs would very likely exceed the number of available GPRs
on x86... (there are 5 usable GPRs, 3 of which would be needed for the
pointers, and 2 or more of which would likely be needed in the "compound
op").
using SSE regs for the pointers would take 1.5 SSE regs, and would not as
likely pos a problem (I can be more flexible with how many GPRs are used
within the operation), and if I use the "magic operation" trick (calling an
ASM thunk to do the operation), this overhead is negligable ('movhps mem,
xmm' and 'mov mem, reg' are not much different in terms of clocks...).
note, this only really applies to mat3 and mat4, as mat2 (like vec4 and
quat), can be passed (and manipulated) entirely in SSE registers.
>>>> - C requires all C objects to map onto a contiquous sequence of C
>>>> characters.
>>>> - C implements an "offset operator" which indexes contiguous sequences
>>>> of
>>>> C
>>>> characters.
>>>
>>> I don't get these, could you explain what you mean here, and what the
>>> relevance for lowlevel is? I think you mean that most constructs are
>>> either
>>> static structures or pointers to them.
>>>
>>
>> I think the point is that, C bases its fundamental operating model on a
>> byte-mapped address space, which many other languages do not adhere to
>> (as
>> such, C is much closer to the Von Neumann ideal than are many other
>> languages...).
>
> All languages that I named do it in the same way. Stop reading eighties
> books :-)
>
ok.
>>> While it saves maybe on the compiler in a 4k environment, I'm more
>>> interested in lowlevel TARGET than HOST. So while this may true, could
>>> you
>>> please explain why having a normal array in addition to a pointer is a
>>> bad
>>> thing?
>>>
>>
>> I am not sure the intent of this point exactly...
>> for C99 at least one needs an actual array type (in addition to pointer
>> types), it just happens that arrays can be converted to pointers fairly
>> easily...
>
> Contrary to popular perception, a static array does not have a pointer in
> the language sense. There is no room allocated for the precalculated
> object,
> you cannot change it.
>
> C is indeed a bit obscure about the difference between pointers and
> arrays,
> but I never really considered it a feature. I always attributed that to a
> result of some minor memory saving in the original compiler or to its
> funky
> preprocessor architecture.
>
I don't know the reason.
in my case, pointers and arrays "look about the same", but are handled
differently internally.
ammusingly, my low-level codegen (or, more correctly, the codegen in the
process of being rewritten) now supports pass-by-value usage of arrays,
although C is not so likely to use this.
in C, it would be if you could be like:
float a[16], b[16];
...
put something in b...
...
a=b;
I might make my IL also support this...
>> note, however, that FWIW one can add their own "string" type via library
>> code...
>
> No you can't. A stringtype and emulating it are two different things. The
> compiler can't check and enforce your string code.
>
yeah, maybe, only the runtime does it at runtime...
well, I could very well include builtin "managed strings" in the new IL.
it can be noted that due to technical issues, my strings will not be
strictly in conformance with the JVM's definitions, primarily in that my
framework's strings are not instances of "java.lang.String" (AFAIK, JVM
implementations tend to use internal loading trickery to make this work...).
in my case, I am likely to do something else: likely, this class will wrap
strings, rather than be strings.
that, or strings can be secretly marshelled (my Java support has not
actually gotten that far yet, as I am still beating with the endless
intricacies of getting all this various stuff to work together in my
framework...).
>> I guess eliminating buffer overflow could require bounds-checking and
>> similar...
>
> Hard to do on dynamically allocated types without explicit compiler
> support.
>
yes, but it is worth noting that I also have .NET style "jagged" and
"square" arrays, which do include bounds checking...
of course, like objects, full IL and compiler support for them is
incomplete...
I have yet to decide on whether or not to expose them in the C frontend (as
a compiler extension):
int foo[64]; //C-style array
int[64] foo; //managed array
even then, some trickery in the compiler could still pull off (limited)
bounds checking even without dynamically allocating the arrays. this would
be limited in that things like passing an array as a pointer would likely
break such support, apart from far more expensive trickery, namely doing a
pointer-based lookup to find the array definition in order to do the bounds
check (worse for stack-based arrays, which would require back-tracking and
having the compiler keep track of metadata for things like stack layout, ...
that or me implementing proper debug info...), ...
>> but, for whatever reason, many newer languages think it a good idea to
>> waste
>> lots of memory with unicode strings (vs UTF-8 strings...).
>
> That is a hard and emotional subject. I had a lot of opinions about that,
> but the more I dove into it, the more confused one gets.
>
I guess the big issue is whether one wants to regard strings more as atoms,
streams, or as arrays.
I tend to regard strings as either atoms or streams, and so UTF-8 makes the
most sense (after all, IME ASCII is the most common character set, even with
lots of text originating nowhere near the US...).
I guess if one wants to regard a string as an array, then UTF-8 is awkward
since characters may be different sizes, and so it is felt the
closer-to-uniform indexing is justified (since indexing by char in a UTF-8
string requires scanning through the string).
however, it is not difficult to support both varieties, and this is actually
what I do at present...
>> although, one can still debate if Delphi is really Pascal, or more like
>> Pascal++...
>
> It's both, just like C++ also includes C.
yes, but C is not C++...
as I see it, Pascal is the older definition, which may include all the
stuff, and varieties, generally accepted to exist to begin with.
Delphi and newer Delphi-alikes may be a superset of Pascal, and far more
normalized than the rest of Pascal land, however, there is much in Pascal
land which is not Delphi, such as Ada, many older varieties, ...
it is like, how C++ and Objective-C are both OO-based C supersets, but each
is a very different beast from the other...
the C camp has actually put much emphasis on standardization, such that
nearly any compiler can be written "to the standards", and have code
generally work for it (source works between compilers, often binary code
will link between compilers, ...). one is safe so long as they don't rely to
heavily on compiler extensions...
Pascal has much weaker standards, and so people resort to an alternative
means:
each implementation clones other popular implementations...
this would be much the same as if, in C land, as opposed to people writing
compilers following the standards, everyone just started using gcc and MSVC
as their reference implementations...
From what I read you can't even make a linked list with them. From what I
saw some things were also only possible in unsafe context, but I don't know
how common those are.
Sure you can, but you'd have to import malloc since it's not normally
available
And 16-bit structure and array limitations.
>>> what I remember of Pascal at least, was from a compiler I was using in
>>> the late 90s, and with my reference material being books from the 80s...
>>
>> Which compiler? Afaik the only other option is GNU Pascal and that
>> supports
>> arithmetic too (wouldn't be surprised if it did since the eighties)
>>
>
> I remember I used FreePascal / FPK Pascal back then, and a few others, only
> I don't remember which it was...
I can't remember that this didn't work, and my memory goes back to about
summer 1997, with some shorter dalliances before. I'll ask Florian
> I think the issue is that the language GNU Fortran accepts is too much
> newer, and so F77 is no longer accepted...
We emulated some 16-bitisms for TPs sake in a separate dialect mode. Fun
part was that most of the real code ran faster in FPC's 32-bit with a bit of
realmode emulation than in 16-bit mode:-)
>> Speaking about archaic.....
>
> yep, I guess this is what higher-end computers were using at the time...
>
> (PC's then, had floppy drives, and a FAT-style filesystem, or at least in
> the late 80s...).
Some of mine still do. Floppy and FAT that is. The floppy is only obsolete
after Windows XP. (XP still requires it as only option to postload drivers
for storage systems, short of slipstreaming them in)
>>> C doesn't need a string type...
>>
>> Well, I don't agree. At least not anymore since the era that compiler's
>> have enough memory.
>>
>
> but, it has 'char *', which can do strings fairly nicely (nevermind UTF-8
> and wchar_t strings...).
I never saw "char *" as nice. Even the fact that it could do as lowest
common denomitor type in low level routines is offset by the disasterous
choice to make it nul terminated.
> in my compiler, I made wchar_t a builtin type (in most cases, aliased to
> unsigned short, FWIW).
In Delphi/FPC it is too, widechar. Delphi already made it default
(D2009), FPC is catching up (but that will probably take another year.
Parttime slowliness :_)
> the C typesystem, and the change to a more language neutral form (AKA:
> integers types are based on size, ...)
> left some anomolies (such as, apart
> from making the frontend mildly CPU-specific, I can either not support LP64
> on x86-64, or force LP64 on x86, both of which would raise issue...).
Brr. That's bad. You need ILP32 on x86 and both LP64 and LLP64 on x86_64.
Maybe even ILP64 for AIX iirc. For non x86(_64) you most definitely need
ILP64.
> this is because 'sizeof(long)' depends on arch, and previously the sig
> handling code for 'long' had made it variable-sized, but now I have
> designated it as being a fixed 64 bits, which otherwise makes a problem for
> x86 (where much code on x86 may end up assuming a 32-bit long...).
I typically do not map headers to base types. IOW I map per header the used
types to the basetypes, sometimes OS dependant.
>> wonder, since it was designed by Delphi's author. It is roughly Delphi
>> syntax with curly braces and a few C operators.
>
> odd, I had thought C# had grabbed C's pointer system as-is, but granted I
> had not looked at this aspect much...
Afaik you can't even make a linked list. And for other things, while
supported, you have to manually lock types in the VM, which means that if
you change an existing method to use pointers, it can turn "unsafe", and
then potentially "safe" callers have to mark objects that they send as locked.
An implementation detail that changes the calling conventions is evil IMHO.
Having very limited pointers is a fact of life in a VM language though, and
not really a problem. But it makes drawing analogies from C# to C a bit
moot. One could only confuse C#'s pointers with C pointers if one is very
confused.
> so, you are saying, they are close enough so one could probably just do an
> alternative parser and compile it along with these other C-family
> languages?...
No idea. I just reacted on
1) vague claims about Pascal's pointer support. There is truth in there, but
only in versions dead for twenty years, give or take a leftover mainframe or
two.
2) analogies between C# and C with respect to pointers, which are IMHO
false.
If your IL can do C, you can do Pascal pretty much. The hurdle will be
more the module/unit concept to combine pieces of codes to mainprograms
(contrary to C having inclusion as single instrument)
> I use them for memory copying as well...
>
> but they are used for many other tasks:
> floating point;
> vectors and quats;
> 128 bit integers and floats;
> 128 bit pointers;
> 'long long' (x86);
> ...
Yup. But not blind and general purpose. Usually in specialized code. Because
if you run the average numerical code over SSE2 instead of the FPU, you'll
get into trouble wrt precision and IEEE compliancy (exception handling)
> recently, I have added SSE half-register support, which could be useful
> mainly for long-long (x86), and possible register spillover (x86 and
> x86-64...).
Doing such things might bring you into trouble with the systems ABI, and
slow down due to the ABI requiring preservation of those registers. Make
sure you can turn it on and off (or maybe only for leaf-routines)
> in the long-long case, it would likely be primarily beneficial as LL will
> only take about 1/2 the register space (presently, LL uses an entire SSE
> register). I am a little less certain though, as certain LL operations may
> be made less efficient (since the LL may be in the upper-half of the reg,
> and otherwise has to "cooperate" with whatever other value it may be sharing
> the register with, meaning I can't just do full-register operations in these
> cases...).
On x86 how many algorithms are dependant on 64-bit integer support? Only
AES I guess?
> I guess if one wants to regard a string as an array, then UTF-8 is awkward
> since characters may be different sizes, and so it is felt the
> closer-to-uniform indexing is justified (since indexing by char in a UTF-8
> string requires scanning through the string).
UTF-16 has surrogate pairs too. I never really found out if those are used a
lot. Some say the Chinese pages above the BMP are in active use, and some
not.
>> It's both, just like C++ also includes C.
>
> yes, but C is not C++...
That's a matter of definition.
> as I see it, Pascal is the older definition, which may include all the
> stuff, and varieties, generally accepted to exist to begin with.
> Delphi and newer Delphi-alikes may be a superset of Pascal, and far more
> normalized than the rest of Pascal land, however, there is much in Pascal
> land which is not Delphi, such as Ada, many older varieties, ...
Ada is not Pascal, and never was. And nearly all varieties are dead.
> it is like, how C++ and Objective-C are both OO-based C supersets, but each
> is a very different beast from the other...
I think if C had been dead, people would routine refer to procedural parts
of C++ as C (and in fact they do now)
> the C camp has actually put much emphasis on standardization, such that
> nearly any compiler can be written "to the standards", and have code
> generally work for it (source works between compilers, often binary code
> will link between compilers, ...). one is safe so long as they don't rely to
> heavily on compiler extensions...
That has been the case with Pascal too. The original subset is mostly
supported by all, though slightly less workable than C's. However the
Borland ones branched off in the early eighties. I work with it since 1990,
and can't remember any other way.
> Pascal has much weaker standards, and so people resort to an alternative
> means: each implementation clones other popular implementations...
That's not true. That is a picture C people like to paint to boost their own
prejudices and superiority feelings.
Fact is that since the early nineties, Pascal is way more homogenous around
the dominant Borland versions and clones than modern C ever was. If you
forget about the odd mainframe pascal user.
While C has a bit standarization, the standarized part is so slow, that any
non trivial app is non-portable. That is pretty normal for standards (the
ISO Pascal standards are the same), but it leads to a false sense of
portability.
> this would be much the same as if, in C land, as opposed to people writing
> compilers following the standards, everyone just started using gcc and MSVC
> as their reference implementations...
Which partially happens. But it is very difficult to compare these matters
black and white.
I'm not sure if I'll ever support non-integer based numeric formats in my
slow-going, in-progress, C compiler. If I do, it'll likely be x87
coprocessor instructions. I don't believe I need to go beyond that do
implement what I need from a C compiler.
> by having good integration with 'cdecl' and 'SysV' at the ABI level, as
well
> as good C integration and acceptable C++ integration, one effectively has
> interoperability with a good portion of "everything"...
Thanks. That's nice to know.
> nearly anything else can be patched up with "bridges"...
... You're the expert here. :-)
> my main reason for deprecating typedef would be to make the parser faster
> and allow for context-independent parsing.
How to you intend to catch code with typedefs? It'd be much like catching
code with floats, with an IL that didn't support them. I've developed code
to determine the location of typedef's. It adds some code to the
parser/lexer combination. I say it's a FSM, but it's not exactly. I speed
it up by using a hash function to eliminate string comparisons, but now I've
got a 32-bit hash values which I need to be 16-bit. And, I've got gigantic
switches, which I need to be much smaller so they are ANSI C compatible.
> as-is, to parse C damn near every name token ends up having to be checked
to
> see if it might be a type, whereas eliminating typedef could largely
> eliminate this check.
The nice feature of the "typedef engine" is that it can tell me whether a
token is a typedef declaration, a typdef name, operator, keyword,
identifier. I'm still working on it.
> as a slight cost (of no typedefs), one would end up seeing:
> 'struct <whatwhatever>' and 'union <whatever>' all over the damn place,
but
> this may be acceptable...
Um, why? A you implementing a de-typedef preprocessing front end?...
You're preserving the original code, right? You're just compiling it...
Rod Pemberton
I don't recall the 6510 (6502 variant) having floats... Even for x86 based
PC's, there were numerous x86 processors, up into the 486 series, that
didn't have math coprocessors (e.g., 386SX, 486SX). So, float wasn't a
fundamental type, at least for PC's, until mid '90's.
Rod Pemberton
I don't understand this claim. On modern systems, everything is convertible
into sequences of bytes. Integers, strings, characters, multi-byte
characters are usually directly convertible. Floats and doubles are host
specific, but generally must be converted from the cpu's native bit-encoding
into an equivalent byte sequence, to pass as a parameter to a function. So,
how is it that you can't create a compatible argument type to pass to an
external function by using bytes?
Rod Pemberton
there were ways around this...
not like 64k issues are such a big deal when one only has maybe a few
hundred kB available...
>>>> what I remember of Pascal at least, was from a compiler I was using in
>>>> the late 90s, and with my reference material being books from the
>>>> 80s...
>>>
>>> Which compiler? Afaik the only other option is GNU Pascal and that
>>> supports
>>> arithmetic too (wouldn't be surprised if it did since the eighties)
>>>
>>
>> I remember I used FreePascal / FPK Pascal back then, and a few others,
>> only
>> I don't remember which it was...
>
> I can't remember that this didn't work, and my memory goes back to about
> summer 1997, with some shorter dalliances before. I'll ask Florian
>
yeah...
well, my memory may be wrong as well, it was long ago...
>> I think the issue is that the language GNU Fortran accepts is too much
>> newer, and so F77 is no longer accepted...
>
> We emulated some 16-bitisms for TPs sake in a separate dialect mode. Fun
> part was that most of the real code ran faster in FPC's 32-bit with a bit
> of
> realmode emulation than in 16-bit mode:-)
>
interesting...
>>> Speaking about archaic.....
>>
>> yep, I guess this is what higher-end computers were using at the time...
>>
>> (PC's then, had floppy drives, and a FAT-style filesystem, or at least in
>> the late 80s...).
>
> Some of mine still do. Floppy and FAT that is. The floppy is only obsolete
> after Windows XP. (XP still requires it as only option to postload drivers
> for storage systems, short of slipstreaming them in)
>
yep...
I used to really like floppies because I could boot to them, so they were
useful for OS dev and testing (back when I did this...).
now, we have USB, and computers can generally boot to USB instead (nevermind
that likely one will need to likely retain a real-mode interface to the BIOS
to do so...).
but, alas, I don't do OS-dev anymore (not since I came to the realization
that, in the face of Windows and Linux, custom OS dev is kind of
pointless...).
>>>> C doesn't need a string type...
>>>
>>> Well, I don't agree. At least not anymore since the era that compiler's
>>> have enough memory.
>>>
>>
>> but, it has 'char *', which can do strings fairly nicely (nevermind UTF-8
>> and wchar_t strings...).
>
> I never saw "char *" as nice. Even the fact that it could do as lowest
> common denomitor type in low level routines is offset by the disasterous
> choice to make it nul terminated.
>
in practice the terminator is rarely a problem...
granted, with a little care, it can be handled in UTF-8 (embedded overlong
nul's...).
>> in my compiler, I made wchar_t a builtin type (in most cases, aliased to
>> unsigned short, FWIW).
>
> In Delphi/FPC it is too, widechar. Delphi already made it default
> (D2009), FPC is catching up (but that will probably take another year.
> Parttime slowliness :_)
>
can't be "the default" as this would break C compatibility.
it is used mostly with non-C frontends.
from what I remember, it was "inherited" from the IA-64 name-mangling
scheme, which was the template for my original signature system. the JVM was
a later influence...
>> the C typesystem, and the change to a more language neutral form (AKA:
>> integers types are based on size, ...)
>
>> left some anomolies (such as, apart
>> from making the frontend mildly CPU-specific, I can either not support
>> LP64
>> on x86-64, or force LP64 on x86, both of which would raise issue...).
>
> Brr. That's bad. You need ILP32 on x86 and both LP64 and LLP64 on x86_64.
> Maybe even ILP64 for AIX iirc. For non x86(_64) you most definitely need
> ILP64.
>
yep...
I am now thinking fixed-sized types may not be ideal for the upper-end IL.
it still needs some flexibility, and the frontend compiler needs to be able
to not really care what the arch is...
so, I will probably adopt a convention (for the upper IL):
byte/sbyte
short/ushort
int/uint
long/ulong
...
char/wchar: C-style char, wide-char
I may also drop the "__" everywhere, and instead make it so that any names
which may clash with a keyword are prefixed with '$' (like in C#...).
it is that or "__byte", "__sbyte", ...
>> this is because 'sizeof(long)' depends on arch, and previously the sig
>> handling code for 'long' had made it variable-sized, but now I have
>> designated it as being a fixed 64 bits, which otherwise makes a problem
>> for
>> x86 (where much code on x86 may end up assuming a 32-bit long...).
>
> I typically do not map headers to base types. IOW I map per header the
> used
> types to the basetypes, sometimes OS dependant.
>
yes, ok.
>>> wonder, since it was designed by Delphi's author. It is roughly Delphi
>>> syntax with curly braces and a few C operators.
>>
>> odd, I had thought C# had grabbed C's pointer system as-is, but granted I
>> had not looked at this aspect much...
>
> Afaik you can't even make a linked list. And for other things, while
> supported, you have to manually lock types in the VM, which means that if
> you change an existing method to use pointers, it can turn "unsafe", and
> then potentially "safe" callers have to mark objects that they send as
> locked.
>
> An implementation detail that changes the calling conventions is evil
> IMHO.
> Having very limited pointers is a fact of life in a VM language though,
> and
> not really a problem. But it makes drawing analogies from C# to C a bit
> moot. One could only confuse C#'s pointers with C pointers if one is very
> confused.
>
maybe...
afaik, their power is unleashed mostly in "unsafe" sections...
>> so, you are saying, they are close enough so one could probably just do
>> an
>> alternative parser and compile it along with these other C-family
>> languages?...
>
> No idea. I just reacted on
> 1) vague claims about Pascal's pointer support. There is truth in there,
> but
> only in versions dead for twenty years, give or take a leftover mainframe
> or
> two.
> 2) analogies between C# and C with respect to pointers, which are IMHO
> false.
>
> If your IL can do C, you can do Pascal pretty much. The hurdle will be
> more the module/unit concept to combine pieces of codes to mainprograms
> (contrary to C having inclusion as single instrument)
>
I need something very similar to this to get C# working, so the mechanisms
are already largely in place...
if modules can be mapped more-or-less to C# style namespaces, it should
work...
actually, it is my intent to handle Java packages with more or less the same
machinery...
>> I use them for memory copying as well...
>>
>> but they are used for many other tasks:
>> floating point;
>> vectors and quats;
>> 128 bit integers and floats;
>> 128 bit pointers;
>> 'long long' (x86);
>> ...
>
> Yup. But not blind and general purpose. Usually in specialized code.
> Because
> if you run the average numerical code over SSE2 instead of the FPU, you'll
> get into trouble wrt precision and IEEE compliancy (exception handling)
>
ok.
my FPU support is essentially broken at present.
if re-implemented, the FPU would likely be handled with trickery to make it
look like registers...
I hadn't really taken into account precision and compliance issues...
>> recently, I have added SSE half-register support, which could be useful
>> mainly for long-long (x86), and possible register spillover (x86 and
>> x86-64...).
>
> Doing such things might bring you into trouble with the systems ABI, and
> slow down due to the ABI requiring preservation of those registers. Make
> sure you can turn it on and off (or maybe only for leaf-routines)
>
not really, since these are not callee-preserve, the ABI need not care much
what I do with them...
(I never said I would be preserving them...).
mostly this is for internal operations, but when done everything typically
flattens back out into actual variables, and in cases where GRPs run out.
typically everything is re-synchronized on calls (regs flushed out to stack,
...).
>> in the long-long case, it would likely be primarily beneficial as LL will
>> only take about 1/2 the register space (presently, LL uses an entire SSE
>> register). I am a little less certain though, as certain LL operations
>> may
>> be made less efficient (since the LL may be in the upper-half of the reg,
>> and otherwise has to "cooperate" with whatever other value it may be
>> sharing
>> the register with, meaning I can't just do full-register operations in
>> these
>> cases...).
>
> On x86 how many algorithms are dependant on 64-bit integer support? Only
> AES I guess?
>
LL shows up occasionally...
luckily it is fairly rare...
>> I guess if one wants to regard a string as an array, then UTF-8 is
>> awkward
>> since characters may be different sizes, and so it is felt the
>> closer-to-uniform indexing is justified (since indexing by char in a
>> UTF-8
>> string requires scanning through the string).
>
> UTF-16 has surrogate pairs too. I never really found out if those are used
> a
> lot. Some say the Chinese pages above the BMP are in active use, and some
> not.
>
yes, but typically the surrogate pairs are assumed to be pairs of
codepoints...
>>> It's both, just like C++ also includes C.
>>
>> yes, but C is not C++...
>
> That's a matter of definition.
>
>> as I see it, Pascal is the older definition, which may include all the
>> stuff, and varieties, generally accepted to exist to begin with.
>
>> Delphi and newer Delphi-alikes may be a superset of Pascal, and far more
>> normalized than the rest of Pascal land, however, there is much in Pascal
>> land which is not Delphi, such as Ada, many older varieties, ...
>
> Ada is not Pascal, and never was. And nearly all varieties are dead.
>
Ada sure looks like Pascal though...
other stuff I had read had classified Ada as "a Pascal...".
>> it is like, how C++ and Objective-C are both OO-based C supersets, but
>> each
>> is a very different beast from the other...
>
> I think if C had been dead, people would routine refer to procedural parts
> of C++ as C (and in fact they do now)
>
I disagree...
C is a language in its own right...
>> the C camp has actually put much emphasis on standardization, such that
>> nearly any compiler can be written "to the standards", and have code
>> generally work for it (source works between compilers, often binary code
>> will link between compilers, ...). one is safe so long as they don't rely
>> to
>> heavily on compiler extensions...
>
> That has been the case with Pascal too. The original subset is mostly
> supported by all, though slightly less workable than C's. However the
> Borland ones branched off in the early eighties. I work with it since
> 1990,
> and can't remember any other way.
>
ok.
>> Pascal has much weaker standards, and so people resort to an alternative
>> means: each implementation clones other popular implementations...
>
> That's not true. That is a picture C people like to paint to boost their
> own
> prejudices and superiority feelings.
>
> Fact is that since the early nineties, Pascal is way more homogenous
> around
> the dominant Borland versions and clones than modern C ever was. If you
> forget about the odd mainframe pascal user.
>
> While C has a bit standarization, the standarized part is so slow, that
> any
> non trivial app is non-portable. That is pretty normal for standards (the
> ISO Pascal standards are the same), but it leads to a false sense of
> portability.
>
potentially the case...
sadly, most of the common de-facto C libraries (such as OpenGL, ...), are
not parts of "the standard", none the less, GL is standardized in its own
right.
>> this would be much the same as if, in C land, as opposed to people
>> writing
>> compilers following the standards, everyone just started using gcc and
>> MSVC
>> as their reference implementations...
>
> Which partially happens. But it is very difficult to compare these matters
> black and white.
ok.
FWIW, my effort is not really a gcc or MSVC clone...
I do my own extensions, and many gcc'isms are not supported in my case...
FWIW, core language exitensions are typically rare in practice, and most of
the variation is in the libraries...
thus, a compiler written to the standard can generally handle most code just
fine.
Some machines didn't have hardware floating point. But many did and on those
it would have been a basic type, and on the others it made sense to emulate
floating point where it was required.
The first machine I used (a pdp-10) did have a floating point unit, and
probably so did machines going back to the 50's. Then I came across the Z80
(and later 8088) that didn't have one, and for these I to needed to write
software floating point routines (although with 8051 I didn't bother...)
The point is the languages I created at the time had floating point
('reals') as a built-in type to the language. It wouldn't have made sense
(as I think someone suggested) to have simply removed them from the
language, or even from any IL, whenever the target hardware didn't have
native support for them.
Anyway we're in 2009 now...
--
Bart
because they are "unsafe" does not mean they can't be used, only that one
has to declare the section unsafe (and allow that in certain contexts the VM
be allowed to reject the code...).
OTOH, many similar restrictions could allow C code to be "verified", which
can't be done with full use of pointers, while other C code would be free to
use pointers/... as it pleases...
yep, and when targetting archs which didn't have floats, and people didn't
want to pay the cost of FP emulation, there were huge masses of elaborate
fixed-point code...
if floating point were unnecessary, I doubt people would have much need
fixed-point either...
(more recently, the C standard specifies an addon for built-in fixed point
types for archs which lack FPU, or for people I guess who want to use
it...).
> --
> Bart
>
this is a different issue...
bytes are fundamental to the machine.
a single integer type would likely break the ABI...
or, if you mean only having a single integer type during computations, but
still storing to specific-sized memory, this is already done internally
(well, there are 3 types: 32-bits, 64-bits, and 128-bits...), so types like
char/short/... are "promoted" on load, and "demoted" on store, and in some
cases with a little extra logic to keep them in range (char and short may
thus be a little slower for calculations, as internal logic may be in place
to make sure that calculations don't go outside the accepted ranges for
these types...).
but, in all the internal logic "type" gets a little hairy anyways...
it is not like I have isolated towers for the logic for each type, rather,
it is all big masses of code and "predicates" (many types are handled
generally via various predicates, ...).
so, for clarity:
a predicate is a function which will return true for values with some
certain property, but false for others...
for example, a 128 bit integer might be true for some predicates, but false
for others:
Integer -> T
Float -> F
Number -> T
SSE -> T
GPR -> F
128Bits -> T
64Bits -> F
32Bits -> F
SmallInt -> F (32 bits or less)
SmallLong -> F (64 bits or less)
Pointer -> F
Array -> F
...
many of these predicates are, in turn, depend on smaller and more
specialized predicates:
Basic -> T (type is a "basic" type, as in not
struct/union/array/pointer/...)
BasicLit -> T (type is basic or a basic literal)
...
so, most code need not care about the entire type system, only about certain
specific predicates.
for example, the register allocator does not care about what is the exact
type of the value (though the type is passed in and stored), but instead is
mostly concerned with predicates representing which type of register it
should allocate...
often, it is branched out then to more specific types handling as needed...
>
> Rod Pemberton
>
>
yeah...
well, it depends on uses...
I just found SSE stuff to be really convinient (vs x87 which is actually
fairly awkward...).
I may re-add x87 as a faked register machine though (though likely it would
only fake maybe 4 or 6 regs, as to do more would require a memory-backed
mapping...). sadly, being able to allocate FPU regs would actually make it
less convinient for random FPU-based calculations, which would then either
need to be reworked in terms of these regs, or require flushing these regs
to free up the FPU...
FWIW, doing math primarily via SSE seems to work fairly well...
>> by having good integration with 'cdecl' and 'SysV' at the ABI level, as
> well
>> as good C integration and acceptable C++ integration, one effectively has
>> interoperability with a good portion of "everything"...
>
> Thanks. That's nice to know.
>
yep...
partly is is because ABIs have largely normalized at the machine level, and
where most non-C languages still effectively interoperate with the outside
world via a C-based ABI...
C++ based ABIs are a little more rare, and typically a lot more complicated
(as well as the C++ ABI itself being fairly complicated).
at this level, I am partly doing my own ABI, and intend to interface with
C++ land more indirectly, namely through the use of code bridging and
wrapping...
for "unmanaged classes", I will try to have the same layout and behavior as
would a C++ class (I will probably be following gcc-style conventions
though).
>> nearly anything else can be patched up with "bridges"...
>
> ... You're the expert here. :-)
>
maybe...
>> my main reason for deprecating typedef would be to make the parser faster
>> and allow for context-independent parsing.
>
> How to you intend to catch code with typedefs? It'd be much like catching
> code with floats, with an IL that didn't support them. I've developed
> code
> to determine the location of typedef's. It adds some code to the
> parser/lexer combination. I say it's a FSM, but it's not exactly. I
> speed
> it up by using a hash function to eliminate string comparisons, but now
> I've
> got a 32-bit hash values which I need to be 16-bit. And, I've got
> gigantic
> switches, which I need to be much smaller so they are ANSI C compatible.
>
well, for now all I will probably do is be like "don't use them", or maybe
rig up a warning.
later, I would likely simply drop support for the mechanism from the parser
(code with typedefs would then fail to parse...).
the main thing of typedef is that primarily it is a mechanism for human
coders, but is not as much relevant to machine generated code.
>> as-is, to parse C damn near every name token ends up having to be checked
> to
>> see if it might be a type, whereas eliminating typedef could largely
>> eliminate this check.
>
> The nice feature of the "typedef engine" is that it can tell me whether a
> token is a typedef declaration, a typdef name, operator, keyword,
> identifier. I'm still working on it.
>
yeah, using a big hash to look up tokens is an idea, just thus far I have
not done so...
as is, my parsers tend to check tokens linearly...
my preprocessor is an exception, as my PP uses hashing to look up macros.
but, applying this to a parser would be more difficult, since one is largely
dealing with small collections of fixed-form logic...
to apply hashing to a parser would likely require breaking each construction
into its own function, and somehow still keeping track of lexical context,
...
it would likely be a lot more applicable to a syntax where there are large
numbers of keywords and a much less structured syntax... (such as something
more English-like...).
>> as a slight cost (of no typedefs), one would end up seeing:
>> 'struct <whatwhatever>' and 'union <whatever>' all over the damn place,
> but
>> this may be acceptable...
>
> Um, why? A you implementing a de-typedef preprocessing front end?...
> You're preserving the original code, right? You're just compiling it...
>
this is an IL, and the frontend does a bit more than removing typedefs (it
is a compiler in its own right...). the frontend actually parses several
different languages, and a C-style IL is just a "new" way to do the IL (it
will replace my former RPN-based IL at this level...).
the main thing though, is just that 'struct <whatever>' will appear all
throughout the IL code...
in RPNIL, the same thing would show up as 'X<whatever>;'...
a recent mystery is the IL's notation for handling line numbers.
the current leaning is to handle it like before, namely:
/*<linenum>*/ and /*"<filename>"<linenum>*/
but, I am left wondering if I should add a prefix:
/*L:<linenum>*/ and /*L:"<filename>"<linenum>*/
I guess also possible is to add a colon:
/*:<linenum>*/ and /*"<filename>":<linenum>*/
but, my thinking is that (like before), there is probably not much other
reason for comments of this sort to appear in the IL (never really was an
issue before, and user comments are generally not seen at this level
anyways...).
so:
/*418*/ i=(j*251)+(*s++); /*419*/ k=i;
means these statements were generated from whatever was located on lines
418-419 of whatever the last-named file was...
/*"smoke.c"420*/
is like the above, but indicates the filename as well...
(in RPNIL, this info is passed directly via opcodes, and in ASM it is back
to comments again...).
all the passed line numbers are mostly used for error and warning messages
at the various levels...
>
> Rod Pemberton
>
>
Well, it seems it may have been a holdover from the B language. The B
language terminated strings with a special character. The documents I've
read say it's the '*e' character or an ASCII EOT. They don't say whether
terminating strings was a language design choice or a constraint of the host
system. They also don't say whether this came from BCPL.
I thought I read in the C history papers that they debated over the
implementation of C strings. But, I can't seem to locate the quote...
cr88192 might be interested in the following article. It tells how an early
C compiler (PCC) works.
"A Tour Through the Portable C Compiler" by S.C. Johnson
http://wolfram.schneider.org/bsd/7thEdManVol2/porttour/porttour.html
Rod Pemberton
I think it was, like all things C, it worked first order, and it was keeping
the compiler cheap. (and by that I mean in needing the minimal amount of
memory to compile a large program, what I've read about its conception that
was a very important design parameter).
So the problem is that both that cheapness is outdated, but also they could
not have forseen the security problems that lead to the "n" variants.
There is also a worst case with longer strings for algorithms that need the
length early, because you need to traverse the string to get to the end,
possibly needing two passes instead of one. (e.g. palindrome check)
> I thought I read in the C history papers that they debated over the
> implementation of C strings. But, I can't seem to locate the quote...
If you find one, I'd be interested. I think it is also simply that the gap
in philosophy between C and C++ is so large.
well, hell, it works, and this is the important part...
I also remember that MS-DOS tended to terminate strings in many places with
'$', which was rather inconvinient if you might actually want to use this
character for something.
> cr88192 might be interested in the following article. It tells how an
> early
> C compiler (PCC) works.
>
> "A Tour Through the Portable C Compiler" by S.C. Johnson
> http://wolfram.schneider.org/bsd/7thEdManVol2/porttour/porttour.html
>
yes, ok.
actually, it almost makes me wonder of the merits of just shoving my
middle-IL compiler directly on top of my codegen, and forsaking the use (for
now) of a secondary lower-level IL.
actually, I could do so, while still keeping them as separate libraries, by
essentially throwing out (in this case) some of my usual modularity
requirements.
this would mean that the BMC compiler would have as its lower portion (or,
in place of its existing lower portion) code to directly interface with the
low-level codegen machinery.
modularity would suck, but at least it could probably "get the job" done
fairly well...
a slight awkwardness though is that my codegen typically uses 2 passes,
where the first pass is used to work out some of the details (such as how to
go about allocating registers, basically it is a dummy pass with output
disabled), and the second pass actually generates the code.
a tradeoff would be to make a compromise:
I don't use a full IL.
instead, I could make an API which basically uses function calls to produce
bytecoded objects, which are then used by the codegen and other things.
(don't know if plain TAC is better, or if I should require things to be in
SSA form?...).
forcing SSA form would avoid needing an extra pass to get BC into SSA form
(this had been planned for my IR/DKA-0 idea...).
actually, a bytecode construction API seems like a good idea, and would also
make implementing an IR/DKA parser less horrible (I did a partial parser,
and parsing directly from C-like expressions into bytecode is, very
ugly...).
the API would likely fairly thinly wrap the bytecode, but would internally
manage tasks like looking up symbols and literals, emitting opcodes, ...
which are otherwise needed for direct bytecode production.
it would also reduce the severity of the modularity violation.
otherwise, I would possibly end up having to make the entire BMC compiler
run in multiple passes, which would be, IMO, kind of stupid...
(FWIW, I suspect using only a single pass would create some difficulties for
things like register allocation, but could be hacked over to make it work,
at some likely performance cost...).
oh well, I am now considering no longer deprecating typedefs in BMC, as
typedef may well just be "too damn useful".
instead, I will likely only drop IL support for features which are of "less
certain utility" (in particular, enums, bitfields, ...). these will then be
left as the domain of the frontend (as are namespaces, nested
structures/classes, nested functions, ...).
it may be noted that my usual handling of enums is simply to replace them
with integers, and replace the enum value names with compile-time constants
(and, more so, I have found to be fairly rare in practice, as most C code
seems to go the "#define" route instead...).
>
> Rod Pemberton
>
>
Have you heard of the IL called Ten15? "Ten15 is an algebraically specified
abstract machine. ..."
http://en.wikipedia.org/wiki/Ten15
Apparently, it led to ANDF and TDF used in the TenDRA C compiler. But, it
says it has an issue with being an IL for C... Supposedly, TDF fixed the
issue.
This was interesting too:
http://en.wikipedia.org/wiki/Flex_machine
Rod Pemberton
went and looked at these.
although potentially interesting, these are not likely to be particularly
useful in my case.
the issue is mostly that my approach is much lower level:
the "actual" level of abstraction is at the level of ASM and the ABI's...
as such, the IL is itself a minor part of the project, more just a way of
more conviniently handling (and to some extent, normalizing) code-generation
tasks. I am also more or less limited to the C-style way of regarding issues
such as types and memory layout (doing types or data layout differently
would essentially break binary compatibility with existing C code).
an exception to the above is that there are some "managed" types, but these
represent an essentially non-orthogonal system (various parts of the system
are non-orthogonal).
>
> Rod Pemberton
>
>