I have a question about C standard which SAGE packages should conform
to. Is it C99? Plain ANSI C? "Modern ANSI C", i.e. ANSI C which
borrowed several features from C99?
I am working on Python<=>ALGLIB interface with intention to make it
usable from SAGE. It is Python interface to the pure C package,
created using CTYPES. I.e. C part of this interface is not C extension
to Python package, it is totally separate shared library which is used
through CTYPES foreign function library.
Python-C interface generator is ready, now I work in automatic
generation of C code. One week, I think, and it will be ready. But I
can't decide what C standard to choose.
As for me, I'd like to use ANSI C with several widely supported
features from C99 - inline functions, "//" comments, maybe something
else. But this thing should be compiled for SAGE, so I don't know can
I use something beyond ANSI C or not.
There is no information on this subject at the SAGE Developer Guide,
so I am asking this question here.
--
With best regards,
Sergey mailto:sergey.b...@alglib.net
> On 8 Jun., 19:25, Bill Hart <goodwillh...@googlemail.com> wrote:
>> Flint for example, uses C99. However, use of C99 has caused problems
>> for porting to Windows.
>
> Windows... ;-)
>
>
>> My recommendation would be to stick with ANSI C as much as
>> possible. I
>> doubt comments // will be a problem.
>
> Though many compilers (and preprocessors) support this, I would avoid
> them unless the code *is* C99 or C++.
Why?
>> The inline sematics are different
>> in c99 and in general cause a screwup across compiler versions.
>
> If using ANSI C, one should either use macros instead or at least use
> an INLINE which then expands to "inline" or nothing, depending on the
> compiler/platform.
>
>
>> I would say, in a .h file static inline is fine, otherwise eschew the
>> use of inline.
>>
>> Other problems for porting to windows include:
>>
>> 1) Variable declarations which are not at the start of a block
>> *including* in for loops, e.g. for (long i = 0; i < big; i++) is
>> invalid except in c99 (many c99 implementations don't even support
>> this).
>
> That's a really superfluous feature, and can be avoided (as well as
> any local variable declarations within the "middle" of a body).
I actually really like putting variable declarations as close as
possible to where they're used, but I guess that's a matter of style.
I'm all for standards, but personally I don't care much about the -
ansi -pedantic flags--I've got better things to worry about. Some
people do find this kind of thing important though, and they can spend
their time changing // to /* */.
>> 2) printf("%ld") - on windows this needs to sometimes be
>> printf("%lld")
>
> That's not only a problem on Windows; one should use the inttypes.h
> macros (C99) or equivalent ones.
> Note that "long long" is not ANSI C.
But we do use long long all the time in Sage, so you're safe there.
>> 3) mpz_set_ui instead of setting to a limb (a limb is not necessarily
>> an unsigned long on Windows)
>>
>> 4) pointer arithmetic (assuming a pointer fits into a long), use
>> intptr_t (oh wait, you can't, that's c99 only)
>>
>> Actually, best to not use C if you want it to compile on anything. C
>> is broken. (Partly joking).
But only partly :) I have the same opinion.
Back to your original question, you're talking about using/creating a
Python C interface generator. In this case, you may want to just
consider using Cython?
- Robert
>> Though many compilers (and preprocessors) support this, I would avoid
>> them unless the code *is* C99 or C++.
>
> Why?
There are several good reasons - well, at least in my opinion good reasons.
1) It encourages a culture of "I stick to standards most times, but I
vary in some occasions". That in general is what causes portability
issues.
I admit there is also the culture of "it works for me so it must be
ok", which is more serious.
2) Most compilers is not all compilers. Why stop someone using their compiler?
3) By avoiding such constructs, it is possible to user stricter
switches on compilers, so avoiding not only C++ style comments, but
also other dubious things which will be even less portable.
I would suggest that whatever method is used (personally given C99 is
an 11 year old standard, I think that should be ok), that the code is
also checked on the Sun Studio compiler, which can be downloaded for
both Linux and Solaris.
http://developers.sun.com/sunstudio/
That catches some serious errors which gcc misses and avoids
non-portable GNU-specific constructs, which are not portable to native
Unix compilers. The Unix native compilers in general give better
performance than gcc.
> I actually really like putting variable declarations as close as possible to
> where they're used, but I guess that's a matter of style.
Yes, it is a matter of style.
> I'm all for
> standards, but personally I don't care much about the -ansi -pedantic
> flags--I've got better things to worry about. Some people do find this kind
> of thing important though, and they can spend their time changing // to /*
> */.
I don't find it sufficiently important to spend my time changing // to
/* */, but I do feel things like -ansi and -pedantic are useful.
Although you feel you have got better things to worry about, it is
that attitude that means many years have to be spent porting code from
one platform to another.
>>> Actually, best to not use C if you want it to compile on anything. C
>>> is broken. (Partly joking).
>
> But only partly :) I have the same opinion.
>
C has stood the test of time far more than any other language. I
suspect people will be writing code in C, well after languages like
PHP, Python, ruby have all since been superseded by some other new
"hot" language. Many languages have in their time been the hot
language, but then fall into obscurity. I suspect Python will do too.
The fact Python 3 is not backwards compatible with python 2 has
already made that more likely.
> Back to your original question, you're talking about using/creating a Python
> C interface generator. In this case, you may want to just consider using
> Cython?
>
> - Robert
Why add in a step that generates non-portable code?
Dave
> On 8 June 2010 21:14, Robert Bradshaw <robe...@math.washington.edu>
> wrote:
>> On Jun 8, 2010, at 11:05 AM, leif wrote:
>
>>> Though many compilers (and preprocessors) support this, I would
>>> avoid
>>> them unless the code *is* C99 or C++.
>>
>> Why?
>
> There are several good reasons - well, at least in my opinion good
> reasons.
>
> 1) It encourages a culture of "I stick to standards most times, but I
> vary in some occasions". That in general is what causes portability
> issues.
>
> I admit there is also the culture of "it works for me so it must be
> ok", which is more serious.
Certainly the latter is more of a problem than the former.
> 2) Most compilers is not all compilers. Why stop someone using their
> compiler?
Do any modern compilers not understand one-line comments?
> 3) By avoiding such constructs, it is possible to user stricter
> switches on compilers, so avoiding not only C++ style comments, but
> also other dubious things which will be even less portable.
OK, fair enough.
> I would suggest that whatever method is used (personally given C99 is
> an 11 year old standard, I think that should be ok), that the code is
> also checked on the Sun Studio compiler, which can be downloaded for
> both Linux and Solaris.
>
> http://developers.sun.com/sunstudio/
>
> That catches some serious errors which gcc misses and avoids
> non-portable GNU-specific constructs, which are not portable to native
> Unix compilers. The Unix native compilers in general give better
> performance than gcc.
While this may have been true for the GCC 1.x and 2.x series, most
recent benchmarks I've ever seen are ambivalent at best, with
different people claiming different directions. Now I have seen backed
up claims that the (closed) Intel compilers are quite good, and Visual
Studio is blazing fast at C++.
Now, what really matters is how it works on stuff you use. If/when
Sage compiles with Sun Studio, then we could do some interesting
benchmarks.
I think C's longevity is due to its ubiquity and simplicity, but the
language itself has many fundamental flaws for most applications.
(Well, flaw here is a relative term--pointers and manual memory
management are ideal tools for writing drivers and other low level
code, but superfluous and error prone for many higher level
applications.)
Personally, I usually look at C as the new, portable assembly (though
JITs and VMs are competing in that space as well).
>> Back to your original question, you're talking about using/creating
>> a Python
>> C interface generator. In this case, you may want to just consider
>> using
>> Cython?
>>
>> - Robert
>
> Why add in a step that generates non-portable code?
He was asking about writing a Python/C interface generator, so I
suggested using an existing one.
On that note, please, please file bugs for any non-portable (or even
non-ANSI) code that Cython generates. Though I have a more lax
attitude about human written code, I think machine generated code can
and should be as close to standards compliant as possible. It may also
be interesting to set up a Sun Studio compile in the Cython build farm.
- Robert
I'm sure you could do this in Cython. It probably would be hard
(isn't everything worth doing?), but at least Cython is widely used,
debugged, tested, developed, etc.
William
>
>
> --
> With best regards,
> Sergey mailto:sergey.b...@alglib.net
>
> --
> To post to this group, send an email to sage-...@googlegroups.com
> To unsubscribe from this group, send an email to sage-devel+...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/sage-devel
> URL: http://www.sagemath.org
>
--
William Stein
Professor of Mathematics
University of Washington
http://wstein.org
> Hello,
>
> thanks to all who've replied to this discussion!
>
> Taking into account what was said above, I've decided to target
> pure
> ANSI C, i.e. C without newer constructs like // comments and
> other
> stuff from newer standards. I don't want to use C99 because it
> isn't
> supported by MSVC (de facto standard under Windows).
>
> I think that I'll make exception for static inlines because they can
> be easily turned on/off with just one #define. Another exception - I
> want to make use of SSE intrinsics (if they are provided by compiler),
> but slower ANSI C equivalent will be provided so one #define - and
> everything will be ANSI.
>
>
> On 9 июн, 00:14, Robert Bradshaw wrote:
>> Back to your original question, you're talking about using/creating a
>> Python C interface generator. In this case, you may want to just
>> consider using Cython?
>
> Ctypes gives me better control over situation. Arrays and records with
> complex fields (which are records/arrays too) are hard to represent in
> Cython, I think.
By records, are you talking about C structs? Cython can handle
(nested) structs and arrays just fine, and easier (in my opinion) than
ctypes can. Actually, I can't think of anything control that ctypes
gives that Cython doesn't (though I'd be glad to be corrected).
> Cython is a good general purpose tool which speeds up
> computations and may be used to generate wrappers. But I think that it
> is better to develop specialized solution for ALGLIB.
>
> Furhtermore, I want to pass multiple precision types between Python
> and ALGLIB - it is hard to implement efficiently with general purpose
> tools. I just have no other chouice :)
I'd be glad to know exactly what is special about writing ALGLIB
that's more difficult to do with using Cython. Either Cython would
work for you (in which case you're still free to not use it) or
that'll be good food for thought on something we need to make better.
- Robert
You wrote 10 июня 2010 г., 7:58:49:
> I'd be glad to know exactly what is special about writing ALGLIB
> that's more difficult to do with using Cython.
I can't say that it is difficult to do with Cython. I just want to say
that for me - and just for me - Cython has no benefits over ctypes.
Cython advantages are: a) improved performance of Python code, and b)
easier integration of external libraries. I don't need (a) because all
time-consuming operations are done in external library. As for (b),
the most important aspect of Cython is that it makes easier
_non-automated_, manual creation of interfaces. But all my interface
code is automatically generated :)
Ctypes, from the other side, is a part of standard library, so code
relying on Ctypes will be more portable. It also provides me with
direct access to shared library linking mechanism.
>> Ctypes gives me better control over situation. Arrays and records
>> with complex fields (which are records/arrays too) are hard to
>> represent in Cython, I think.
>
> By records, are you talking about C structs? Cython can handle
> (nested) structs and arrays just fine, and easier (in my opinion)
> than ctypes can. Actually, I can't think of anything control that
> ctypes gives that Cython doesn't (though I'd be glad to be
> corrected).
>
I am talking about C structs which contain dynamically allocated
arrays which should be freed when the structure is destroyed. I'll
explain it below in more details.
I develop Python<->ALGLIB interface keeping in mind one more goal - to
apply same interface generation technology to another programming
languages.
The idea is to create binary release of ALGLIB, which includes
assembly optimized implementations of algorithms, and to access its
functionality from multiple languages. So we will have one shared
library alglib.so (or alglib.dll) which can be used from several
programming environments - Python, .NET, maybe Java...
Furthermore, I want it to be efficient, especially for large problems
(it should be able to work with matrices up to several GBs). Such
ambitious goal requires me to develop very complex interface.
Several examples:
* all data structures are platform-oblivious (necessary for seamless
integration with C#; lesson I've learned from work on MPIR interface)
* all records have special constructor and destructor functions which
allocate and deallocate dynamic arrays associated with their fields.
* if calling environment provides me with matrix stored in contiguous
memory block, interface will try to reuse this memory as much as
possible
* all allocations/deallocations are tracked, i.e. upon returning from
ALGLIB function we can say whether input arrays were unchanged,
used to store result, or result was stored in another location.
* everything said above requires complex communication protocols to be
implemented.
It took most part of the May to implement, but now I can say that
interface part is almost over :) You can see that it goes far beyond
SAGE and Python, although future integration in SAGE is very important
for me.
And, with such complex interface, you can see that independently of
what I'll use - Ctypes or Cython - I'll have a lot of work to do :)
It would be great if you could check your code with the Sun Studio compiler,
which you can get for free for Linux or Solaris.
Sage builds on Solaris systems with SPARC processors. These will never have SSE
instructions, as the processor is completely different to Intel/AMD. You can
check if the system has a sparc processor by detecting if __sparc__ is defined.
In which case, you can immediately rule out there being SSE instructions of any
sort.
You wrote 10 июня 2010 г., 15:25:03:
> It would be great if you could check your code with the Sun Studio
> compiler, which you can get for free for Linux or Solaris.
It is not great, it is must-have. Building ALGLIB on every platform
where SAGE builds and with every popular compiler is top priority for
me.
> Sage builds on Solaris systems with SPARC processors. These will
> never have SSE instructions, as the processor is completely
> different to Intel/AMD. You can check if the system has a sparc
> processor by detecting if __sparc__ is defined.
I have no experience with SPARC, but I've studied different CPU
architectures before starting this project. "Generic C" build with all
optimizations turned off should work on SPARC and other non x86 ISA's.
BTW, what compilers support this __sparc__ symbol?
Having checked again, __sparc is preferable. That is supported by gcc, g++, cc
and CC (the Sun compilers)
kirkby@t2:[~] $ cat test.c
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char **argv) {
#ifdef __sparc
printf("I'm a SPARC\n");
#endif
}
First on a Solaris 10 machine based on a Sun SPARC processor.
kirkby@t2:[~] $ /opt/SUNWspro/bin/cc test.c
kirkby@t2:[~] $ ./a.out
I'm a SPARC
kirkby@t2:[~] $ /opt/SUNWspro/bin/CC test.c
kirkby@t2:[~] $ ./a.out
I'm a SPARC
kirkby@t2:[~] $ gcc test.c
kirkby@t2:[~] $ ./a.out
I'm a SPARC
kirkby@t2:[~] $ g++ test.c
kirkby@t2:[~] $ ./a.out
I'm a SPARC
So both the Sun and GNU compilers support __sparc.
On another Solaris machine, but this time based on an Intel Xeon.
drkirkby@hawk:~$ /opt/sunstudio12.1/bin/cc test.c
drkirkby@hawk:~$ ./a.out
drkirkby@hawk:~$ /opt/sunstudio12.1/bin/CC test.c
drkirkby@hawk:~$ ./a.out
drkirkby@hawk:~$ gcc test.c
drkirkby@hawk:~$ ./a.out
drkirkby@hawk:~$ g++ test.c
drkirkby@hawk:~$ ./a.out
drkirkby@hawk:~$ uname -a
SunOS hawk 5.11 snv_134 i86pc i386 i86pc
So I think __sparc will be ok on Solaris. It is defined on a SPARC system and it
is not defined on an Intel system.
One can run Linux on a SPARC too, but that is very very rare. I'm not aware of
any active development of any Linux distro for the SPARC processor.
I've also got machines running AIX, tru64, HP-UX and IRIX. None of the CPUs in
those sorts of machines would support SSE. Clearly there are many system which
will never support SSE, though Solaris SPARC is the most critical for Sage.
I doubt the Intel Itanium processors support SSE either, though I'm not 100%
sure of that.
Dave
> Having checked again, __sparc is preferable. That is supported by gcc, g++, cc
> and CC (the Sun compilers)
Thanks a lot!
> I've also got machines running AIX, tru64, HP-UX and IRIX. None of
> the CPUs in those sorts of machines would support SSE. Clearly there
> are many system which will never support SSE, though Solaris SPARC
> is the most critical for Sage.
Well, I am from x86 world and (currently) have no access to anything
beyond 32/64-bit Intel CPUs. I'll move to optimizations specific to
other ISA's when I'll get hands on machines with these CPU's :) But
now I have to optimize at least for x86.
> I doubt the Intel Itanium processors support SSE either, though I'm
> not 100% sure of that.
Itanium doesn't support SSE, you are right.
The UltraSPARC architecture does define a set of SIMD multimedia
instructions referred to as "VIS", though these only support integer
and fixed-point fractions.
On 2010-Jun-10 18:03:33 +0400, Sergey Bochkanov <sergey.b...@alglib.net> wrote:
>Itanium doesn't support SSE, you are right.
It may support an alternative SIMD instruction set.
--
Peter Jeremy