integer fields and native types
Ken Raeburn
issues on the project site, but they're getting more minor or more
subtle. (And while I've had trouble running Mono on multiple PowerPC
Mac systems, somehow running a2c.exe from v4 is working a lot better.)
There's an issue I haven't raised in the tracker because I'm not sure
how to approach fixing it, or what other people and projects would
want. Currently integer types can be "huge integers" (arbitrary
sized byte arrays), or "native integers" ("int", and the code assumes
this means 32-bit signed values). In Kerberos and related protocols,
we've got signed and unsigned 32-bit integral values on the wire (and
could someday easily have 64-bit ones), but we do want to be able to
build code on embedded systems where "int" might be only 16 bits.
Ideally I'd like to see generated code use types like uint32_t and
int64_t, though not every platform has to have them. ISO C 99
requires them in stdint.h *if* the platform has types of *exactly*
those sizes and if the signed types use twos-complement
representation. POSIX requires that {u,}int{8,16,32}_t all be
defined, and I would expect some future version to require the 64-bit
versions. Older systems may have some of them in sys/types.h, or
inttypes.h, or not at all.
We could include limits.h, and use INT_MAX, LONG_MAX etc to select
which of int/long/long long/__int64 are used, and define A2C_UINT32
and such types accordingly. This is what we do for 16- and 32-bit
types in our installed header for Kerberos; for GSSAPI, we generate a
system-specific header that includes certain header files we found to
be present, and others we assume always exist, and then uses uint32_t
unconditionally.
Letting the user provide the type names to use in the .asn input file
might also work. If we assume that any such provided types will
match a signed or unsigned version of char, short, int, long, long
long (if supported), or __int64 (on Windows), and that all types of
the same size and signedness are handled exactly the same, then we
could store the size and signedness in the descriptor, and have the
runtime code walk through the options. From a purist perspective,
it's not a great idea, because "int" and "long" shouldn't be treated
as the same even if they're the same size. If compilers get really
clever about alias analysis, that might bite us someday.
I'm not really crazy about any of these solutions, but nor do I
really want to treat all integer fields as "huge" integers. Anyone
have better ideas?
Blake Ramsdell
> I'm not really crazy about any of these solutions, but nor do I
> really want to treat all integer fields as "huge" integers. Anyone
> have better ideas?
My opinion is to only store integers as binary (A2C_INTEGER_HUGE), and
provide helper functions to get out the integer size you want, and
also potentially to measure whether an integer will overflow.
Something like:
A2C_ERROR A2C_INTEGER_To_Int(A2C_INTEGER_HUGE* p, int* pValue);
A2C_ERROR A2C_INTEGER_To_Long(A2C_INTEGER_HUGE* p, long* pValue);
And indicate an overflow condition in the returned A2C_ERROR.
But I think that generally the implementor has an idea of what sizes
make sense that are going to come out of an INTEGER, so maybe this
approach isn't a bad idea, and saves some creepy union hack or
equivalent.
Blake
--
Blake Ramsdell | http://www.blakeramsdell.com
Ken Raeburn
> My opinion is to only store integers as binary (A2C_INTEGER_HUGE), and
> provide helper functions to get out the integer size you want, and
> also potentially to measure whether an integer will overflow.
I like the "native integer" idea because it saves space and reduces
allocations, and having the range checking done for me by the ASN.1
code would be convenient. It's just that the sizes I want to deal
with have specific numbers of bits, not specific C types.
> Something like:
>
> A2C_ERROR A2C_INTEGER_To_Int(A2C_INTEGER_HUGE* p, int* pValue);
> A2C_ERROR A2C_INTEGER_To_Long(A2C_INTEGER_HUGE* p, long* pValue);
.. and unsigned versions, I assume. That'd work. Then for my
purposes I could always use the _Long or _ULong versions, and do
additional range checking myself as needed. It won't help much when
it comes to 64-bit support though, that I'll still have to manage
myself if and when the time comes.
Ken
Blake Ramsdell
> .. and unsigned versions, I assume. That'd work. Then for my
> purposes I could always use the _Long or _ULong versions, and do
> additional range checking myself as needed. It won't help much when
> it comes to 64-bit support though, that I'll still have to manage
> myself if and when the time comes.
A2C_ERROR _A2C_INTEGER_To_NativeInteger(A2C_INTEGER_HUGE* p, void*
pValue, size_t nValueSize);
#define A2C_INTEGER_To_NativeInteger(I, N)
_A2C_INTEGER_To_NativeInteger((I), &(N), sizeof(N))
Does that smell right? I confess I'm not good enough with platform
issues to know whether or not unaligned byte writes to offsets within
an integer are legal / a good idea.
It also doesn't do "the right thing" with signed-ed-ness (if you have
eight bits of BER INTEGER with the high bit set, and stick that in a
larger integer, it should sign extend).
Ken Raeburn
> On 10/14/07, Ken Raeburn <rae...@raeburn.org> wrote:
>> .. and unsigned versions, I assume. That'd work. Then for my
>> purposes I could always use the _Long or _ULong versions, and do
>> additional range checking myself as needed. It won't help much when
>> it comes to 64-bit support though, that I'll still have to manage
>> myself if and when the time comes.
>
> A2C_ERROR _A2C_INTEGER_To_NativeInteger(A2C_INTEGER_HUGE* p, void*
> pValue, size_t nValueSize);
>
> #define A2C_INTEGER_To_NativeInteger(I, N)
> _A2C_INTEGER_To_NativeInteger((I), &(N), sizeof(N))
>
> Does that smell right? I confess I'm not good enough with platform
> issues to know whether or not unaligned byte writes to offsets within
> an integer are legal / a good idea.
In general I think it's okay, but as I read about GCC's alias
analysis getting stricter, I find I'm less sure about some of the
subtleties of these things. I *think* byte access is okay, but other
types other than the "real" type of the object are not. (I'd want to
check on that.) One thing to watch for, the byte order in large
types isn't necessarily one of 123456... or ...654321, though maybe
only PDP-11 and one or two other platforms actually don't fit those
two categories.
> It also doesn't do "the right thing" with signed-ed-ness (if you have
> eight bits of BER INTEGER with the high bit set, and stick that in a
> larger integer, it should sign extend).
If you assume N is a simple lvalue and it's okay to evaluate it more
than once, ((N)=0, (N)--, (N)>0) will tell you whether it's signed or
unsigned. So, pass one more argument to the function....
(We'd still be assuming twos-complement with the sign bit at the top,
which I think ISO C doesn't guarantee us, but any system we're likely
to run into in practice will implement.)
Ken
epei...@gmail.com
decoding internally as huge and then converts to "int". (see runtime/C/
A2C_Integer.c) This means that decoding 02 05 00 ff ff ff ff - (which
cannot be represented as a signed 32 bit integer - but can as
unsigned) is actually properly decoded
and then correctly fails in conversion to signed int - which is what
a2c works in.
So - the HUGE allocation is already taking place... Perhaps better
access is required in specification of what comes out.
Maybe the "override functions" feature of A2C to provide the encoders/
decoders might work here - but I am unsure if this is really the most
efficient long term solution - but one could write that
A2C_UNSIGNED_INTEGER representation decoder which handles the edge
cases... Perhaps the "validate contents of fields issue" will also
play into this...
Ezra
Blake Ramsdell
> So - the HUGE allocation is already taking place... Perhaps better
> access is required in specification of what comes out.
> Maybe the "override functions" feature of A2C to provide the encoders/
> decoders might work here - but I am unsure if this is really the most
> efficient long term solution - but one could write that
> A2C_UNSIGNED_INTEGER representation decoder which handles the edge
> cases... Perhaps the "validate contents of fields issue" will also
> play into this...
Yeah, I'm personally leaning toward the "helper function to get the
integer type you want, with a status indication of overflow"
direction. The number of integer types makes me crabby (all of the
permutations of signed / unsigned and common lengths).
Ezra Peisach
On Oct 29, 8:17 am, "Blake Ramsdell" <bla...@gmail.com> wrote:
> Yeah, I'm personally leaning toward the "helper function to get the
> integer type you want, with a status indication of overflow"
> direction. The number of integer types makes me crabby (all of the
> permutations of signed / unsigned and common lengths).
>
It probably is not too bad. Consider that you need a generic function
for
huge integer to n-bytes for signed/unsigned. Then all "helper"
functions need simply call
this generic function w/ the size of the type they represent (passing
in a void * pointer)...
This would allow for future expansion.
I am not sure if the HUGE der/ber encoders will need modification for
unsigned representations - or if one could simply create the HUGE
integer w/ and extra leading byte with 00 if the high bit is set. I
know there a specific rules for high bits being set...
Ezra
jim...@nwlink.com
bits today (i.e if you start with 00 f0 it knows that this is a
positive number).
THe initial reasoning for having the native integer was simply due to
the fact that a large number of specifications have small integers in
them and this allows for more effiecent encoding by the user for this
specific types. Think version numbers on certificates or CMS and
similar items. Although I was thinking about potentially adding a
native long to the system, I did not really want to think about adding
all of the unsigned versions as well. One question is if these
actually occur offen enough to increase the encoder/decoder sizes to
to deal with them.
I would tend to think that generic approach that I would support would
be that of providing helper functions for short and unsigned versions
of things on the basis that they are not widely used today in ASN.1
specifications. I could be convinced otherwise but it would need some
work.
On the question of providing these for specific sizes, do you have
some type of internal compiler support on your 16-bit platform to do
64-bit integer arithmetic?
jim
Ken Raeburn
> One question is if these
> actually occur offen enough to increase the encoder/decoder sizes to
> to deal with them.
I don't know, I've mostly only played with the Kerberos-related
modules. If other ASN.1 modules tend to have only very small or
unbounded integer values, perhaps it's more appropriate to put the
Int32/UInt32 support in the glue code for Kerberos, and not worry
about it. Especially since, as Ezra pointed out, the native integer
support does the allocation of a huge integer anyways under the
covers, the only memory savings would be in not having multiple huge-
integer values represented simultaneously.
> On the question of providing these for specific sizes, do you have
> some type of internal compiler support on your 16-bit platform to do
> 64-bit integer arithmetic?
If the implementor wants to comply with C99, it would have to, as
"long long" is required to be supported and at least 64 bits. And
many pre-C99 compilers still seem to support "long long" as a 64-bit
type these days. But for small platforms with pre-C99 support, there
may indeed be no 64-bit arithmetic type.
GCC, for example, will break down 64-bit additions into 32-bit
additions if necessary, and on small machines like the AVR
microcontroller, the 32-bit operation may itself be composed of
multiple smaller operations.
Of course, if you don't have a 64-bit arithmetic C type, you can
synthesize the operations at the C level once you decide on a
representation for for the 64-bit values (struct with two 32-bit
values?), but it's kind of tedious. One reason I'd recommend against
it is that the compilers are moving in the direction of C99, so the
support will be there eventually, and we'd still have the awkward
workaround in place. And there's the question of whether the API
should vary between platforms that have 64-bit types and those that
don't, or if it should use the awkward form on all platforms. I
think I might opt for using real 64-bit types when they're available,
and not providing that support when they're not.
Ken