Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

STC - A "Standard Template Containers" library for C, similar to STL

315 views
Skip to first unread message

tylo

unread,
Mar 3, 2021, 10:16:22 AM3/3/21
to
If you are envying c++ for their container classes, I made a decent library that should cover you. It is unusual with "templated" classes in C, but these are simple in use, and will save you from casting / throwing away valuable type information - which again may obscure buggy code. The lib is also very fast.

https://github.com/tylov/STC

Ben Bacarisse

unread,
Mar 3, 2021, 4:36:19 PM3/3/21
to
That's an impressive piece of work.

--
Ben.

Chris M. Thomasson

unread,
Mar 3, 2021, 4:50:41 PM3/3/21
to
On 3/3/2021 7:16 AM, tylo wrote:
> If you are envying c++ for their container classes, I made a decent library that should cover you. It is unusual with "templated" classes in C, but these are simple in use, and will save you from casting / throwing away valuable type information - which again may obscure buggy code. The lib is also very fast.
>
> https://github.com/tylov/STC
>

Only for x86. Why not use C11 atomics?

Chris M. Thomasson

unread,
Mar 3, 2021, 4:56:38 PM3/3/21
to
Ahhhh, you are using atomic built-in's for gcc and intrinsic's for msvc.
But that limits you. If you can use C11, then go for it.

John Dill

unread,
Mar 3, 2021, 5:17:44 PM3/3/21
to
On Wednesday, March 3, 2021 at 10:16:22 AM UTC-5, tylo...@gmail.com wrote:
> If you are envying c++ for their container classes, I made a decent library that should cover you. It is unusual with "templated" classes in C, but these are simple in use, and will save you from casting / throwing away valuable type information - which again may obscure buggy code. The lib is also very fast.
>
> https://github.com/tylov/STC

Awesome, I love to see anyone working on generic containers in C.
Will give it a look see.

Best regards,
John D.

olcott

unread,
Mar 3, 2021, 5:21:11 PM3/3/21
to
Wouldn't adding generics to C require at least some preprocessing of the
source file?

--
Copyright 2021 Pete Olcott

"Great spirits have always encountered violent opposition from mediocre
minds." Einstein

Ben Bacarisse

unread,
Mar 3, 2021, 5:45:33 PM3/3/21
to
olcott <No...@NoWhere.com> writes:

> On 3/3/2021 4:17 PM, John Dill wrote:
>> On Wednesday, March 3, 2021 at 10:16:22 AM UTC-5, tylo...@gmail.com wrote:
>>> If you are envying c++ for their container classes, I made a decent
>>> library that should cover you. It is unusual with "templated"
>>> classes in C, but these are simple in use, and will save you from
>>> casting / throwing away valuable type information - which again may
>>> obscure buggy code. The lib is also very fast.
>>>
>>> https://github.com/tylov/STC
>>
>> Awesome, I love to see anyone working on generic containers in C.
>> Will give it a look see.
>
> Wouldn't adding generics to C require at least some preprocessing of
> the source file?

"Generics" are no really the same as "generic containers" (you may not
know that C already has _Generic). But anyway, the cited code makes
extensive use of preprocessing, so your requirement is met.

--
Ben.

John Dill

unread,
Mar 3, 2021, 6:05:15 PM3/3/21
to
On Wednesday, March 3, 2021 at 5:21:11 PM UTC-5, olcott wrote:
> On 3/3/2021 4:17 PM, John Dill wrote:
> > On Wednesday, March 3, 2021 at 10:16:22 AM UTC-5, tylo...@gmail.com wrote:
> >> If you are envying c++ for their container classes, I made a decent library that should cover you. It is unusual with "templated" classes in C, but these are simple in use, and will save you from casting / throwing away valuable type information - which again may obscure buggy code. The lib is also very fast.
> >>
> >> https://github.com/tylov/STC
> >
> > Awesome, I love to see anyone working on generic containers in C.
> > Will give it a look see.
> >
> > Best regards,
> > John D.
> >
> Wouldn't adding generics to C require at least some preprocessing of the
> source file?

Generics are not quite the same as what I mean by generic containers.

Generic containers can be implemented in two flavors. There are those
like SGLIB, the containers in the linux kernel, and now STC that employ a
macro based approach to basically instantiate an interface for a give type.
The main advantage is type safety in the use of the interface, meaning the
compiler helps you out when you make mistakes. They can be pretty fast
as well. Reading the code on the other hand is not particularly easy to
follow.

The alternate flavor is representing objects as a type-less blob of bytes,
and using casting when you want to operate on that object, typically
requiring storage of the element size in the container. The GLib and
CCL use this kind of approach, though CCL has some typed container
generation as well. The main advantage is readability and simplicity at
with an increased risk of type abuse. It's simpler to package this as
a library.

I've never been much of a fan of Generics in C, I still can't fathom why
they passed on the typeof feature of GCC when they standardized it.

Best regards,
John D.

olcott

unread,
Mar 3, 2021, 6:21:39 PM3/3/21
to
Not quite the same thing as C++ templates yet perhaps helpful for
implementing generic containers.

I created std::vector for Turbo "C" 1.5 to run on my HP 200LX handheld.
It only had the most basic functionality. Without having templates I
think that I had to use the macro pre-proccessor #define statements to
specify the underlying types.

olcott

unread,
Mar 3, 2021, 6:28:25 PM3/3/21
to
I avoided memory allocation throughout my whole career until std::vector
did it for me.

Chris M. Thomasson

unread,
Mar 3, 2021, 11:44:20 PM3/3/21
to
On 3/3/2021 7:16 AM, tylo wrote:
> If you are envying c++ for their container classes, I made a decent library that should cover you. It is unusual with "templated" classes in C, but these are simple in use, and will save you from casting / throwing away valuable type information - which again may obscure buggy code. The lib is also very fast.
>
> https://github.com/tylov/STC
>

Downloaded it, and successfully ran an example: complex.c

I get the following output:


arr_a: (6, 4)
value (5, 1) is: 3.141593
destroy 0
destroy 0
destroy 0
destroy 0
destroy 0
destroy 0
destroy 0
destroy 0
destroy 0
destroy 0
destroy 0
destroy 0
destroy 0
destroy 0
destroy 0
destroy 0
destroy 0
destroy 0
destroy 0
destroy 0
destroy 0
destroy 3.14159
destroy 0
destroy 0


Does that look right to you?

Chris M. Thomasson

unread,
Mar 3, 2021, 11:45:35 PM3/3/21
to
On 3/3/2021 8:44 PM, Chris M. Thomasson wrote:
> On 3/3/2021 7:16 AM, tylo wrote:
>> If you are envying c++ for their container classes, I made a decent
>> library that should cover you. It is unusual with "templated" classes
>> in C, but these are simple in use, and will save you from casting /
>> throwing away valuable type information - which again may obscure
>> buggy code. The lib is also very fast.
>>
>> https://github.com/tylov/STC
>>
>
> Downloaded it, and successfully ran an example: complex.c
[...]

Just ran demos.c and got:


one-two-three-seven-five.
one-two-three-four-five.
one-two-three-four-five *** one-two-three-four-five.
find "four": four-five *** one-two-three-four-five
append: one two three four five six seven eight

VECTORDEMO1
erase - 3: 1600
0: 400
1: 900
2: 2500
3: 3600
4: 4900
5: 6400
6: 8100

VECTORDEMO2
names[1]: Jane
sorted: Chris
sorted: Jane
sorted: Mary

LISTDEMO1
value: 0
value: 1
value: 2
value: 3
value: 4
value: 5
value: 6
value: 7
value: 8
value: 9
spliced: 100
spliced: 101
spliced: 102
spliced: 103
spliced: 104
spliced: 105
spliced: 106
spliced: 107
spliced: 108
spliced: 109
spliced: 0
spliced: 1
spliced: 2
spliced: 3
spliced: 4
spliced: 5
spliced: 6
spliced: 7
spliced: 8
spliced: 9
sorted: -99
sorted: 1
sorted: 2
sorted: 3
sorted: 4
sorted: 5
sorted: 6
sorted: 7
sorted: 8
sorted: 9
sorted: 101
sorted: 102
sorted: 103
sorted: 104
sorted: 106
sorted: 107
sorted: 108
sorted: 109
sorted: 1000

SETDEMO1
set: 8
set: 11

MAPDEMO1
val 8: 64

MAPDEMO2
long: Hello: 64
long: Groovy: 200
short: Hello: 64
short: Groovy: 200

MAPDEMO3
entry: Map: test
entry: Make: my
entry: Sunny: day
size 3: remove: Make: my
size 2
entry: Map: test
entry: Sunny: day

ARRAYDEMO1
a3: 16: (10, 20, 30) = 6000
a2: 12: (10, 20) = 200
10.200000
10.200000
10.200000
1.000000



Look okay?

Chris M. Thomasson

unread,
Mar 3, 2021, 11:49:19 PM3/3/21
to
Indeed. I have not tried all of the examples, but its working like a
charm so far! :^)

jacobnavia

unread,
Mar 4, 2021, 4:16:58 AM3/4/21
to
Le 03/03/2021 à 16:16, tylo a écrit :
> If you are envying c++ for their container classes, I made a decent library that should cover you. It is unusual with "templated" classes in C, but these are simple in use, and will save you from casting / throwing away valuable type information - which again may obscure buggy code. The lib is also very fast.
>
> https://github.com/tylov/STC
>

I worked 2 years in a similar project. I have tried to propose that
project in this group.

Your project presents a path I did not take. For two reasons:

1: No debugger. Since the macros aren't inline functions, very few
debuggers will be able to follow the macrologie to its humen readable
sources. A macro will have many other macros expanded, that will in turn
have another more lower level macros, etc. So, at the end you are
presented with a mess of statements where finding the bug is absolutely
impossible...

Then, you are forced to printf debugging, what surfaces now in the
output presented by Mr Thomasson and others.

2: Code bloat. I haven't tried, but it would be interesting to know the
size of a high level macro in terms of code size. Code bloat slows down
the machine since it destroys the code cches. Since some people have
compiled this, I would like to know how much is the size of an expanded
macro for accessing the nth memeber of a list, for instance.

David Brown

unread,
Mar 4, 2021, 6:09:10 AM3/4/21
to
On 04/03/2021 10:16, jacobnavia wrote:
> Le 03/03/2021 à 16:16, tylo a écrit :
>> If you are envying c++ for their container classes, I made a decent
>> library that should cover you. It is unusual with "templated" classes
>> in C, but these are simple in use, and will save you from casting /
>> throwing away valuable type information - which again may obscure
>> buggy code. The lib is also very fast.
>>
>> https://github.com/tylov/STC
>>
>
> I worked 2 years in a similar project. I have tried to propose that
> project in this group.

I think your library led to interest here, but I suspect that the need
for a container library in C is a good deal lower than most people
think. When you hear about such a library - yours or this new one -
there's an immediate reaction of "that's great - it will save a lot of
effort, and be useful in all sorts of cases". And yet it barely gets
used at all. That doesn't mean that such a library could not have been
used, but that for many reasons, it doesn't get used.

People generally use C at a lower level than other languages. If you
need good string handling, or map types, or variable sized vectors, and
you want them to be neat and convenient to work with, then C is not the
language for job. It is the language to /implement/ the code to make
these work quickly - not the the language for /using/ them.

To my mind, if you need these kinds of structures to form such a major
part of your coding that a generic container library is very helpful,
then C is unlikely to be the best choice of language to use in the first
place. It is more likely that in your C code, you only need one type of
map, or one type of queue, and then you make it yourself with the
balance you want between performance, simplicity, debugability, thread
safety, etc.

Perhaps I am wrong about that, and there is real potential for wider use
of such libraries.


>
> Your project presents a path I did not take. For two reasons:
>
> 1: No debugger. Since the macros aren't inline functions, very few
> debuggers will be able to follow the macrologie to its humen readable
> sources. A macro will have many other macros expanded, that will in turn
> have another more lower level macros, etc. So, at the end you are
> presented with a mess of statements where finding the bug is absolutely
> impossible...
>

It does look like scarily many macros. Debugging of inline functions
can be hard enough - macros are worse. But some things can only be done
with macros, when you don't know the types in advance.

> Then, you are forced to printf debugging, what surfaces now in the
> output presented by Mr Thomasson and others.
>
> 2: Code bloat. I haven't tried, but it would be interesting to know the
> size of a high level macro in terms of code size. Code bloat slows down
> the machine since it destroys the code cches. Since some people have
> compiled this, I would like to know how much is the size of an expanded
> macro for accessing the nth memeber of a list, for instance.

It might be interesting to compare the two libraries (and C++ with the
standard library), with a couple of examples - one in which there is
only one "map" or "queue" type used in the program, and one in which
there are several types, perhaps used in separately compiled units. You
could also compare how the application code looks for using these
different libraries, to see which looks clearer, is easier to
understand, easier to get right, and harder to get wrong.

Bart

unread,
Mar 4, 2021, 7:01:04 AM3/4/21
to
On 04/03/2021 11:09, David Brown wrote:
> On 04/03/2021 10:16, jacobnavia wrote:

>> 1: No debugger. Since the macros aren't inline functions, very few
>> debuggers will be able to follow the macrologie to its humen readable
>> sources. A macro will have many other macros expanded, that will in turn
>> have another more lower level macros, etc. So, at the end you are
>> presented with a mess of statements where finding the bug is absolutely
>> impossible...
>>
>
> It does look like scarily many macros. Debugging of inline functions
> can be hard enough - macros are worse. But some things can only be done
> with macros, when you don't know the types in advance.

When I tried one example with my compiler, I had to increase the
hard-coded limit on ##-pasted tokens in one macro from from 100 to 1000.

My view is that this isn't real C programming; it's like a script
language implemented with macros, and coding consists of lots of macro
invocations:

clist_fx list = clist_fx_init();
...
c_forrange (i, int, n)
clist_fx_push_back(&list, stc64_uniformf(&rng, &dist)), ++m;
...
int removed = clist_fx_remove(&list, 30);
clist_fx_insert_after(&list, clist_fx_before_begin(&list), 5);
clist_fx_push_back(&list, 500);
clist_fx_push_front(&list, 1964);
clist_fx_iter_t it = clist_fx_before_begin(&list);
printf("Full: ");
c_foreach (i, clist_fx, list)
printf(" %g", *i.ref);

Yuck. As for errors, if I change one of those &list to list, at least it
pinpoints the error properly, but with gcc is also gives an intriguing
trace back over the nested macros, which I imagine can be rather
intimidating:

list.c:42:24: error: incompatible type for argument 1 of
'clist_fx_push_back'
42 | clist_fx_push_back(list, 500);
| ^~~~
| |
| clist_fx {aka struct <anonymous>}
In file included from list.c:3:
../stc/clist.h:225:38: note: expected 'clist_fx *' {aka 'struct
<anonymous> *'} but argument is of t
ype 'clist_fx' {aka 'struct <anonymous>'}
225 | clist_##X##_push_back(clist_##X* self, Value value) { \
| ~~~~~~~~~~~^~~~
../stc/clist.h:204:5: note: in expansion of macro '_c_implement_clist_7'
204 | _c_implement_clist_7(X, Value, valueCompareRaw, valueDel,
valueFromRaw, valueToRaw, RawV
alue) \
| ^~~~~~~~~~~~~~~~~~~~
../stc/clist.h:62:21: note: in expansion of macro 'using_clist_7'
62 | using_clist_7(X, Value, valueCompare,
c_trivial_del, c_trivial_fromraw,
c_trivial_toraw, Value)
| ^~~~~~~~~~~~~
../stc/clist.h:60:21: note: in expansion of macro 'using_clist_3'
60 | using_clist_3(X, Value, c_default_compare)
| ^~~~~~~~~~~~~
../stc/ccommon.h:55:24: note: in expansion of macro 'using_clist_2'
55 | #define _c_CAT( A, B ) A ## B
| ^
../stc/ccommon.h:61:30: note: in expansion of macro '_c_CAT'
61 | #define _c_SELECT(NAME, NUM) _c_CAT( NAME ## _, NUM)
| ^~~~~~
../stc/ccommon.h:63:37: note: in expansion of macro '_c_SELECT'
63 | #define c_MACRO_OVERLOAD(NAME, ...) _c_SELECT(NAME,
_c_VA_ARG_SIZE(__VA_ARGS__))(__VA_ARGS__
)
| ^~~~~~~~~
../stc/clist.h:58:28: note: in expansion of macro 'c_MACRO_OVERLOAD'
58 | #define using_clist(...) c_MACRO_OVERLOAD(using_clist,
__VA_ARGS__)
| ^~~~~~~~~~~~~~~~
list.c:5:1: note: in expansion of macro 'using_clist'
5 | using_clist(fx, double);
| ^~~~~~~~~~~

Another factor is how much compilation of a large program written like
this would be slowed down by this degree of CPP use.

The library is very clever (I've seen a few like this and they have to
be), but I feel is more a tour de force.

David Brown

unread,
Mar 4, 2021, 8:15:26 AM3/4/21
to
On 04/03/2021 13:00, Bart wrote:
> On 04/03/2021 11:09, David Brown wrote:
>> On 04/03/2021 10:16, jacobnavia wrote:
>
>>> 1: No debugger. Since the macros aren't inline functions, very few
>>> debuggers will be able to follow the macrologie to its humen readable
>>> sources. A macro will have many other macros expanded, that will in turn
>>> have another more lower level macros, etc. So, at the end you are
>>> presented with a mess of statements where finding the bug is absolutely
>>> impossible...
>>>
>>
>> It does look like scarily many macros.  Debugging of inline functions
>> can be hard enough - macros are worse.  But some things can only be done
>> with macros, when you don't know the types in advance.
>
> When I tried one example with my compiler, I had to increase the
> hard-coded limit on ##-pasted tokens in one macro from from 100 to 1000.

Well, that's a positive outcome from the testing :-)

>
> My view is that this isn't real C programming; it's like a script
> language implemented with macros, and coding consists of lots of macro
> invocations:
>
>     clist_fx list = clist_fx_init();
>     ...
>     c_forrange (i, int, n)
>         clist_fx_push_back(&list, stc64_uniformf(&rng, &dist)), ++m;
>     ...
>     int removed = clist_fx_remove(&list, 30);
>     clist_fx_insert_after(&list, clist_fx_before_begin(&list), 5);
>     clist_fx_push_back(&list, 500);
>     clist_fx_push_front(&list, 1964);
>     clist_fx_iter_t it = clist_fx_before_begin(&list);
>     printf("Full: ");
>     c_foreach (i, clist_fx, list)
>         printf(" %g", *i.ref);
>
> Yuck. As for errors, if I change one of those &list to list, at least it
> pinpoints the error properly, but with gcc is also gives an intriguing
> trace back over the nested macros, which I imagine can be rather
> intimidating:
>

I think this is a very interesting point. It goes to the heart of
making a useful library - it should be easy to write correct code, and
hard to write incorrect code. The kind of error messages you get from
common compilers (with common flags, such as "-O2 -Wall -Wextra" and
various "-std" choices for gcc) is vital to making a library that is
useful. The code needs to be checked from this viewpoint with at least
gcc, clang and MSVC.
I don't expect the pre-processing speed to be a significant issue.

Anton Shepelev

unread,
Mar 4, 2021, 11:19:32 AM3/4/21
to
David Brown:

> It does look like scarily many macros. Debugging of
> inline functions can be hard enough -- macros are worse.

You can always produce the necessary expansions and reuse
them.

--
() ascii ribbon campaign - against html e-mail
/\ http://preview.tinyurl.com/qcy6mjc [archived]

Eli the Bearded

unread,
Mar 4, 2021, 1:46:17 PM3/4/21
to
In comp.lang.c, jacobnavia <ja...@jacob.remcomp.fr> wrote:
> Your project presents a path I did not take. For two reasons:
>
> 1: No debugger. Since the macros aren't inline functions, very few
> debuggers will be able to follow the macrologie to its humen readable
> sources. A macro will have many other macros expanded, that will in turn
> have another more lower level macros, etc. So, at the end you are
> presented with a mess of statements where finding the bug is absolutely
> impossible...
>
> Then, you are forced to printf debugging, what surfaces now in the
> output presented by Mr Thomasson and others.
>
> 2: Code bloat. I haven't tried, but it would be interesting to know the
> size of a high level macro in terms of code size. Code bloat slows down
> the machine since it destroys the code cches. Since some people have
> compiled this, I would like to know how much is the size of an expanded
> macro for accessing the nth memeber of a list, for instance.

Changing the topic, but this reminds me of the Carlini example in the
2020 IOCCC winners. Through macro expansion at compile time, 266k of a
276k binary was one string. And since that string was a printf() format,
you can't even use printf() to debug it.

"[I]ts primary purpose is to serve as The One True Debugger"
-- the hints file on printf() from Carlini

Elijah
------
has never noticed IOCCC get discussed here

Malcolm McLean

unread,
Mar 4, 2021, 2:41:14 PM3/4/21
to
On Thursday, 4 March 2021 at 16:19:32 UTC, Anton Shepelev wrote:
> David Brown:
> > It does look like scarily many macros. Debugging of
> > inline functions can be hard enough -- macros are worse.
>
> You can always produce the necessary expansions and reuse
> them.
>
C++ programmers are used to convoluted error messages.
The basic problem is that the linker won't support overloading,
operator names, namespaces, or templates. So it all goes into
a very complicated identifier, which is fed back to the programmer.

Ian Collins

unread,
Mar 4, 2021, 4:19:16 PM3/4/21
to
On 05/03/2021 00:09, David Brown wrote:
> On 04/03/2021 10:16, jacobnavia wrote:
>>
>> 1: No debugger. Since the macros aren't inline functions, very few
>> debuggers will be able to follow the macrologie to its humen readable
>> sources. A macro will have many other macros expanded, that will in turn
>> have another more lower level macros, etc. So, at the end you are
>> presented with a mess of statements where finding the bug is absolutely
>> impossible...
>>
>
> It does look like scarily many macros. Debugging of inline functions
> can be hard enough - macros are worse. But some things can only be done
> with macros, when you don't know the types in advance.

Try compiling one of the bench marks with -E...

With cvec_benchmark.cpp (C++ bits removed), "using_cvec(x, size_t);"
expands to a 4096 character line.

The code is still surprisingly quick!

--
Ian.

Chris M. Thomasson

unread,
Mar 5, 2021, 2:35:46 AM3/5/21
to
Its not nearly as cryptic as the chaos-pp package!

https://github.com/rofl0r/chaos-pp

;^o

tylo

unread,
Mar 7, 2021, 5:51:11 PM3/7/21
to
Thanks all of you for trying out the lib. I have made some replies to your comments below.

torsdag 4. mars 2021 kl. 14:15:26 UTC+1 skrev David Brown:
> On 04/03/2021 13:00, Bart wrote:
> > On 04/03/2021 11:09, David Brown wrote:
> >> On 04/03/2021 10:16, jacobnavia wrote:
> >
> >>> 1: No debugger. Since the macros aren't inline functions, very few
> >>> debuggers will be able to follow the macrologie to its humen readable
> >>> sources. A macro will have many other macros expanded, that will in turn
> >>> have another more lower level macros, etc. So, at the end you are
> >>> presented with a mess of statements where finding the bug is absolutely
> >>> impossible...

The using_clist(...) template macro simply implement a number of regular inline (or non-inline) functions and declare some types. It does not use nested level macros other than for the purpose of passing default "template" parameters (see below). A debugger will work exactly as if these functions were written by hand. The compiler *may* also refer to the original macro if functions are used wrongly, but many compilers e.g. gcc, clang will give the correct line reference in the source code. . See my next comment.

> >> It does look like scarily many macros. Debugging of inline functions
> >> can be hard enough - macros are worse. But some things can only be done
> >> with macros, when you don't know the types in advance.

The first part of most of the container headers (in particular the maps) have a number of macros defined, but these are just there for enabling overloading of e.g. the using_cmap() macro. These enable me to put in default values for the "template" parameters. I could have made only a single macro which generated all the code, but then you would have to pass in up to 12 template parameters just to declare a cmap<int, int>.

> > When I tried one example with my compiler, I had to increase the
> > hard-coded limit on ##-pasted tokens in one macro from from 100 to 1000.
> Well, that's a positive outcome from the testing :-)

That was a surprisingly low default. The macros are really not that long, e.g. one of the longest is 127 lines of code, but each method/type names ares pasted together from two or three of the macro parameters. I have used four different compilers for testing (gcc, clang, vc, and tinycc) and none made complaints.

> > My view is that this isn't real C programming; it's like a script
> > language implemented with macros, and coding consists of lots of macro
> > invocations:
> >
> > clist_fx list = clist_fx_init();
> > ...
> > c_forrange (i, int, n)
> > clist_fx_push_back(&list, stc64_uniformf(&rng, &dist)), ++m;
> > ...
> > int removed = clist_fx_remove(&list, 30);
> > clist_fx_insert_after(&list, clist_fx_before_begin(&list), 5);
> > clist_fx_push_back(&list, 500);
> > clist_fx_push_front(&list, 1964);
> > clist_fx_iter_t it = clist_fx_before_begin(&list);
> > printf("Full: ");
> > c_foreach (i, clist_fx, list)
> > printf(" %g", *i.ref);

This has only two macro invocations (c_forrange, c_foreach) and those are completely optional to use, just shorthands for for-loops. Check their documentation/implementation in ccommon.h.
Not exactly sure you get so much output. When I compiled this with gcc 9.20, all I get is:

list.c: In function 'main':
list.c:13:24: error: incompatible type for argument 1 of 'clist_fx_push_back'
13 | clist_fx_push_back(list, 500);
| ^~~~
| |
| clist_fx {aka struct <anonymous>}
In file included from list.c:1:
list.c:5:1: note: expected 'clist_fx *' {aka 'struct <anonymous> *'} but argument is of type 'clist_fx' {aka 'struct <anonymous>'}
5 | using_clist(fx, int);
| ^~~~~~~~~~~

And with clang, the first line is:

list.c:13:24: error: passing 'clist_fx' to parameter of incompatible type 'clist_fx *'; take the address with &
clist_fx_push_back(list, 500);
^~~~
and it then refers to a couple of times to the using_clist() macros after that, ending with:
stc/clist.h:225:38: note: expanded from macro '_c_implement_clist_7'
clist_##X##_push_back(clist_##X* self, Value value) { \

I don't find this too bad.

> > Another factor is how much compilation of a large program written like
> > this would be slowed down by this degree of CPP use.
> >
> I don't expect the pre-processing speed to be a significant issue.
> > The library is very clever (I've seen a few like this and they have to
> > be), but I feel is more a tour de force.

I would recommend you to try TinyC, https://repo.or.cz/w/tinycc.git
When compiling the example on the STC main page with 6 different containers, it compiles + links virtually instantaneously!, and create an executable of 27 Kb without any dependencies.
It doesn't create optimized code, but is very nice for development as you can also run C-files as scripts (compile, link all in memory).

tylo

unread,
Mar 8, 2021, 3:23:59 AM3/8/21
to
torsdag 4. mars 2021 kl. 12:09:10 UTC+1 skrev David Brown:
> I think your library led to interest here, but I suspect that the need
> for a container library in C is a good deal lower than most people
> think. When you hear about such a library - yours or this new one -
> there's an immediate reaction of "that's great - it will save a lot of
> effort, and be useful in all sorts of cases". And yet it barely gets
> used at all. That doesn't mean that such a library could not have been
> used, but that for many reasons, it doesn't get used.

I think there is some truth in this, but not for the reasons you point out next.

> People generally use C at a lower level than other languages. If you
> need good string handling, or map types, or variable sized vectors, and
> you want them to be neat and convenient to work with, then C is not the
> language for job. It is the language to /implement/ the code to make
> these work quickly - not the the language for /using/ them.
>
> To my mind, if you need these kinds of structures to form such a major
> part of your coding that a generic container library is very helpful,
> then C is unlikely to be the best choice of language to use in the first
> place.
>
> Perhaps I am wrong about that, and there is real potential for wider use
> of such libraries.

I do think you are wrong about this. I programmed in c++ almost exclusively for 10 years, but I would have no qualms writing a larger program in C which fully used this STC library. I would rather say that a lot of C programmers are quite conservative, but also less used to work with data structures like this (naturally, as there are no such things in C).
Also, a large portion of C programmers develops for embedded devices, and have often electronics background rather than computer science. Often such programs does not even use dynamic memory.

> It is more likely that in your C code, you only need one type of
> map, or one type of queue, and then you make it yourself with the
> balance you want between performance, simplicity, debugability, thread
> safety, etc.

This is the widespread myth. Sure, you can roll you own simple stack, but once you need something more (linked list, map, set, even just a dynamic vector), you will likely create something very limiting, and quite possible buggy code that isn't even as fast as you think. Once you have elements of strings, you must manage destruction and lifetime of the objects. The other aspect of this is that you code is not using standard components, and therefore much harder to read, maintain and verify correctness of.

> > 2: Code bloat. I haven't tried, but it would be interesting to know the
> > size of a high level macro in terms of code size. Code bloat slows down
> > the machine since it destroys the code cches. Since some people have
> > compiled this, I would like to know how much is the size of an expanded
> > macro for accessing the nth memeber of a list, for instance.
> It might be interesting to compare the two libraries (and C++ with the
> standard library), with a couple of examples - one in which there is
> only one "map" or "queue" type used in the program, and one in which
> there are several types, perhaps used in separately compiled units. You
> could also compare how the application code looks for using these
> different libraries, to see which looks clearer, is easier to
> understand, easier to get right, and harder to get wrong.

Good points. I have done some of this. STC library code is extremely compact. Only the methods that are used is linked in when you use the default static linkage.

David Brown

unread,
Mar 8, 2021, 5:46:57 AM3/8/21
to
I certainly agree that a lot of C programmers are conservative. C is a
good choice for conservative work (and "conservative" is the right
attitude for some kinds of programming) - no other general-purpose and
efficient language has the kind of long-term stability that C has.

> Also, a large portion of C programmers develops
> for embedded devices, and have often electronics background rather
> than computer science. Often such programs does not even use dynamic
> memory.

It is accurate to say that a large proportion of embedded devices are
programmed in C - I don't know if it is correct to say that a large
proportion of C programmers develop for embedded devices. (I'd love to
see some statistics on that one.) You are right that small-systems
embedded programmers usually try to avoid dynamic memory - I rarely use
it, except for network programming where it is difficult to avoid
entirely. That applies equally to my embedded C++ programming (and to
assembly, in the old days when I used that much more).

>
>> It is more likely that in your C code, you only need one type of
>> map, or one type of queue, and then you make it yourself with the
>> balance you want between performance, simplicity, debugability,
>> thread safety, etc.
>
> This is the widespread myth.

It is not a myth - people /do/ make their own structures as needed when
programming in C. It might not be a good idea, or the most effective
use of the programmers' time - that's another matter.

> Sure, you can roll you own simple stack,
> but once you need something more (linked list, map, set, even just a
> dynamic vector), you will likely create something very limiting, and
> quite possible buggy code that isn't even as fast as you think. Once
> you have elements of strings, you must manage destruction and
> lifetime of the objects. The other aspect of this is that you code is
> not using standard components, and therefore much harder to read,
> maintain and verify correctness of.
>

There are no standard containers for C - calling your library "standard"
does not make it so. If you (or Jacob) gain enough users, developers,
maintainers, documenters for your library that it can be called a "de
facto" standard and a fair proportion of C programmers were familiar
with it, then it becomes "easy to read" and its correctness can be taken
as given. But you don't have that. (I realise this is a
chicken-and-egg challenge.) If I download your library and use it with
my code, it is not easier to read and maintain than my own little list
structure that I am trying to replace. It is /harder/ to read, because
your library does so much more. It is /harder/ to verify, because I
don't know your code, I don't know if it is correct, if it will work
with my choice of compiler, flags, target processor, threading system,
etc. I don't know if it will work with interrupt functions, with lists
in read-only memory, etc. And to find out, I need to look at the code
for a dozen different list features that are irrelevant for me but are
part of your "standard" list container.

/If/ you can get your library into common use, then your arguments hold
- until then, the opposite is true. This is the reality of the
situation, unfortunate and unfair though it might seem.


>>> 2: Code bloat. I haven't tried, but it would be interesting to
>>> know the size of a high level macro in terms of code size. Code
>>> bloat slows down the machine since it destroys the code cches.
>>> Since some people have compiled this, I would like to know how
>>> much is the size of an expanded macro for accessing the nth
>>> memeber of a list, for instance.
>> It might be interesting to compare the two libraries (and C++ with
>> the standard library), with a couple of examples - one in which
>> there is only one "map" or "queue" type used in the program, and
>> one in which there are several types, perhaps used in separately
>> compiled units. You could also compare how the application code
>> looks for using these different libraries, to see which looks
>> clearer, is easier to understand, easier to get right, and harder
>> to get wrong.
>
> Good points. I have done some of this. STC library code is extremely
> compact. Only the methods that are used is linked in when you use the
> default static linkage.
>

Static linkage is good if the code is only used within one translation
unit (i.e., one C file, unless you are doing something really weird).
If you are using the same container structures, or the same functions,
in different C files then you will end up with a lot of code bloat.
(Whether or not code bloat is an issue depends on the application. But
since you mentioned embedded systems, it can certainly be an issue on
some small systems.)

C does not have a good solution for this. Some C toolchains can be
helpful, if you pick the right options, allowing the compiler and linker
to cooperate on merging duplicate code sections and eliminated sections
that are not actually used.

To get better, you need to move to C++, where templates and "inline"
give you better control of code duplication.

tylo

unread,
Mar 8, 2021, 8:36:34 AM3/8/21
to
Ok, last post from me for now because of time:

mandag 8. mars 2021 kl. 11:46:57 UTC+1 skrev David Brown:
> >> It is more likely that in your C code, you only need one type of
> >> map, or one type of queue, and then you make it yourself with the
> >> balance you want between performance, simplicity, debugability,
> >> thread safety, etc.
> >
> > This is the widespread myth.
> It is not a myth - people /do/ make their own structures as needed when
> programming in C. It might not be a good idea, or the most effective
> use of the programmers' time - that's another matter.

Bad wording from me, I think we agree here. I meant people tend to think it is best to roll their own "standard" datatypes whenever they need them,
which I think is unwise for many reasons, unless it is as simple as a (fixed size) stack.

> > lifetime of the objects. The other aspect of this is that you code is
> > not using standard components, and therefore much harder to read,
> > maintain and verify correctness of.
> >
> There are no standard containers for C - calling your library "standard"
> does not make it so. If you (or Jacob) gain enough users, developers,
> maintainers, documenters for your library that it can be called a "de
> facto" standard and a fair proportion of C programmers were familiar
> with it, then it becomes "easy to read" and its correctness can be taken
> as given. But you don't have that. (I realise this is a
> chicken-and-egg challenge.)

I kind of made the assumption that a library like STC should be a sort of "pseudo" standard, it could make C a much more useful language than it actually is. C has a lot of attractive properties which c++ lost along the way, a few of them are simplicity and low resource demands (compile, linkage, size), but can be still as fast as c++ code. And let's be honest here, the average C application source code looks crap, with tons of casting, anonymous void pointers and poor API. Every application use their own custom structures to represent collections of data, and maps are rarely used. Take python as contrast: even the simplest program use one or multiple maps - because it is availabe - and useful.

> If I download your library and use it with
> my code, it is not easier to read and maintain than my own little list
> structure that I am trying to replace. It is /harder/ to read, because
> your library does so much more. It is /harder/ to verify, because I
> don't know your code, I don't know if it is correct, if it will work
> with my choice of compiler, flags, target processor, threading system,
> etc. I don't know if it will work with interrupt functions, with lists
> in read-only memory, etc. And to find out, I need to look at the code
> for a dozen different list features that are irrelevant for me but are
> part of your "standard" list container.

When I talked about my lib as "standard", I referred to that each container is mapping c++ standard container classes, and that the API's base method names mapped roughly to names used by c++ container method. But of course, the challenge is that you cannot and should not trust it to be bug-free and to work as you expect before a larger community has used and tested it over a period (chicken and egg as you said).

I tried to make a very focused lib with only containers. I does add a few unneeded "convenience" macros an methods, which could be removed to enhance the focus, but it does not add much complexity atm.).

> /If/ you can get your library into common use, then your arguments hold
> - until then, the opposite is true. This is the reality of the
> situation, unfortunate and unfair though it might seem.

I accept this. I don't think it is unfair, but admit it may feel like casting pearls before swines, because people will continue to hold back the C language by either avoiding such containers, or making their own poor versions of them, and to program even high-level abstractions as it was low-level details. I think this eventually will result in that C becomes a marginal language, unfortunately.

> Static linkage is good if the code is only used within one translation
> unit (i.e., one C file, unless you are doing something really weird).
> If you are using the same container structures, or the same functions,
> in different C files then you will end up with a lot of code bloat.
> (Whether or not code bloat is an issue depends on the application. But
> since you mentioned embedded systems, it can certainly be an issue on
> some small systems.)
>
> C does not have a good solution for this. Some C toolchains can be
> helpful, if you pick the right options, allowing the compiler and linker
> to cooperate on merging duplicate code sections and eliminated sections
> that are not actually used.
>
> To get better, you need to move to C++, where templates and "inline"
> give you better control of code duplication.

STC will by default generate static or inline methods, but if you globally define STC_HEADER, the files only expose the API (like traditional headers), and you must create one c-file with #define STD_IMPLEMENTATION before including and declaring all your containers you use in your app (by the using_.. macros).
I totally accept that this is simpler in C++, but this should be managable in C as well.

David Brown

unread,
Mar 8, 2021, 9:19:52 AM3/8/21
to
On 08/03/2021 14:36, tylo wrote:
> Ok, last post from me for now because of time:
>
> mandag 8. mars 2021 kl. 11:46:57 UTC+1 skrev David Brown:
>>>> It is more likely that in your C code, you only need one type
>>>> of map, or one type of queue, and then you make it yourself
>>>> with the balance you want between performance, simplicity,
>>>> debugability, thread safety, etc.
>>>
>>> This is the widespread myth.
>> It is not a myth - people /do/ make their own structures as needed
>> when programming in C. It might not be a good idea, or the most
>> effective use of the programmers' time - that's another matter.
>
> Bad wording from me, I think we agree here. I meant people tend to
> think it is best to roll their own "standard" datatypes whenever they
> need them, which I think is unwise for many reasons, unless it is as
> simple as a (fixed size) stack.

Fair enough.

>
>>> lifetime of the objects. The other aspect of this is that you
>>> code is not using standard components, and therefore much harder
>>> to read, maintain and verify correctness of.
>>>
>> There are no standard containers for C - calling your library
>> "standard" does not make it so. If you (or Jacob) gain enough
>> users, developers, maintainers, documenters for your library that
>> it can be called a "de facto" standard and a fair proportion of C
>> programmers were familiar with it, then it becomes "easy to read"
>> and its correctness can be taken as given. But you don't have that.
>> (I realise this is a chicken-and-egg challenge.)
>
> I kind of made the assumption that a library like STC should be a
> sort of "pseudo" standard, it could make C a much more useful
> language than it actually is. C has a lot of attractive properties
> which c++ lost along the way, a few of them are simplicity and low
> resource demands (compile, linkage, size), but can be still as fast
> as c++ code.

Speaking for myself, I think the resources needed to compile and link
code are almost irrelevant. It does not bother me that a toolchain
installation is 700 MB. I have lots of versions of toolchains of that
size on my system. So what? I could install a new toolchain every
week, and never run out of space on even the cheapest of hard drives.
My biggest programs take perhaps 30 seconds to compile and link - /if/ I
do a clean, non-incremental, no ccache, single-threaded build. But I
don't do that for my work - so builds are typically no more than a
second or two.

I fully agree that C has some attractive properties - I am less in
agreement that C++ has "lost" them. They are different languages, with
different balances and different pros and cons. C values stability and
(relative) simplicity over features - C++ is willing to be big and
changeable in order to have more features.

> And let's be honest here, the average C application
> source code looks crap, with tons of casting, anonymous void pointers
> and poor API.

Mine do not. This kind of thing is an indication of bad design, bad
programming style, bad development methodology, bad management. It
certainly occurs in C programming - but it occurs in /all/ programming.
And a containers library will not change that.

> Every application use their own custom structures to
> represent collections of data, and maps are rarely used. Take python
> as contrast: even the simplest program use one or multiple maps -
> because it is availabe - and useful.

Some of that is a matter of using the features that are on hand in the
language you picked - some of it is picking the language with the
features you want.

>
>> If I download your library and use it with my code, it is not
>> easier to read and maintain than my own little list structure that
>> I am trying to replace. It is /harder/ to read, because your
>> library does so much more. It is /harder/ to verify, because I
>> don't know your code, I don't know if it is correct, if it will
>> work with my choice of compiler, flags, target processor, threading
>> system, etc. I don't know if it will work with interrupt functions,
>> with lists in read-only memory, etc. And to find out, I need to
>> look at the code for a dozen different list features that are
>> irrelevant for me but are part of your "standard" list container.
>
> When I talked about my lib as "standard", I referred to that each
> container is mapping c++ standard container classes, and that the
> API's base method names mapped roughly to names used by c++ container
> method.

If I want the C++ standard library, I know where to find it. And I
think that for most people familiar enough with the C++ standard library
for this similarity to be helpful, they will most likely choose C++
instead of C and your new library. But since any C containers library
will need names for types and functions, approximately copying those of
existing libraries is probably as good a choice as any.

> But of course, the challenge is that you cannot and should
> not trust it to be bug-free and to work as you expect before a larger
> community has used and tested it over a period (chicken and egg as
> you said).

Yes.

(And though I am not sure of your library's success in the C community -
Jacob's has not been as popular as he might have liked - don't take my
comments as criticism of your ideas. They are meant to show where I
think you will see challenges.)

>
> I tried to make a very focused lib with only containers. I does add a
> few unneeded "convenience" macros an methods, which could be removed
> to enhance the focus, but it does not add much complexity atm.).
>
>> /If/ you can get your library into common use, then your arguments
>> hold - until then, the opposite is true. This is the reality of the
>> situation, unfortunate and unfair though it might seem.
>
> I accept this. I don't think it is unfair, but admit it may feel like
> casting pearls before swines, because people will continue to hold
> back the C language by either avoiding such containers, or making
> their own poor versions of them, and to program even high-level
> abstractions as it was low-level details. I think this eventually
> will result in that C becomes a marginal language, unfortunately.
>

I also think that C will become a marginal language. But I think it
/should/ become a marginal language. I think C is the wrong choice for
a great deal of code that is written in C at the moment. C has its
place - in some kinds of systems code, in some low-level portable
libraries, in very small embedded systems, and as a common denominator
as an interface between libraries and programs written in different
languages. In my view, its use for applications is in the past. If you
need a container structure beyond what you can easily put together
yourself, and it is not so specialised that you /need/ to create it
yourself, then you should be questioning if C is the right choice of
language for the task. (I don't want to be too general here -
/sometimes/ it will be the right choice.)

Often you see questions about "how to I write a C program that does some
analysis and manipulation of text files?" To me, you first have to ask
"Is this a task to help me learn advanced C programming? Or is this the
kind of challenge I think is fun?" If the answer to these questions is
no, then the answer to the first one is "You don't write it in C. You
write it in Python. Your code will be much shorter, simpler, clearer,
far more likely to be correct, and will probably run faster with any
samples that are big enough for speed to matter."

tylo

unread,
Mar 8, 2021, 11:23:08 AM3/8/21
to
mandag 8. mars 2021 kl. 15:19:52 UTC+1 skrev David Brown:

> I also think that C will become a marginal language. But I think it
> /should/ become a marginal language. I think C is the wrong choice for
> a great deal of code that is written in C at the moment. C has its
> place - in some kinds of systems code, in some low-level portable
> libraries, in very small embedded systems, and as a common denominator
> as an interface between libraries and programs written in different
> languages. In my view, its use for applications is in the past. If you
> need a container structure beyond what you can easily put together
> yourself, and it is not so specialised that you /need/ to create it
> yourself, then you should be questioning if C is the right choice of
> language for the task.

Good points again. I acknowledge we have different views on a few things, but that's fine. Thanks for an interesting discussion.

Manfred

unread,
Mar 8, 2021, 12:53:17 PM3/8/21
to
Define marginal. But before that how do you classify server code?
Application or system? I think it is best a class on its own, and we all
know there's a lot of it written in C - and, as far as I have seen, with
good results, meaning that C is the right choice for the task.
For interactive applications I agree that since the last few decades
better choices than C are available.

Defining marginal.
If you mean with respect to the number of programs being /written/, then
I agree with you.
With respect to the amount of code that is or will be /run/, then,
irrespective of the fact this may be code that has already been written,
I don't think C is likely to become marginal any time soon.

David Brown

unread,
Mar 8, 2021, 2:01:54 PM3/8/21
to
I don't want to try to be too specific here - that would, I think,
inevitably end up either wrong to so limited as to be of little value.
I'm giving an opinion and rough prediction, rather than anything
accurate (which is always difficult about future trends!). As a general
pattern, I feel that more is written in C than is appropriate, and also
that many things that used to be written in C are now written in other
languages. That does not mean that C was not the right choice for those
tasks before - it means I think it is generally not the best choice in
as many cases as it used to be.

You mention servers as a class. It used to be the case that the only
sensible choice of language for a webserver, for example, was C. Later,
C++ might have been a reasonable option. And this would interact with
server side languages like PHP and Python running in separate processes,
passing data through pipes. Now if you wanted to write a new webserver
with a view to running Python server code, you'd likely write the whole
server in Python (the time-critical parts would be in C modules anyway).
If you wanted to run Ruby or PHP, you might pick those languages for
the webserver too. If it was to be a general purpose webserver, perhaps
Go would be the choice. There are many options, all giving better
developer productivity with lower risk of errors than C. It is rare
that the greater potential for raw speed of C is really necessary.

> Defining marginal.
> If you mean with respect to the number of programs being /written/, then
> I agree with you.
> With respect to the amount of code that is or will be /run/, then,
> irrespective of the fact this may be code that has already been written,
> I don't think C is likely to become marginal any time soon.
>

That is an important distinction, and I agree with you. It is like
noting that most C code is written for 64-bit systems - most C code is
/run/ on 8-bit systems. (Or that Hamlet is reckoned to be the most
written-about character, real or fictional, in the UK. But he is not
the most read-about character!)

Chris M. Thomasson

unread,
Mar 8, 2021, 8:05:48 PM3/8/21
to
On 3/3/2021 7:16 AM, tylo wrote:
> If you are envying c++ for their container classes, I made a decent library that should cover you. It is unusual with "templated" classes in C, but these are simple in use, and will save you from casting / throwing away valuable type information - which again may obscure buggy code. The lib is also very fast.
>
> https://github.com/tylov/STC
>

Are you familiar with a the container_of macro? Its quite handy. Let's
see if I can implement it from memory... Here it goes:

_________________________
#include <stdio.h>
#include <stddef.h>
#include <assert.h>


struct ct_slist_node
{
struct ct_slist_node* next;
};

struct ct_slist
{
struct ct_slist_node* head;
};

#define ct_container_of(m_ptr, m_type, m_member) \
((void*)((unsigned char*)(m_ptr) - offsetof(m_type, m_member)))


void
ct_slist_push(
struct ct_slist* self,
struct ct_slist_node* node
) {
node->next = self->head;
self->head = node;
}

struct ct_slist_node*
ct_slist_pop(
struct ct_slist* self
) {
struct ct_slist_node* node = self->head;
if (node) self->head = node->next;
return node;
}



struct foo
{
int a;
struct ct_slist_node* next;
char b;
long c;
};



int main(void)
{
struct foo foo;
struct foo* pfoo = ct_container_of(&foo.next, struct foo, next);

printf("&foo.next = %p\n", &foo.next);
printf("&foo = %p\n", &foo);
printf("pfoo = %p\n", pfoo);

assert(&foo == pfoo);

{
#define N 42

struct foo foo[N];
struct ct_slist slist = { NULL };

printf("\npush...\n");
for (unsigned int i = 0; i < N; ++i)
{
printf("&foo[%u] = %p\n", i, foo + i);
ct_slist_push(&slist, &foo[i].next);
}

printf("\npop...\n");
for (unsigned int i = 0; i < N; ++i)
{
struct ct_slist_node* node = ct_slist_pop(&slist);
struct foo* pfoo = ct_container_of(node, struct foo, next);
printf("pfoo = %p = &foo[%u - %u - 1] = %p\n", pfoo, N, i,
&foo[N - i - 1]);
assert(pfoo == &foo[N - i - 1]);
}
}

return 0;
}

_________________________

Can you compile this?

Chris M. Thomasson

unread,
Mar 8, 2021, 8:18:33 PM3/8/21
to
On 3/8/2021 5:05 PM, Chris M. Thomasson wrote:
> On 3/3/2021 7:16 AM, tylo wrote:
>> If you are envying c++ for their container classes, I made a decent
>> library that should cover you. It is unusual with "templated" classes
>> in C, but these are simple in use, and will save you from casting /
>> throwing away valuable type information - which again may obscure
>> buggy code. The lib is also very fast.
>>
>> https://github.com/tylov/STC
>>
>
> Are you familiar with a the container_of macro? Its quite handy. Let's
> see if I can implement it from memory... Here it goes:
>
> _________________________
> #include <stdio.h>
> #include <stddef.h>
> #include <assert.h>
>
>
> struct ct_slist_node
> {
>     struct ct_slist_node* next;
> };
>
> struct ct_slist
> {
>     struct ct_slist_node* head;
> };
>
> #define ct_container_of(m_ptr, m_type, m_member) \
>     ((void*)((unsigned char*)(m_ptr) - offsetof(m_type, m_member)))
> [...]
> struct foo
> {
>     int a;
>     struct ct_slist_node* next;
>     char b;
>     long c;
> };[...]

Oh damn it! The next member in struct foo should be:

struct foo
{
int a;
struct ct_slist_node next;
char b;
long c;
};

Yikes! This is what I get for typing code in the damn newsreader!

Chris M. Thomasson

unread,
Mar 8, 2021, 8:41:13 PM3/8/21
to
[...]

Imvvvvho, server code, especially a scalable server meant to handle tens
of thousands, hundreds of thousands, of concurrent connections, is best
written in C or C++. I remember way back creating highly scalable
servers that could handle 50,000 concurrent connections on WinNT 4.0
using IOCP.

Chris M. Thomasson

unread,
Mar 8, 2021, 8:45:22 PM3/8/21
to
This brings back many memories of having to deal with non-paged memory
limitations on Windows.

Manfred

unread,
Mar 9, 2021, 11:16:18 AM3/9/21
to
A webserver is somewhat peculiar nowadays, because the Internet is so
widespread, and you find a plethora of different contexts where a
"webserver" is deployed, from ISPs to IOT chips. Being so widespread,
some higher level languages even ship most of the http stack as part of
their standard library - a webserver in Java is next to trivial.

More in general, thinking things like e.g. fileservers or VPN gateways,
as far as I can see C pretty hard to beat.

David Brown

unread,
Mar 9, 2021, 11:55:31 AM3/9/21
to
On 09/03/2021 17:16, Manfred wrote:

>
> A webserver is somewhat peculiar nowadays, because the Internet is so
> widespread, and you find a plethora of different contexts where a
> "webserver" is deployed, from ISPs to IOT chips. Being so widespread,
> some higher level languages even ship most of the http stack as part of
> their standard library - a webserver in Java is next to trivial.

Fair enough.

>
> More in general, thinking things like e.g. fileservers or VPN gateways,
> as far as I can see C pretty hard to beat.
>
I think you would be very hard pushed to make a fileserver in C that is
much faster than, say Go or Rust. It would likely be a challenge even
to make it measurably faster than Python.

The thing about a lot of servers these days is that the application code
is not the bottleneck. That code spends most of its time waiting for
data from disks, data through networks, data being encrypted or
compressed (the encryption and compression stuff being written in C,
regardless of the higher level language).

And even when processor work at the high level code point (rather than
underlying libraries or OS code) is the issue, certain languages other
than C can make it easier to make more efficient use of the processor.
In C, you do things manually (memory management, threading, etc.). This
gives you full control - but it means you need to do the work in the
code, and that might not be the priority for the development. Your C
code needs to handle memory allocation and deallocation, both taking
time - a garbage collected language with more sophistication might do
all the freeing in the background on another thread, reducing the effort
needed during the critical paths.

I am not saying you are wrong here - I have no numbers to justify such a
claim. But I think the situation is a good deal more complex than just
"C has minimal overheads and is therefore the fastest language". It may
be the fastest language for some many of code (though other compiled
languages like C++, Fortran, Ada, D, etc., could challenge it), but the
code is often not the only consideration for the speed of a task.

Kaz Kylheku

unread,
Mar 9, 2021, 12:19:49 PM3/9/21
to
On 2021-03-09, David Brown <david...@hesbynett.no> wrote:
> On 09/03/2021 17:16, Manfred wrote:
>
>>
>> A webserver is somewhat peculiar nowadays, because the Internet is so
>> widespread, and you find a plethora of different contexts where a
>> "webserver" is deployed, from ISPs to IOT chips. Being so widespread,
>> some higher level languages even ship most of the http stack as part of
>> their standard library - a webserver in Java is next to trivial.
>
> Fair enough.
>
>>
>> More in general, thinking things like e.g. fileservers or VPN gateways,
>> as far as I can see C pretty hard to beat.
>>
> I think you would be very hard pushed to make a fileserver in C that is
> much faster than, say Go or Rust. It would likely be a challenge even
> to make it measurably faster than Python.

We can make a fileserver in C that might not be faster, but will have a
smaller incremental flash footprint and less RAM use.

Manfred

unread,
Mar 9, 2021, 12:43:06 PM3/9/21
to
I'm not talking about performance only. I see the fact that in C you
need to manage all of the resources manually as a bonus for this kind of
development, because control trumps easiness in this context - obviously
you need to do things right, and you may use reliable libraries that
suit your needs.

As an example, you mention garbage collection. This is exactly the kind
of thing that in principle I do not want in a server process. It /adds/
complexity to the design (contrary to naive beliefs) because it adds
constraints to the memory model, and in addition it reduces
predictability of process execution.

David Brown

unread,
Mar 9, 2021, 1:58:49 PM3/9/21
to
Ah, "flash footprint" implies small embedded system - and then size
often matters. (C++ would typically be as good as or better than C,
however.) And code speed also usually matters more on such a system.

Siri Cruise

unread,
Mar 10, 2021, 12:25:54 AM3/10/21
to
In article <s26hkv$b9p$1...@gioia.aioe.org>,
"Chris M. Thomasson" <chris.m.t...@gmail.com> wrote:

> Are you familiar with a the container_of macro? Its quite handy. Let's
> see if I can implement it from memory... Here it goes:

You can make it nicer with a larger #define.

#define Stack(T) SingleLinkList(Stack, T)

#define Queue(T) \
SingleLinkList(Queue, T) \
static inline Queue##T append##T (Queue##T top, T value) { \
Queue ##T e = push##T(0, value); \
if (!top) return e; \
for (Queue##T *p=&top; *(p); ) p = &(*(p)->next); \
*(p)->next = e; return top; \
}

#define List(T) \
SingleLinkList(List, T) \
static inline List##T after##T (List##T list, T value) { \
if (!list) return push##T(0, value); \
List##T e = push##T(list->next, value); \
list->next = e; return e; \
}
static inline Queue##T reverse##T (List##T list) { \
for (List##T prev=0, curr=list, next=0; ; \
prev=curr, curr=next) \
if (curr) { \
next = curr->next; curr->next = prev; \
}else \
return prev; \
}

#define SingleLinkList(C, T) \
typedef struct C##T *C##T; \
struct Container##T {C##T next; T value;}; \
static inline C##T nihil##T (void) { \
return 0; \
} \
static inline C##T push##T (C##T next, T value) { \
C##T e = GCMALLOC(struct C##T); \
e->next = next; e->value = value; return e; \
} \
static inline C##T pop##T (C##T e) { \
return e ? e->next : 0; \
} \
static inline T top##T (C##T e) { \
if (e) return e->value; \
T v; memset(&v, 0, sizeof v); \
return v; \
} \
static inline int empty##T (C##T e) { \
return !e; \
}

You can make it even more intricate, such as dealing with
Stack(struct {int inner;}) by using a general macro processor. I
have my own, but you can manhandle m4 into working.

--
:-<> Siri Seal of Disavowal #000-001. Disavowed. Denied. Deleted. @
'I desire mercy, not sacrifice.' /|\
Discordia: not just a religion but also a parody. This post / \
I am an Andrea Doria sockpuppet. insults Islam. Mohammed

tylo

unread,
Mar 16, 2021, 5:49:51 PM3/16/21
to
I know it's a few days now, but I already use container_of extensively in stc/clist.h to get the node address from value address inside the node. Btw, clist is circular, and therefore support both push_back and push_front without extra pointers. Here in non-templated form:

struct clist_node {
struct clist_node* next;
Value value;
};
typedef struct {
clist_node_t* last; // last->next is head, last is either last or pre-first from an iterators POV.
} clist;

Thiago Adams

unread,
Apr 13, 2021, 2:34:18 PM4/13/21
to
On Wednesday, March 3, 2021 at 12:16:22 PM UTC-3, twrote:
> If you are envying c++ for their container classes, I made a decent library that should cover you. It is unusual with "templated" classes in C, but these are simple in use, and will save you from casting / throwing away valuable type information - which again may obscure buggy code. The lib is also very fast.
>
> https://github.com/tylov/STC

This is something that I have interest but I didn't find
the time to analyze the code.

The good thing I saw is that you are using macros instead of runtime
performance penalties using function pointers for instance.

One problem I see (even in C++) is that depending on how cheap is the copy
of the ITEM type the interface may change. For instance, vector of int compared
with vector of some struct. To push back the item we may use pointer to struct,
but for int the use value directly. The same when returning lets say vector_back we
may return the value for int but for structs we return the pointer.

What is your design for that?



wij

unread,
Apr 15, 2021, 12:51:40 PM4/15/21
to
On Wednesday, 3 March 2021 at 23:16:22 UTC+8, tylo...@gmail.com wrote:
> If you are envying c++ for their container classes, I made a decent library that should cover you. It is unusual with "templated" classes in C, but these are simple in use, and will save you from casting / throwing away valuable type information - which again may obscure buggy code. The lib is also very fast.
>
> https://github.com/tylov/STC

I abandoned STL since the 1st time using it, facade is fancy, but as you
know more, the time invested in psudo-directives/implemtational-concepts
is not so worthy, nothing to envy about. IMHO, What C should envy about is probably member functions, RAII, overloading,...sizeof(char)=1,...those simple, basic improvements.
And the ability to simulate STL in C is really something to envy about.

Kaz Kylheku

unread,
Apr 15, 2021, 2:09:04 PM4/15/21
to
On 2021-04-15, wij <wyn...@gmail.com> wrote:
> overloading,...sizeof(char)=1,...those simple, basic improvements.

sizeof(char) == 1 in C; this is not a C++ thing.

sizeof('c') == 1 is the C++ thing.

It's needed so that func('c') will call the func(char) overload of func.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal

David Brown

unread,
Apr 15, 2021, 2:27:43 PM4/15/21
to
On 15/04/2021 20:08, Kaz Kylheku wrote:
> On 2021-04-15, wij <wyn...@gmail.com> wrote:
>> overloading,...sizeof(char)=1,...those simple, basic improvements.
>
> sizeof(char) == 1 in C; this is not a C++ thing.
>
> sizeof('c') == 1 is the C++ thing.
>
> It's needed so that func('c') will call the func(char) overload of func.
>

It is not the size that is important, it is the fact that in C++ a
character literal is of /type/ "char" rather than an "int", as in C.
"int" and "char" might have the same size on some platforms, but the
types are still different.

It is (IMHO) just a historical glitch that character constants are "int"
in C - making them "char" is far more logical. But in C, it usually
does not matter, while in C++ it can be important (as you say, for
overloading and the like).

Bart

unread,
Apr 15, 2021, 3:59:07 PM4/15/21
to
I was wondering what my systems language did about 'char'. I get these
results:

println char.typestr # Shows c8 (internal code for 'char')
println 'A'.typestr # c64
println 'A' # A

So 'A' has char/c8 type, but because of widening rules, it ends up at 64
bits. Not as i64 but as a wide char version c64 to retain identity.

Unlike C, it tends not to pass narrow types (narrower than 64 bits, or
in C, narrower than 32) to and from functions (what's the point?). They
are mainly storage types or to use in conversions.

Anyway that makes it possible to print 'A' as A rather than 65.

anti...@math.uni.wroc.pl

unread,
Apr 16, 2021, 12:14:40 AM4/16/21
to
David Brown <david...@hesbynett.no> wrote:
>
> I also think that C will become a marginal language. But I think it
> /should/ become a marginal language. I think C is the wrong choice for
> a great deal of code that is written in C at the moment. C has its
> place - in some kinds of systems code, in some low-level portable
> libraries, in very small embedded systems, and as a common denominator
> as an interface between libraries and programs written in different
> languages. In my view, its use for applications is in the past. If you
> need a container structure beyond what you can easily put together
> yourself, and it is not so specialised that you /need/ to create it
> yourself, then you should be questioning if C is the right choice of
> language for the task. (I don't want to be too general here -
> /sometimes/ it will be the right choice.)
>
> Often you see questions about "how to I write a C program that does some
> analysis and manipulation of text files?" To me, you first have to ask
> "Is this a task to help me learn advanced C programming? Or is this the
> kind of challenge I think is fun?" If the answer to these questions is
> no, then the answer to the first one is "You don't write it in C. You
> write it in Python. Your code will be much shorter, simpler, clearer,
> far more likely to be correct, and will probably run faster with any
> samples that are big enough for speed to matter."

You exagerate about speed. A lot of programmers make severe
performance mistakes (like turning something that should be
linear into quagratic one). So using library code (for example
via Python) makes sense. However, if you actually need to
write code (say the task in not supported by available libraries)
then C normally will give advantage. Much depends on
granularity of task, if you need to handle small pieces of
data than C can give significant speedups. Specifically
in context of text, some tasks are done using regular
expressions, in such case using C+lex can give short and
fast programs. If you need to do anything interesting, like
looking at separate words again C has speed advantages.
More generaly, in languages like Python essentially all
"interesting" data structures are full of pointers.
In C programmer has option to use small items like
characters, small integers, bitfields, etc. That leads
to significantly more compact data. And experience shows
that on average such data is faster to process than data
that uses a lot of indirection via pointers. Of course the
same advantage holds for C++ and few other languages that
allow programmer-controlled layout of data. Several
other languages force a lot of pointer indirection, and
various other overheads.

Concerning C becoming marginal: I think that today C is
language of choice for performance-critical parts of
programs, in particular for performance critical libraries.

It would be nice to have language giving good performance for
low level tasks and higher level features than C, but
I think that today there are no language _clearly_ better
than C. C++ is quite good in performance part, but
suffers from complexity and incompatibilities (in particular
needs special care if code is supposed to be called from
other languages). Other lanuages lag in tool support,
platform support and/or libraries.

Concerning writing applications in C, there is some merit
in writing whole thing in one language. If there are
performance critial parts, then there is some reason to
use C. If C is good for whole depends on there factors:
- does code benefit from static typing?
- is code better expressed as function calls (as opposed
to operators or object orientation)?
- is automatic memory management preferable?
- what is expected liftime of application (in particular
if it is stable or quickly evolving)?
and many others

C is flexible enough that one can work around missing
features, but clearly language which directly supports
what is needed is better. Still, IMO in many cases C
is not a bad choice.

--
Waldek Hebisch

Thiago Adams

unread,
Apr 16, 2021, 7:19:43 AM4/16/21
to
On Monday, March 8, 2021 at 11:19:52 AM UTC-3, David Brown wrote:
[...]
>C has its
> place - in some kinds of systems code, in some low-level portable
> libraries, in very small embedded systems, and as a common denominator
> as an interface between libraries and programs written in different
> languages. In my view, its use for applications is in the past
[...]

I use C in small applications that could possibly use any other
language or script language available.
Why?
- Because I don't need to install hundreds of megabytes of
something else and force other people who need to use my
tool/code to install it. Something that also will became obsolete
in few years.
- Because the executable is small and it can compile everywhere.
- I can use the same code without changing in 20 years ahead.

and I can use the knowledge and code in a more serious program
that would benefit from C portability, performance ...

Malcolm McLean

unread,
Apr 16, 2021, 7:57:50 AM4/16/21
to
A good example of a sophisticated string algorithm would be a suffix tree.
These are very memory hungry, and often used on DNA sequences which
are hundred of megabytes in length.
A package is available for Python of course. I just had a look at one. It is
written in Python and is about the same length as the C version which is floating
round the web (300 lines of Python as opposed to 500 lines of C, but the C has
block comments).

Chris M. Thomasson

unread,
Apr 16, 2021, 4:05:32 PM4/16/21
to

wij

unread,
Apr 17, 2021, 12:47:35 AM4/17/21
to
How is this related to C?

Chris M. Thomasson

unread,
Apr 17, 2021, 12:55:03 AM4/17/21
to
Oh, sorry for going off topic. I read where somebody mentioned "A good
example of a sophisticated string algorithm would be a suffix tree" in
this thread, and it triggered ropes in my mind. Well, at least, they
both can be implemented in C. :^)

wij

unread,
Apr 17, 2021, 2:11:13 AM4/17/21
to
That is fine and good for me. Because some one asked the same question to the post:
https://groups.google.com/g/comp.lang.c/c/pNZU4rRwkcE

Often, people post pure language (e.g. C) questions. While the following
replies from others may ask for more 'unrelated' background information, "in order to"
provide appropriate answer (or to understand the 'language' question better)

Keith Thompson

unread,
Apr 17, 2021, 2:22:50 AM4/17/21
to
wij <wyn...@gmail.com> writes:
> On Saturday, 17 April 2021 at 12:55:03 UTC+8, Chris M. Thomasson wrote:
>> On 4/16/2021 9:47 PM, wij wrote:
[...]
>> > How is this related to C?
>> >
>> Oh, sorry for going off topic. I read where somebody mentioned "A good
>> example of a sophisticated string algorithm would be a suffix tree" in
>> this thread, and it triggered ropes in my mind. Well, at least, they
>> both can be implemented in C. :^)
>
> That is fine and good for me. Because some one asked the same question to the post:
> https://groups.google.com/g/comp.lang.c/c/pNZU4rRwkcE

Yes, that was me, in response to your post "Fast algorithm computing
multiplications of 2^n number of decreasing natural numbers", which
appeared to have nothing at all to do with C.

I'm not sure why you chose to respond to that by saying the same thing
on an unrelated thread rather than, oh, I don't know, actually answering
the question.

> Often, people post pure language (e.g. C) questions. While the
> following replies from others may ask for more 'unrelated' background
> information, "in order to" provide appropriate answer (or to
> understand the 'language' question better)

--
Keith Thompson (The_Other_Keith) Keith.S.T...@gmail.com
Working, but not speaking, for Philips Healthcare
void Void(void) { Void(); } /* The recursive call of the void */

Malcolm McLean

unread,
Apr 17, 2021, 7:50:04 AM4/17/21
to
Suffix trees are for fast searching and matching. Ropes are for fast editing. So
you'd use a suffix tree for DNA sequences, a rope for a long text in a text
editor.
But like most algorithms / data structures, they can be implemented in a wide
variety of languages. However C is a good language of choice for the
"reference implementation" because the intended audience will almost certainly
be familiar with C, because of C's simplicity, and because C constructs map
very efficiently to hardware operations.

Bart

unread,
Apr 17, 2021, 8:11:08 AM4/17/21
to
How big text files are we talking about for speed to be enough of an
issue that ropes need to be used?

My main editor runs as interpreted code; it can edit files up to a
million lines with no discernible lag. Which anyway doesn't affect
editing within a line.

The data structure used is an array of strings, one per line. More
efficient might be a linked list, but no need for that yet.

Malcolm McLean

unread,
Apr 17, 2021, 8:22:59 AM4/17/21
to
On Saturday, 17 April 2021 at 13:11:08 UTC+1, Bart wrote:
> On 17/04/2021 12:49, Malcolm McLean wrote:
> > On Saturday, 17 April 2021 at 05:55:03 UTC+1, Chris M. Thomasson wrote:
>
> >> Oh, sorry for going off topic. I read where somebody mentioned "A good
> >> example of a sophisticated string algorithm would be a suffix tree" in
> >> this thread, and it triggered ropes in my mind. Well, at least, they
> >> both can be implemented in C. :^)
> >>
> > Suffix trees are for fast searching and matching. Ropes are for fast editing. So
> > you'd use a suffix tree for DNA sequences, a rope for a long text in a text
> > editor.
> How big text files are we talking about for speed to be enough of an
> issue that ropes need to be used?
>
You need to be able to update the representation of the text and refresh the
screen within the key repeat time, which can go up to 30 times per second
though ten times a second is more common.
>
> My main editor runs as interpreted code; it can edit files up to a
> million lines with no discernible lag. Which anyway doesn't affect
> editing within a line.
>
> The data structure used is an array of strings, one per line. More
> efficient might be a linked list, but no need for that yet.
>
You can throw big problems at a modern computer without too much
regard for efficiency. However if you can't use a flat buffer, then there's a
good argument for using a rope rather than devising your own ad hoc
structure, which will then be intimately tied to the line-break algorithm,
which will be tied to the font description.

Bart

unread,
Apr 17, 2021, 8:59:28 AM4/17/21
to
On 17/04/2021 13:22, Malcolm McLean wrote:
> On Saturday, 17 April 2021 at 13:11:08 UTC+1, Bart wrote:
>> On 17/04/2021 12:49, Malcolm McLean wrote:
>>> On Saturday, 17 April 2021 at 05:55:03 UTC+1, Chris M. Thomasson wrote:
>>
>>>> Oh, sorry for going off topic. I read where somebody mentioned "A good
>>>> example of a sophisticated string algorithm would be a suffix tree" in
>>>> this thread, and it triggered ropes in my mind. Well, at least, they
>>>> both can be implemented in C. :^)
>>>>
>>> Suffix trees are for fast searching and matching. Ropes are for fast editing. So
>>> you'd use a suffix tree for DNA sequences, a rope for a long text in a text
>>> editor.
>> How big text files are we talking about for speed to be enough of an
>> issue that ropes need to be used?
>>
> You need to be able to update the representation of the text and refresh the
> screen within the key repeat time, which can go up to 30 times per second
> though ten times a second is more common.
>>
>> My main editor runs as interpreted code; it can edit files up to a
>> million lines with no discernible lag. Which anyway doesn't affect
>> editing within a line.
>>
>> The data structure used is an array of strings, one per line. More
>> efficient might be a linked list, but no need for that yet.
>>
> You can throw big problems at a modern computer without too much
> regard for efficiency.

I doubt know if you regard the original RPi as modern (it came out about
a decade ago and was considered slow even then). But I remember porting
my interpreter there (via a C program). Then my intepreted editor was
still more responsive than the built-in editors like Nano and I think Vim.


> However if you can't use a flat buffer, then there's a
> good argument for using a rope rather than devising your own ad hoc
> structure, which will then be intimately tied to the line-break algorithm,
> which will be tied to the font description.
>

Maybe it's just me; I prefer to stick with the simplest possible data
structures and algorithms as much as possible. I've looked at ropes
before and decided they were too complicated.

0 new messages