Use m4 and make to compose C programs

[luser droog]

I'm trying to build my new project in a very modular fashion,
with unit tests for each module.
https://github.com/luser-dr00g/inca/tree/master/olmec

Starting with the minimal unit-testing framework from here:
http://www.jera.com/techinfo/jtns/jtn002.html
I've add main() functions to my 2 modules io.c and st.c
guarded by #ifdef TESTMODULE.

Without TESTMODULE defined, the module will compile to a .o
file and link normally with the rest of the application
(once that's written). A testing program of a single unit
is coordinated by a short "test" source file. The io_test.c
program looks like this

#define TESTMODULE
#include "io.c"

and similarly for the other one, st_test.c.

Now I wanted to compose these two programs into one program
which executes all the tests sequentially.

So I wrote this all_tests.c program to coordinate name-mangling
and avoid conflicts.

# include <stdio.h>

# define main io_main
# define tests_run io_tests_run
# define all_tests io_all_tests
# include "io_test.c"
# undef main
# undef tests_run
# undef all_tests
int io_test(){
printf("running io_test\n");
return io_main();
}

# define main st_main
# define tests_run st_tests_run
# define all_tests st_all_tests
# include "st_test.c"
# undef main
# undef tests_run
# undef all_tests
int st_test(){
printf("running st_test\n");
return st_main();
}

int main(){
return
0 || io_test() || st_test() ;
}


Now this was fine for just 2 modules. But I didn't want to
have to keep this updated as I add new ones. I tried
using X-macros, knowing I could pass everything through
`cpp -P | indent -gnu -i4 -br -ce -cdw -nbc -brf -brs -l100 -bbo`.
But, a cpp macro cannot expand with an embedded newline.
So even though I can generate a #define line, I can't make 2.

So, I wrote this in m4 to generate the C program based on a
macro called UNITS. At first I defined it explicitly with
define(`UNITS', (io,st))
But then I added more machinery to generate this list with make.
Unfortunately, I couldn't get make to create a comma-separated
list. Thus, still more machinery had to go into the m4 file to
do the conversion.

all_tests.m4:

divert(`-1')
# http://www.gnu.org/savannah-checkouts/gnu/m4/manual/m4-1.4.17/html_node/Foreach.html#Foreach
# foreach(x, (item_1, item_2, ..., item_n), stmt)
# parenthesized list, simple version
define(`foreach', `pushdef(`$1')_foreach($@)popdef(`$1')')
define(`_arg1', `$1')
define(`_foreach', `ifelse(`$2', `()', `',
`define(`$1', _arg1$2)$3`'$0(`$1', (shift$2), `$3')')')

define(`UNITS', (patsubst(UNITS,`\W',`,')))

divert`'dnl
`#' include <stdio.h>
foreach(`unit', UNITS, `
`#' define main unit`'_main
`#' define tests_run unit`'_tests_run
`#' define all_tests unit`'_all_tests
`#' include "unit`'_test.c"
`#' undef main
`#' undef tests_run
`#' undef all_tests
int unit`'_test(){
printf("running unit`'_test\n");
return unit`'_main();
}
')dnl

int main(){
return
0 foreach(`unit', UNITS, ` || unit`'_test() ') ;
}


makefile:

testprogs= $(notdir $(wildcard ./*_test.c))
unitprogs= $(subst _test,,$(testprogs))
units= $(basename $(unitprogs))

test:all_tests
./all_tests
all_tests.c:all_tests.m4 makefile $(unitprogs)
m4 -D UNITS="$(units)" $< >$@


And the funny part is I spent the whole day on this
instead of fixing the failing module.

josh@cadabra ~/inca/olmec
$ touch st.c

josh@cadabra ~/inca/olmec
$ make test
m4 -D UNITS="io st" all_tests.m4 >all_tests.c
cc all_tests.c -o all_tests
./all_tests
running io_test
ALL TESTS PASSED
Tests run: 4
running st_test
t->key != 42
Tests run: 1
makefile:7: recipe for target 'test' failed
make: *** [test] Error 1

[Ian Collins]

luser droog wrote:
> I'm trying to build my new project in a very modular fashion,
> with unit tests for each module.
> https://github.com/luser-dr00g/inca/tree/master/olmec
>
> Starting with the minimal unit-testing framework from here:
> http://www.jera.com/techinfo/jtns/jtn002.html
> I've add main() functions to my 2 modules io.c and st.c
> guarded by #ifdef TESTMODULE.
>
> Without TESTMODULE defined, the module will compile to a .o
> file and link normally with the rest of the application
> (once that's written). A testing program of a single unit
> is coordinated by a short "test" source file. The io_test.c
> program looks like this
>
> #define TESTMODULE
> #include "io.c"

I'll snip here and ask the obvious question: why can't you just unit
test the code as is?

Everything I write has unit tests (I use TDD) and I never have anything
conditionally compiled.

--
Ian Collins

[luser droog]

On Saturday, August 22, 2015 at 1:01:24 AM UTC-5, Ian Collins wrote:
> luser droog wrote:
> > I'm trying to build my new project in a very modular fashion,
> > with unit tests for each module.
> > https://github.com/luser-dr00g/inca/tree/master/olmec
> >
> > Starting with the minimal unit-testing framework from here:
> > http://www.jera.com/techinfo/jtns/jtn002.html
> > I've add main() functions to my 2 modules io.c and st.c
> > guarded by #ifdef TESTMODULE.
> >
> > Without TESTMODULE defined, the module will compile to a .o
> > file and link normally with the rest of the application
> > (once that's written). A testing program of a single unit
> > is coordinated by a short "test" source file. The io_test.c
> > program looks like this
> >
> > #define TESTMODULE
> > #include "io.c"
>
> I'll snip here and ask the obvious question: why can't you just unit
> test the code as is?

I'm afraid I don't understand the question. How do you

"unit test the code as is"?

>
> Everything I write has unit tests (I use TDD) and I never have anything
> conditionally compiled.
>

I imagine the testing code has to be separated into the "test" file.
The way I've come up with lets me keep the test code together with
the implementation. I think (for me) it will help keep the testing
updated and used more.

I've also wanted to do something with m4 for a while. Here finally
was something cpp couldn't do.

[Ian Collins]

luser droog wrote:
> On Saturday, August 22, 2015 at 1:01:24 AM UTC-5, Ian Collins wrote:
>>
>> I'll snip here and ask the obvious question: why can't you just unit
>> test the code as is?
>
> I'm afraid I don't understand the question. How do you
> "unit test the code as is"?

The code has nothing conditional or test specific in it. What is built
for production is what is tested.

>> Everything I write has unit tests (I use TDD) and I never have anything
>> conditionally compiled.
>
> I imagine the testing code has to be separated into the "test" file.

Test, in my case each logical module has its own test set of tests,
everything outside of that module is mocked.

> The way I've come up with lets me keep the test code together with
> the implementation. I think (for me) it will help keep the testing
> updated and used more.

The easiest way to do that is to set your build to have make = "make
test". That way you can't build without testing.

> I've also wanted to do something with m4 for a while. Here finally
> was something cpp couldn't do.

Maybe, but is the cost (in hard to read code) worth the benefit?

You should be testing production code built as production code,
optimisations, compiler options and all.

--
Ian Collins

[luser droog]

On Saturday, August 22, 2015 at 2:22:27 AM UTC-5, Ian Collins wrote:
> luser droog wrote:
> > On Saturday, August 22, 2015 at 1:01:24 AM UTC-5, Ian Collins wrote:
> >>
> >> I'll snip here and ask the obvious question: why can't you just unit
> >> test the code as is?
> >
> > I'm afraid I don't understand the question. How do you
> > "unit test the code as is"?
>
> The code has nothing conditional or test specific in it. What is built
> for production is what is tested.

With respect, this sounds to me more like a prejudice than a conclusion.
My source files are organized into two sections.

...
//module code
...

#ifdef TESTMODULE
...
//unit tests
...
int main(){
// call unit tests
}
#endif

I consider it a "partitioned file". It contains an "embedded resource".
The module code is not contaminated thereby (that I can see from my POV).

> >> Everything I write has unit tests (I use TDD) and I never have anything
> >> conditionally compiled.
> >
> > I imagine the testing code has to be separated into the "test" file.
>
> Test, in my case each logical module has its own test set of tests,
> everything outside of that module is mocked.

This sentence also describes my implementation, excepted that I haven't
needed to mock anything yet.

> > The way I've come up with lets me keep the test code together with
> > the implementation. I think (for me) it will help keep the testing
> > updated and used more.
>
> The easiest way to do that is to set your build to have make = "make
> test". That way you can't build without testing.

Good idea. There's no application code yet, but this is the current
behavior.

> > I've also wanted to do something with m4 for a while. Here finally
> > was something cpp couldn't do.
>
> Maybe, but is the cost (in hard to read code) worth the benefit?

Is it really so hard to read? It's as direct and clear as I am (so far)
able to make it.

> You should be testing production code built as production code,
> optimisations, compiler options and all.
>

I think I can instruct the makefile to do this.

BTW, I've posted the same material to SO with a little more details
and nicer formatting.

http://stackoverflow.com/questions/32163935/compose-a-combined-test-suite-program-from-a-collection-of-unit-tests

[Ian Collins]

luser droog wrote:
> On Saturday, August 22, 2015 at 2:22:27 AM UTC-5, Ian Collins wrote:
>> luser droog wrote:
>>> On Saturday, August 22, 2015 at 1:01:24 AM UTC-5, Ian Collins wrote:
>>>>
>>>> I'll snip here and ask the obvious question: why can't you just unit
>>>> test the code as is?
>>>
>>> I'm afraid I don't understand the question. How do you
>>> "unit test the code as is"?
>>
>> The code has nothing conditional or test specific in it. What is built
>> for production is what is tested.
>
> With respect, this sounds to me more like a prejudice than a conclusion.
> My source files are organized into two sections.
>

> ....
> //module code
> ....
>
> #ifdef TESTMODULE
> ....
> //unit tests
> ....

> int main(){
> // call unit tests
> }
> #endif
>
> I consider it a "partitioned file". It contains an "embedded resource".
> The module code is not contaminated thereby (that I can see from my POV).

I'd need to see a real example, but the normal practice with unit
testing is to keep the source under test and the tests in separate
source files.

Some unit test frameworks (CxxTest is one example) generate their main()
by paring the tests and generating code. Others use macros to build
their test suites.

>>>> Everything I write has unit tests (I use TDD) and I never have anything
>>>> conditionally compiled.
>>>
>>> I imagine the testing code has to be separated into the "test" file.
>>
>> Test, in my case each logical module has its own test set of tests,
>> everything outside of that module is mocked.
>
> This sentence also describes my implementation, excepted that I haven't
> needed to mock anything yet.

Take so time to look as some of the unit test frameworks out there and
you will see a common pattern. This hasn't happened by chance, they end
up that way in order to implement all of the features users want. Look
at both C and C++ frameworks, both can be used to unit test C code.

>>> I've also wanted to do something with m4 for a while. Here finally
>>> was something cpp couldn't do.
>>
>> Maybe, but is the cost (in hard to read code) worth the benefit?
>
> Is it really so hard to read? It's as direct and clear as I am (so far)
> able to make it.

Introducing an unfamiliar macro processing language will inevitably make
it harder for programmers to read. It also introduces a dependency on
m4. Is there windows m4 port? Some existing frameworks use a scripting
(the ones I've used use Python) for code generation, but the users
doesn't see the code.

--
Ian Collins

[Malcolm McLean]

On Monday, August 24, 2015 at 12:15:39 AM UTC+1, Ian Collins wrote:
>
> I'd need to see a real example, but the normal practice with unit
> testing is to keep the source under test and the tests in separate
> source files.
>

Then you can't unit test a static function.

[Richard Damon]

Since UNIT test are generally done on API boundaries (that is the normal
definition of a 'Unit'), static functions aren't normally unit tested in
isolation, but are tested as part of the unit they are part of.

Without an API, it is very hard to write a unit test.

The 'API' of internal functions also tend to be a bit more fluid than
the external API of a module, so are less viable for building tests, as
tests are more apt to need to be changed to met changing needs.

[Malcolm McLean]

However static functions often have more clearly defined behaviour
than the API. Consider a JPEG loader. You won't expose the frequency
transform. But constructing a JPEG file that puts the transform
through its paces is quite hard, and then you've got to isolate the
problem if the test fails. Testing the forwards and reverse transforms
against each other is quite easy, and you see problems like off by
one which might not be obvious when testing the whole API.

[luser droog]

Link from OP: https://github.com/luser-dr00g/inca/tree/master/olmec

> Some unit test frameworks (CxxTest is one example) generate their main()
> by paring the tests and generating code. Others use macros to build
> their test suites.
>

Indeed yes. The one I've chosen is MinUnit, the minimalist framework which
is just a few macros documented with just a few paragraphs:
http://www.jera.com/techinfo/jtns/jtn002.html

This is all for my new total redesign for an APL system, drawing from the
torturous purgatorio of the inca3 trainwreck.
https://github.com/luser-dr00g/inca/blob/master/inca3.c

In many ways this stage seems to mirror the transition from xpost2 to xpost3.
https://github.com/luser-dr00g/xpost/blob/wiki/EvolutionOfXpost.md

[Perhaps I should have added more of this prefatory to the OP)

> >>>> Everything I write has unit tests (I use TDD) and I never have anything
> >>>> conditionally compiled.
> >>>
> >>> I imagine the testing code has to be separated into the "test" file.
> >>
> >> Test, in my case each logical module has its own test set of tests,
> >> everything outside of that module is mocked.
> >
> > This sentence also describes my implementation, excepted that I haven't
> > needed to mock anything yet.
>
> Take so time to look as some of the unit test frameworks out there and
> you will see a common pattern. This hasn't happened by chance, they end
> up that way in order to implement all of the features users want. Look
> at both C and C++ frameworks, both can be used to unit test C code.
>

IOW Ya AM gonna need it?

> >>> I've also wanted to do something with m4 for a while. Here finally
> >>> was something cpp couldn't do.
> >>
> >> Maybe, but is the cost (in hard to read code) worth the benefit?
> >
> > Is it really so hard to read? It's as direct and clear as I am (so far)
> > able to make it.
>
> Introducing an unfamiliar macro processing language will inevitably make
> it harder for programmers to read. It also introduces a dependency on
> m4. Is there windows m4 port? Some existing frameworks use a scripting
> (the ones I've used use Python) for code generation, but the users
> doesn't see the code.
>

Without having actually researched, but due solely to its age and provenance,
I daresay that m4 is more portable than Python.

Was it not also blessed from the ttys of K and R?

And on the other handle, my whole schtick, my raisin debt, is my nonconformity,
even with all the other nonconformists. I've got to build it myself.
Even if it takes 10 times longer, it will undoubtedly be thereby 10 times more
awesome. And I can cite this thread's URL in the code itself for additional
documentation. Hypertext.

I understand your concerns, but really am covering all those bases which belong
to us.

droog

[luser droog]

> > luser droog wrote:
> > > This sentence also describes my implementation, excepted that I haven't
> > > needed to mock anything yet.

I did once play a duet with a mockingbird. It was on the roof of my little
house, and I grabbed my guitar and played really high up the neck on the
top strings. We traded eights a couple of times before he flew away.

[wil...@wilbur.25thandclement.com]

On the other hand, by committing to unit testing you're forcing yourself to
systematically improve the design of your code at the lowest layers. That
pays important dividends down the road.

There are always external pressures which reduce the quality of code--time,
competition for attention, ceaseless change in dependencies. By committing
to unit testing you're creating a counter-veiling positive pressure, which
is a benefit above and beyond the immediate value of unit testing--quicker
bug discovery, fewer regressions, etc.