Testing
folknology
C Unity - http://sourceforge.net/apps/trac/embunity/wiki
C Mock - http://cmock.sourceforge.net/
Both are C based and thus provide an excellent starting point. Anyone
have any alternatives they would like to put forward?
The second questions is how these tools work with XC, we would have to
drop down to C for testing as they are (I am assuming we could do most
of it from C). OR do we take these tools and extend them into XC
perhaps?
Thoughts?
regards
Al
micke.l...@hotmail.com
Does the "thing" somehow interpret C code ?
On Nov 22, 10:23 am, folknology <a...@folknology.com> wrote:
> Ok to the subject of testing here are a couple of recommendations:
>
> C Unity -http://sourceforge.net/apps/trac/embunity/wiki
Al Wood
<micke.l...@hotmail.com> wrote:
> What will happen with XC code ?
> Does the "thing" somehow interpret C code ?
Just concentrating on Unity first. Unity is basically a unit testing
framework for C, as such it is used to enable one to write tests
(mainly assertions) that can be run of a given piece/module of code.
Unit testing is part of a Test Driven Development (TDD) approach and
helps control improve code certainty and increase agility,
particularly for code that is developed among groups or communities.
The key is to effectively draw up and code assertions of the codes
behaviour before one actually writes the code. Once a a test harness
is created the code maybe written and tested against it. This should
ideally happen at a micro level small test written followed by small
code written tests run, changes made until tests pass, repeat and
rinse. i.e. it should be gradual and incremental rather than grand,
waterfall based cycles. The tests themselves can be aggregated into
test suites.
As to how unity works it appears to use macros, I am still digging
into its operation.
I won't cover Mock objects yet, but basically thats where it gets more
tricky as we effectively have to simulate hardware.
> On Nov 22, 10:23 am, folknology <a...@folknology.com> wrote:
>> Ok to the subject of testing here are a couple of recommendations:
>>
>> C Unity -http://sourceforge.net/apps/trac/embunity/wiki
>> C Mock -http://cmock.sourceforge.net/
>>
>> Both are C based and thus provide an excellent starting point. Anyone
>> have any alternatives they would like to put forward?
>>
>> The second questions is how these tools work with XC, we would have to
>> drop down to C for testing as they are (I am assuming we could do most
>> of it from C). OR do we take these tools and extend them into XC
>> perhaps?
>>
>> Thoughts?
>>
>> regards
>> Al
>
Al Wood
and create a fork : Unity XC
Russ Ferriday
--r
Russ Ferriday -- Systems Architect & Entrepreneur
CEO Topia Systems Ltd.
ru...@topia.com -- +1 (805) 910 7877 -- www.topia.com
Al Wood
Al Wood
other things. We can also hook it up to the simulator which won't be
backward compatible!!
Al Wood
Maybe XCunit will be our first code base?
Russ Ferriday
This would help for our projects that are a mixture of C/XC
But I say this without having looked, so don't quote me.
In general, forks are expensive, because they dilute the community. If you want a feature that's compatible with an existing project, better to extend or make a plug-in.
--r.
Al Wood
> I was suggesting that the tool should be extended to to handle XC, alongside its current capabilities.
> This would help for our projects that are a mixture of C/XC
>
> But I say this without having looked, so don't quote me.
If you take a look at the current SVN head you will see it uses a lot
of function and other pointers that would make it unsuitable for XC
usage. Adding XC support more or less means a rewrite as far as I can
see. It would be the proverbial tail wagging the dog for the Unity
folks, particularly since its aimed at 'C' and not XC ;-)
> In general, forks are expensive, because they dilute the community. If you want a feature that's compatible with an existing project, better to extend or make a plug-in.
This would be a plugin, rewrite is the correct term, unless someone
shows me how we can use void void types, pointers and function
pointers and line numbers in XC.
Al Wood
XC,C,C++,ASM and styles as the standard for our project. That includes
having default languages for projects with rules like only use XC and
if you can't do it in XC use inline ASM etc.. Or whatever the choice
is. What we don't want is prjects written in 4 different languages as
that would be bad! What is our default lang is it XC or C++ or one of
the others?
Russ Ferriday
On Nov 22, 2010, at 9:16 AM, Al Wood wrote:
>
> This would be a plugin, rewrite is the correct term, unless someone
> shows me how we can use void void types, pointers and function
> pointers and line numbers in XC.
OK. Got it!
;)
--r
Russ Ferriday
But the community should be able to help answer these questions for specific projects.
--r.
Al Wood
> In other contexts, the respective foundation has supported education and consideration of best practices, but has not ruled on details like this.
>
I'm just thinking about libraries fully inter-operating more than
anything else, and building on each project/library foundation.
Al Wood
the conclusion we are going to have to think very carefully about
testing with XC. Most unit testing (not even mock objects) relies on
good functional coverage. In XC's case however, we aren't just dealing
with imperative function or method calling operations, we are in fact
operating within concurrent sequential processes, via channels and
ports, not to mention concurrency. Thus unless someone already has
experience with this we will have to start from scratch in uncharted
TDD territory.
We need basically need mechanisms to handle channel based testing in
addition to standard function based tests. In addition we will also
need to find a way to create mock ports and also deal with common
concurrency issues that need to be tested for. Clearly this will be a
project in it's self
Thoughts?
Russ Ferriday
On Nov 22, 2010, at 12:54 PM, Al Wood wrote:
> I have given this some deeper thought this evening and have come to
> the conclusion we are going to have to think very carefully about
> testing with XC. Most unit testing (not even mock objects) relies on
> good functional coverage. In XC's case however, we aren't just dealing
> with imperative function or method calling operations, we are in fact
> operating within concurrent sequential processes, via channels and
> ports, not to mention concurrency. Thus unless someone already has
> experience with this we will have to start from scratch in uncharted
> TDD territory.
Thanks for looking at this in detail, Al.
I agree that it's hard to test CSP. Since each process is potentially a state machine you connect to over one or more channels, we would need the ability to set processes (process networks) to known states, stimulate, then check the outcome.
All of that looks tricky, and will require a lot of work, if it's not to interfere with the system under test.
>
> We need basically need mechanisms to handle channel based testing in
> addition to standard function based tests. In addition we will also
> need to find a way to create mock ports and also deal with common
> concurrency issues that need to be tested for. Clearly this will be a
> project in it's self
Yup.
If you had something as simple as a cache process, you could test it in a test harness before linking it to the rest of your system.
You could also make mock processes, that reacted like the real process.
I tend to write about 30% to 40% extra code, to setup tests for my code. It's a pain, but I can sleep at night.
It makes sense to have whole test setups that then are not loaded into the final product, to help with mem constraints.
--r.
Al Wood
harness, including top level processes. However as you go lower down
into the code and deal with sub processes and then even selects
including channels, timers and ports it becomes difficult to visualise
or imagine how to construct the relevant concurrent and interactive
tests.
Kaspar Bumke
Al Wood
was effectively mock devices and mock controllers, this was in part
due to the nature of the systems I was working on, but I think it
extends way beyond that to actualators, simudevices , and simulators.
I think I'm going to need to revisit that analysis sooner than I
imagined, it appears to be in our near term community destiny.
folknology
before it gets too looong
First of all I know many groan when they here 'test' and 'testing', in
fact some physically start to upchuck at the mention of TDD ;-)
So let me put a few things straight because I think it might help us
see the bigger picture and it might not be exactly what you think,
bear with me for a bit here.
In community projects and code testing can be very useful, in the old
days when we used to have to 'Patch' the mainline we would often
insist many patches running a suite of tests before we even looked at
the submissions. This isn't as many believe because folks like
bureaucracy and or control (there are exceptions ;-) but rather to
prevent the main commiters wasting to much time checking patch
submission quality, it provided a way of setting standards in the
code, and it can be a tedious job otherwise. Obviosuly now with Git
and distributed VCS we don't have patches we just deal with merging
forks, the tools make this much easier, however the quality issue is
still something that has to be addresses and setting some sort of
standard helps when it comes to pulling in from other forks.
There is also something else that tests contribute, testing can if
applied correctly help with the design and structure of software
modules, in order to be tested a certain amount of structure is
required, things like Dependency Injection often play an important
part. I won't dwell on this as that's for another thread suffice to
say it helps improve code, we just have to be careful it doesn't take
it over.
Now here is the interesting part, because XC is based around CSP
principles, testing will need to be done somewhat differently, in fact
IMHO it will need to jump straight into what I call stimulation and
actualation (or virtual activation) this is because we cannot rely on
functions as the single entry to operation and testing, rather
channels play a more central role. Clearly channel operation differs
from functional operation and that reveals new challenges which as in
some ways less like testing and more like simulation and or agile
virtual development.
So in order to take this further I will start a thread about Moxy,
which is really just the start of a conversation about how we tackle
these things in the community and I will got through some things That
I have been working on to see if they fit with what others have been
doing, and believe you me it will be centred not around testing as
such but rather designing and interactive coding using XC, including
how we make all of this stuff fit together.
P.S. I should also add that XCommons is not about writing test code or
frameworks etc.., these are complimentary to the real code projects
which are happening in parallel, like Quadmode, infact Moxy is really
just a discussion rather than code.
Regards
Al
On Nov 23, 1:52 am, Al Wood <a...@folknology.com> wrote:
> Almost Kasper. From the work I did over the summer what I saw emerging
> was effectively mock devices and mock controllers, this was in part
> due to the nature of the systems I was working on, but I think it
> extends way beyond that to actualators, simudevices , and simulators.
> I think I'm going to need to revisit that analysis sooner than I
> imagined, it appears to be in our near term community destiny.
>