Google test & Alternative/embedded oses and Emulators

614 views
Skip to first unread message

Olivier Binda

unread,
Aug 19, 2010, 4:10:39 AM8/19/10
to Google C++ Testing Framework
Hello

1) thanks for the google test project, it's great !

2) I managed to make google test and hudson work for the nintendo ds
through the Desmume emulator on Windows (and probably linux/mac os x
too).

So, it's actually possible to use gtest to test code for embedded
hardware through an emulator,
given that there is a way for the emulator to get the return code and
some strings from the code it runs.

I had to make Desmume read (peek) into the emulated nintendo ds memory
and
to make the main fonction on my test write (poke) into the nds memory
to achieve that.

now, some feature requests :

a) in gtest-port.h, I didn't define any GTEST_OS_* and it didn't
compile with

#define GTEST_HAS_CLONE 0
#define GTEST_HAS_EXCEPTIONS 0
#define GTEST_HAS_GLOBAL_STRING 0
#define GTEST_HAS_GLOBAL_WSTRING 0
#define GTEST_HAS_PTHREAD 0
#define GTEST_HAS_RTTI 0
#define GTEST_HAS_STD_WSTRING 0
#define GTEST_HAS_TR1_TUPLE 0
#define GTEST_HAS_SEH 0
#define GTEST_USE_OWN_TR1_TUPLE 1
#define GTEST_LINKED_AS_SHARED_LIBRARY 0
#define GTEST_CREATE_SHARED_LIBRARY 0

#define GTEST_HAS_DEATH_TEST 0
#define GTEST_HAS_PARAM_TEST 1
#define GTEST_HAS_TYPED_TEST 1
#define GTEST_HAS_TYPED_TEST_P 1
#define GTEST_USES_SIMPLE_RE 1
#define GTEST_CAN_COMPARE_NULL 1
#define GTEST_IS_THREADSAFE 0

So I had to add
#include <unistd.h>

It wasn't clear to me what feature was in or not.
Maybee you could define a GTEST_OS_OTHER to make porting easier,
especially when you cross-compile for embedded system

b) stream redirection doesn't work on the nintendo ds, it doesn't have
dup(), dup2().
At some point, gtest complained about it.

Can't we define explicitely GTEST_HAS_STREAM_REDIRECTION ?
(it is automatically defined somewhere else)

c) I had to rewrite some listeners. I used the TersePrinter example
there
http://code.google.com/p/googletest/source/browse/trunk/samples/sample9_unittest.cc

Sample 9 and Sample 10 (great examples) are missing on this webpage
http://code.google.com/p/googletest/wiki/GoogleTestSamples

d) please, do provide more base examples of stdout (thks for the
TersePrinter) and xml formatters : I had to write my own from scratch,
and every other gtest user has too

e) when writing my xml formatter, I subclassed
XmlUnitTestResultPrinter to reuse some of your code.

I had to remove "private" in XmlUnitTestResultPrinter to do that
though.
And it took some time to make it all work.

Please make it easier for the little guys to write plain/xml
formatters on platforms that may lack features :

I wouldn't have had to rewrite XmlUnitTestResultPrinter if it exported
it's result to a C STRING.
Yet I had to as it exports it to a FILE and as writing/reading files
doesn't work out of the box in the desmume emulator ( and adding file
read/write support to desmume wasn't doable in a reasonnable aount of
time).

I bet I'm not the only one that would like to cross-test with
googletest through emulators (embedded hardware, android, symbian,
windows ce, ...) ?
Best regards,
Olivier Binda
ps : my (un cleaned) code : class Buffer should be really called
FakeStream.
Who knows, people might find a use for it.

My XmlFormatter :

using ::testing::EmptyTestEventListener;
using ::testing::InitGoogleTest;
using ::testing::Test;
using ::testing::TestCase;
using ::testing::TestEventListeners;
using ::testing::TestInfo;
using ::testing::TestPartResult;
using ::testing::UnitTest;
class Buffer {
public :
Buffer(u32 size): size(size){
start = new char[size];
pos = 0;
};
~Buffer(){
delete start;
}
u32 append(char letter){
if (pos+1<size) {
start[pos++] = letter;
return 1;
}
return 0;
}
u32 append(const char* string){
if (!string) return 0;
u32 n=0;
while ((pos+1<size) && *string) {
start[pos++]=*(string++);
n++;
}
return n;
}
Buffer& operator<< (const char * string){
this->append(string);
return *this;
};
Buffer& operator<< (char letter){
this->append(letter);
return *this;
};
char * buffer(){return start;}
char* start;
u32 pos;
u32 size;
};


using::testing::internal::XmlUnitTestResultPrinter;
//using::testing::internal::FormatTimeInMillisAsSeconds;


// This class generates an XML output file.
namespace testing{ namespace internal{

// This class generates an XML output file.
class XmlUnitTestResultPrinter : public EmptyTestEventListener {
public:
explicit XmlUnitTestResultPrinter(const char* output_file);

virtual void OnTestIterationEnd(const UnitTest& unit_test, int
iteration);

//private:
// Is c a whitespace character that is normalized to a space
character
// when it appears in an XML attribute value?
static bool IsNormalizableWhitespace(char c) {
return c == 0x9 || c == 0xA || c == 0xD;
}

// May c appear in a well-formed XML document?
static bool IsValidXmlCharacter(char c) {
return IsNormalizableWhitespace(c) || c >= 0x20;
}

// Returns an XML-escaped copy of the input string str. If
// is_attribute is true, the text is meant to appear as an attribute
// value, and normalizable whitespace is preserved by replacing it
// with character references.
static String EscapeXml(const char* str, bool is_attribute);

// Returns the given string with all characters invalid in XML
removed.
static String RemoveInvalidXmlCharacters(const char* str);

// Convenience wrapper around EscapeXml when str is an attribute
value.
static String EscapeXmlAttribute(const char* str) {
return EscapeXml(str, true);
}

// Convenience wrapper around EscapeXml when str is not an attribute
value.
static String EscapeXmlText(const char* str) { return EscapeXml(str,
false); }

// Streams an XML CDATA section, escaping invalid CDATA sequences as
needed.
static void OutputXmlCDataSection(::std::ostream* stream, const
char* data);

// Streams an XML representation of a TestInfo object.
static void OutputXmlTestInfo(::std::ostream* stream,
const char* test_case_name,
const TestInfo& test_info);

// Prints an XML representation of a TestCase object
static void PrintXmlTestCase(FILE* out, const TestCase& test_case);

// Prints an XML summary of unit_test to output stream out.
static void PrintXmlUnitTest(FILE* out, const UnitTest& unit_test);

// Produces a string representing the test properties in a result as
space
// delimited XML attributes based on the property key="value" pairs.
// When the String is not empty, it includes a space at the
beginning,
// to delimit this attribute from prior attributes.
static String TestPropertiesAsXmlAttributes(const TestResult&
result);

// The output file.
const String output_file_;

GTEST_DISALLOW_COPY_AND_ASSIGN_(XmlUnitTestResultPrinter);
};

class myXmlPrinter : public XmlUnitTestResultPrinter {
public:
myXmlPrinter(Buffer* stream);

virtual void OnTestIterationEnd(const UnitTest& unit_test, int
iteration);

// Prints an XML representation of a TestCase object
void PrintXmlTestCas(Buffer* stream, const TestCase& test_case);

// Prints an XML summary of unit_test to output Buffer stream.
void PrintXmlUnitTes(Buffer* stream, const UnitTest& unit_test);

void OutputXmlTestInf(Buffer* stream,
const char*
test_case_name,
const TestInfo&
test_info);
// The output file.
Buffer* stream;
};

// Creates a new XmlUnitTestResultPrinter.
myXmlPrinter::myXmlPrinter(Buffer* stream)
: XmlUnitTestResultPrinter("not empty"), stream(stream) {
if (!stream) {
fprintf(stderr, "XML output buffer may not be null\n");
fflush(stderr);
exit(EXIT_FAILURE);
}
}

// Called after the unit test ends.
void myXmlPrinter::OnTestIterationEnd(const UnitTest& unit_test,
int /*iteration*/) {
myXmlPrinter::PrintXmlUnitTes(stream, unit_test);
}
std::string FormatTimeInMillisAsSeconds(TimeInMillis ms);


// Returns an XML-escaped copy of the input string str. If
is_attribute
// is true, the text is meant to appear as an attribute value, and
// normalizable whitespace is preserved by replacing it with character
// references.
//
// Invalid XML characters in str, if any, are stripped from the
output.
// It is expected that most, if not all, of the text processed by this
// module will consist of ordinary English text.
// If this module is ever modified to produce version 1.1 XML output,
// most invalid characters can be retained using character references.
// TODO(wan): It might be nice to have a minimally invasive, human-
readable
// escaping scheme for invalid characters, rather than dropping them.

// The following routines generate an XML representation of a UnitTest
// object.
//
// This is how Google Test concepts map to the DTD:
//
// <testsuites name="AllTests"> <-- corresponds to a UnitTest
object
// <testsuite name="testcase-name"> <-- corresponds to a TestCase
object
// <testcase name="test-name"> <-- corresponds to a TestInfo
object
// <failure message="...">...</failure>
// <failure message="...">...</failure>
// <failure message="...">...</failure>
// <-- individual assertion
failures
// </testcase>
// </testsuite>
// </testsuites>


// Prints an XML representation of a TestInfo object.
// TODO(wan): There is also value in printing properties with the
plain printer.
void myXmlPrinter::OutputXmlTestInf(Buffer* stream,
const char*
test_case_name,
const TestInfo&
test_info) {
const TestResult& result = *test_info.result();
*stream << " <testcase name=\""
<< EscapeXmlAttribute(test_info.name()).c_str()
<< "\" status=\""
<< (test_info.should_run() ? "run" : "notrun")
<< "\" time=\"" <<
FormatTimeInMillisAsSeconds(result.elapsed_time()).c_str()
<< "\" classname=\"" <<
EscapeXmlAttribute(test_case_name).c_str()
<< "\"" << TestPropertiesAsXmlAttributes(result).c_str();

int failures = 0;
for (int i = 0; i < result.total_part_count(); ++i) {
const TestPartResult& part = result.GetTestPartResult(i);
if (part.failed()) {
if (++failures == 1)
*stream << ">\n";
*stream << " <failure message=\""
<< EscapeXmlAttribute(part.summary()).c_str()
<< "\" type=\"\">";
const String message =
RemoveInvalidXmlCharacters(String::Format(
"%s:%d\n%s",
part.file_name(), part.line_number(),
part.message()).c_str());
//OutputXmlCDataSection(stream, message.c_str());
*stream << "</failure>\n";
}
}

if (failures == 0)
*stream << " />\n";
else
*stream << " </testcase>\n";
}

// Prints an XML representation of a TestCase object
void myXmlPrinter::PrintXmlTestCas(Buffer* stream,
const TestCase&
test_case) {
char out[1024];
sprintf(out,
" <testsuite name=\"%s\" tests=\"%d\" failures=\"%d\" "
"disabled=\"%d\" ",
EscapeXmlAttribute(test_case.name()).c_str(),
test_case.total_test_count(),
test_case.failed_test_count(),
test_case.disabled_test_count());
*stream << out;
sprintf(out,
"errors=\"0\" time=\"%s\">\n",

FormatTimeInMillisAsSeconds(test_case.elapsed_time()).c_str());
*stream <<out;
for (int i = 0; i < test_case.total_test_count(); ++i) {
StrStream sstream;
OutputXmlTestInf(stream, test_case.name(),
*test_case.GetTestInfo(i));
sprintf(out, "%s", StrStreamToString(&sstream).c_str());
*stream << out;
}
sprintf(out, " </testsuite>\n");
*stream << out;
};

// Prints an XML summary of unit_test to output stream out.
void myXmlPrinter::PrintXmlUnitTes(Buffer* stream,
const UnitTest&
unit_test) {
char out[1024];
sprintf(out, "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n");
*stream<<out;
sprintf(out,
"<testsuites tests=\"%d\" failures=\"%d\" disabled=\"%d\" "
"errors=\"0\" time=\"%s\" ",
unit_test.total_test_count(),
unit_test.failed_test_count(),
unit_test.disabled_test_count(),

FormatTimeInMillisAsSeconds(unit_test.elapsed_time()).c_str());
*stream<<out;
if (GTEST_FLAG(shuffle)) {
sprintf(out, "random_seed=\"%d\" ", unit_test.random_seed());
*stream<<out;
}
sprintf(out, "name=\"AllTests\">\n");
*stream<<out;
for (int i = 0; i < unit_test.total_test_case_count(); ++i)
PrintXmlTestCas(stream, *unit_test.GetTestCase(i));
sprintf(out, "</testsuites>\n");
*stream<<out;
}
}}//namespaces

// End XmlUnitTestResultPrinter



Mike

unread,
Aug 19, 2010, 8:08:43 AM8/19/10
to Google C++ Testing Framework
This is very cool. I am also using googletest to test embedded
software and this kind of functionality interests me very much. Up
until now I have only performed testing on the host environment so I
really interested in seeing where this goes.

Regards.
Mike
> therehttp://code.google.com/p/googletest/source/browse/trunk/samples/sampl...
>
> Sample 9 and Sample 10 (great examples) are missing on  this webpagehttp://code.google.com/p/googletest/wiki/GoogleTestSamples

Vlad Losev

unread,
Aug 20, 2010, 2:59:01 PM8/20/10
to Olivier Binda, Google C++ Testing Framework
Hi Olivier,

It's nice to know Google Test is used even on Nintendo!

We have recently updated the code to compile on systems other than those explicitly recognized in gtest-port.h. Try grabbing code from svn head and see if you can compile it without modifications. 

b) stream redirection doesn't work on the nintendo ds, it doesn't have
dup(), dup2().
At some point, gtest complained about it.

Can't we define explicitely GTEST_HAS_STREAM_REDIRECTION ?
(it is automatically defined somewhere else)

This looks like a real problem for you, and require the source code modification. I have opened issue 305 to track the fix. Star it to get updates.


c) I had to rewrite some listeners. I used the TersePrinter example
there
http://code.google.com/p/googletest/source/browse/trunk/samples/sample9_unittest.cc

Sample 9 and Sample 10 (great examples) are missing on  this webpage
http://code.google.com/p/googletest/wiki/GoogleTestSamples

Thanks for reporting this! I have updated the samples page to also point to #9 and #10.
 
d) please, do provide more base examples of stdout (thks for the
TersePrinter) and xml formatters : I had to write my own from scratch,
and every other gtest user has too

The goal of those samples is rather narrow: they demonstrate how to use the API rather than how to do the console output or to write XML files. It will not hurt to point readers to look at XmlUnitTestResultPrinter and PrettyUnitTestResultPrinter as extra examples of writing listeners, though.

e) when writing my xml formatter, I subclassed
XmlUnitTestResultPrinter to reuse some of your code.

Hrm. Feel free to copy XmlUnitTestResultPrinter and the copy in your code (if it works there - I believe it doesn't use any other internal functionality). But using it directly is not advised. It is considered an implementation detail and may be changed at any time in ways that may break your code that depends on it.

I had to remove "private" in XmlUnitTestResultPrinter to do that
though.
And it took some time to make it all work.

Please make it easier for the little guys to write plain/xml
formatters on platforms that may lack features :

I wouldn't have had to rewrite XmlUnitTestResultPrinter if it exported
it's result to a C STRING.
Yet I had to as it exports it to a FILE and as writing/reading files
doesn't work out of the box in the desmume emulator ( and adding file
read/write support to desmume wasn't doable in a reasonnable aount of
time).

You may have noticed we have started the work to rewrite parts of XmlUnitTestResultPrinter to do streaming output. The idea was to replace FILE calls in all methods XmlUnitTestResultPrinter with streaming to std::ostream and then send the output to an std::ofstream object. This would make XmlUnitTestResultPrinter reusable for writing to strings, network, etc. However, that work has not been high on the priority list - it got preempted by other tasks and we have not been able to finish it so far. You can try your hand in producing a patch to finish it. Please see our code contribution guidelines in http://code.google.com/p/googletest/wiki/DevGuide#Contributing_Code if you wish to do that.


I bet I'm not the only one that would like to cross-test with
googletest through emulators (embedded hardware, android, symbian,
windows ce, ...) ?
Best regards,
Olivier Binda
ps : my (un cleaned) code : class Buffer should be really called
FakeStream.

I wonder, why do you feel you need to write your own output stream object rather than use std::stringstream? Is it not available on your platform?
Regards,
Vlad

Olivier Binda

unread,
Aug 27, 2010, 7:17:19 AM8/27/10
to Google C++ Testing Framework
Hullo.

Thanks for your detailed and kind answer.

>
> It's nice to know Google Test is used even on Nintendo!
>

Yeah ^^
I didn't even know if it could be made to work on the Nintendo before
starting to port it.
There is just an illusion of an os on that hardware.

Reading what googletest was about, I really really wanted to have that
and so... I ended up trying to make it work.


> > We have recently updated the code to compile on systems other than those
>
> explicitly recognized in gtest-port.h.

That's nice !

>Try grabbing code from svn head and see if you can compile it without modifications.
>

arhhh svn... gasp...
I'll probably wait for Googletest 1.6.
Or use git/hg to get the latest svn snapshot


> b) stream redirection doesn't work on the nintendo ds, it doesn't have
>
> > dup(), dup2().
> > At some point, gtest complained about it.
>
> > Can't we define explicitely GTEST_HAS_STREAM_REDIRECTION ?
> > (it is automatically defined somewhere else)
>
> This looks like a real problem for you, and require the source code
> modification. I have opened issue
> 305<http://code.google.com/p/googletest/issues/detail?id=305>to track
> the fix. Star it to get updates.


done


>
> > The goal of those samples is rather narrow: they demonstrate how to use the
>
> API rather than how to do the console output or to write XML files. It will
> not hurt to point readers to look at XmlUnitTestResultPrinter
> and PrettyUnitTestResultPrinter as extra examples of writing listeners,
> though.


fair enough

>
> e) when writing my xml formatter, I subclassed> XmlUnitTestResultPrinter to reuse some of your code.
>
> > Hrm. Feel free to copy XmlUnitTestResultPrinter and the copy in your code


I didn't want to do that : there are lots of hairy things when
handling xml (entities, encodings...)
That's why I tried not to get too deep in gory details
Subclassing and overriding the output methods was the best solution I
found to my problem.


>
> You may have noticed we have started the work to rewrite parts
> of XmlUnitTestResultPrinter to do streaming output. The idea was to replace
> FILE calls in all methods XmlUnitTestResultPrinter with streaming to
> std::ostream and then send the output to an std::ofstream object.

nice

>This would
> make XmlUnitTestResultPrinter reusable for writing to strings, network,
> etc. However, that work has not been high on the priority list - it got
> preempted by other tasks and we have not been able to finish it so far. You
> can try your hand in producing a patch to finish it. Please see our code
> contribution guidelines inhttp://code.google.com/p/googletest/wiki/DevGuide#Contributing_Codeif you
> wish to do that.

fair enough.
I can wait anyway (I already have googletest 1.5 up and running, and I
won't repair while it's not broken (I have limited time/energy and am
working on a few other time consuming projects)
>
> I wonder, why do you feel you need to write your own output stream object
> rather than use std::stringstream?

It's because, homebrewed application for the nintendo ds only have 4mo
of ram minus the memory taken by the application.
So you have to somehow save memory and make it count.
Besides, we only have a 66 Mhz arm cpu. So we have to make every tick
count to.


For this reason, the GUI library that I'm using and contributing to
avoids using the std library and reimplements some of its containers
in a simple/basic/efficient/lightweight form (linked lists, hashmaps,
dynamic arrays).

>Is it not available on your platform?

In fact, I have never used the std library, I have only been doing C++
development for the last 3 months (I learn fast and I would rather use
code I write/can modify...that's one of the reason I like gtest).
We are using the devkitarm and gcc 4.5 toolchain. I have seen a
libstdc++.a file as well as some tr1 headers file,
so we should have the std library.

Before using the std library, I would like to know how much memory it
requires for the usage of some of it's module. But last time I looked
for this information, I didn't find it.


Best regards,
Olivier
Reply all
Reply to author
Forward
0 new messages