[boost] Call for Review: Boost.Test documentation rewrite

[Richard]

[Please do not mail me a copy of your followup]

Boost.Test documentation rewrite:
<http://user.xmission.com/~legalize/tmp/boost.test/libs/test/doc/html/>

Please post all comments to this list.

I've been working on this as time permits for a number of months and I
think now things are "finished" enough to warrant wider review. It is
a true rewrite using quickbook; I generally used the implementation
source code as the answer to my questions about things while writing
the documentation.

Early snapshots were reviewed by Tom Kent, Alexander Lamaison,
Paul A. Bristow and Julian Gonggrijp. A special thanks to them for
providing me with useful feedback and suggestions.

The main thing that still remains to be added is a detailed explanation
of the floating-point comparison algorithms used by BOOST_REQUIRE_CLOSE,
BOOST_REQUIRE_CLOSE_FRACTION and BOOST_REQUIRE_SMALL. The corresponding
examples also need to be updated.

The existing online documentation has had all the math symbols
stripped out of it, so it is pretty useless. I'm probably
just going to typeset the formulas with LaTeX and then capture
them as images that are included in the documentation. (See
<https://svn.boost.org/trac/boost/ticket/9539>)

I'm guessing that modular boost should allow me to clone libs/test,
make all my changes there and then issue a pull request, but I'm not
sure how far along things are with the git transition?
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
The Terminals Wiki <http://terminals.classiccmp.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Alexander Lamaison]

legaliz...@mail.xmission.com (Richard) writes:

> [Please do not mail me a copy of your followup]
>
> Boost.Test documentation rewrite:
> <http://user.xmission.com/~legalize/tmp/boost.test/libs/test/doc/html/>

FWIW, I've been using this as my go-to Boost.Test documentation for
some time now. It's a significant improvement on the existing docs.

Alex

--
Swish - Easy SFTP for Windows Explorer (http://www.swish-sftp.org)

[Peter A. Bigot]

This says it documents version 1.22, but I can't tell whether that's
just because Boost.Test isn't independently versioned and the last
content update was in Boost 1.22.

The develop branch has over 200 changes that have accumulated since
2009, and as I understand how Boost releases are done it's this branch
that gets the most road time, even though it isn't what gets released.

Or maybe those changes don't actually affect users at all, though the
develop branch has commits that discuss a new implementation. I'm still
pretty confused about how Boost.Test evolves.

Peter

[Erik Erlandson]

----- Original Message -----
> legaliz...@mail.xmission.com (Richard) writes:
>
> > [Please do not mail me a copy of your followup]
> >
> > Boost.Test documentation rewrite:
> > <http://user.xmission.com/~legalize/tmp/boost.test/libs/test/doc/html/>
>
> FWIW, I've been using this as my go-to Boost.Test documentation for
> some time now. It's a significant improvement on the existing docs.

+1, I've bookmarked it for a reference.

[Paul A. Bristow]

> -----Original Message-----
> From: Boost [mailto:boost-...@lists.boost.org] On Behalf Of Erik Erlandson
> Sent: Friday, January 03, 2014 2:57 PM
> To: bo...@lists.boost.org
> Subject: Re: [boost] Call for Review: Boost.Test documentation rewrite
> ----- Original Message -----
> > legaliz...@mail.xmission.com (Richard) writes:
> >
> > > [Please do not mail me a copy of your followup]
> > >
> > > Boost.Test documentation rewrite:
> > > <http://user.xmission.com/~legalize/tmp/boost.test/libs/test/doc/htm
> > > l/>
> >
> > FWIW, I've been using this as my go-to Boost.Test documentation for
> > some time now. It's a significant improvement on the existing docs.

ditto - a big improvement with a familiar look'n'feel.

It could be AutoIndexed, but this may be less helpful than for other libraries, but at least a
macros listing would be good?

I hope it will be possible to provide links to the worked examples (this is a lot trickier than
normal and probably needs a bit of bjam magic).

> +1, I've bookmarked it for a reference.

Paul

PS But there are still updates in trunk/develop that are yet to be released?

While things are in a state of major change, would not be a good time to try to get Boost.Test
sorted out?

(This might lead to some documentation changes too of course, but at least changes will be easily
made by anyone with any text editor, and rebuilt by anyone with the Quickbook toolchain set up, and
will 'automatically' be added to the main body of Boost docs).

[Richard]

[Please do not mail me a copy of your followup]

bo...@lists.boost.org spake the secret code
<52C6B3E...@pabigot.com> thusly:

>This says it documents version 1.22, but I can't tell whether that's
>just because Boost.Test isn't independently versioned and the last
>content update was in Boost 1.22.

It's the version of Boost.Test -- I think this should be 1.21 instead of
1.22, however. See <http://www.boost.org/doc/libs/1_55_0/>

I originally was working from trunk, but there's a whole bunch of
stuff that is in trunk that isn't in release so with this snapshot I
rescynchronized to the release branch.

>The develop branch has over 200 changes that have accumulated since
>2009, and as I understand how Boost releases are done it's this branch
>that gets the most road time, even though it isn't what gets released.

I really don't yet understand what is going on with this library in
terms of how it is being maintained and enhanced. Unfortunately the
maintainer is not very responsive.

>Or maybe those changes don't actually affect users at all, though the
>develop branch has commits that discuss a new implementation. I'm still
>pretty confused about how Boost.Test evolves.

Yep.

--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
The Terminals Wiki <http://terminals.classiccmp.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

[Richard]

[Please do not mail me a copy of your followup]

bo...@lists.boost.org spake the secret code

<009f01cf089c$3e4ae5d0$bae0b170$@hetp.u-net.com> thusly:

>It could be AutoIndexed, but this may be less helpful than for other
>libraries, but at least a macros listing would be good?

Everything should be accessible at a glance from the main table of
contents. Is there something missing from there?

>PS But there are still updates in trunk/develop that are yet to be released?

Yes, there are a few minor things that I wrote up based on trunk that
I have removed from this version. Then there a whole pile of changes
that the maintainer has not documented and it is unclear which of
these are really ready or not.

>While things are in a state of major change, would not be a good time to
>try to get Boost.Test
>sorted out?

I'd rather document the current 1.55 release and move forward. It's
already taken over 6 months to get to this point, I see no point in
delaying any further.

--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
The Terminals Wiki <http://terminals.classiccmp.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

[Vicente J. Botet Escriba]

Le 03/01/14 18:10, Richard a écrit :

> [Please do not mail me a copy of your followup]
>
> bo...@lists.boost.org spake the secret code
> <52C6B3E...@pabigot.com> thusly:
>
>> This says it documents version 1.22, but I can't tell whether that's
>> just because Boost.Test isn't independently versioned and the last
>> content update was in Boost 1.22.
> It's the version of Boost.Test -- I think this should be 1.21 instead of
> 1.22, however. See <http://www.boost.org/doc/libs/1_55_0/>

This is just the first version of boost where Boost.Test appeared.

Vicente

[Richard]

[Please do not mail me a copy of your followup]

bo...@lists.boost.org spake the secret code

<52C6FB8...@wanadoo.fr> thusly:

>Le 03/01/14 18:10, Richard a �crit :

>> [Please do not mail me a copy of your followup]
>>
>> bo...@lists.boost.org spake the secret code
>> <52C6B3E...@pabigot.com> thusly:
>>
>>> This says it documents version 1.22, but I can't tell whether that's
>>> just because Boost.Test isn't independently versioned and the last
>>> content update was in Boost 1.22.
>> It's the version of Boost.Test -- I think this should be 1.21 instead of
>> 1.22, however. See <http://www.boost.org/doc/libs/1_55_0/>
>This is just the first version of boost where Boost.Test appeared.

Ah, good catch. I'll correct that, thanks.

[Richard]

[Please do not mail me a copy of your followup]

bo...@lists.boost.org spake the secret code

<52C6FB8...@wanadoo.fr> thusly:

>Le 03/01/14 18:10, Richard a �crit :

>> It's the version of Boost.Test -- I think this should be 1.21 instead of
>> 1.22, however. See <http://www.boost.org/doc/libs/1_55_0/>
>This is just the first version of boost where Boost.Test appeared.

I've corrected this.

[Mathieu Champlon]

Hi,

As others have said, this is really a major and much needed improvement
over the existing documentation.
I especially like the little tricks here and there (heck I've been using
Boost.Test for years and I learned a few !) as well as the design
suggestions ("Testing Protected or Private Members") and the big picture
overviews ("Test Case Design and Maintenance").

Here are a few comments, feel free to discard some, most or all of them. :)

* source code links

When clicking on a source code link (any of the "Example source code" of
the tutorials section for instance) my browser (firefox) downloads the
file and attempts to open it using Visual Studio.
I would rather be able to display the source code as html for a quick
look instead.

* tutorials/testing_with_fixtures.html

The first example has a constructor and destructor for the fixture.
While it's explained a little further that the default ones are fine
here (and it should be obvious to a seasoned C++ developer), I fear that
beginners who skim through the documentation will simply copy-paste this
into their fixtures cluttering their code needlessly. Ideally having it
the other way around (start with a simple fixture without
constructor/destructor, then complicating things further) would be
better, but maybe just moving the "In most cases, the setup will be a
little more involved than just creating an instance variable and there
will be some work done in the constructor of the fixture. We could
simply use the default constructor and destructor of our fixture in this
case. We've written the constructor and destructor explicitly to
emphasize that this is where setup and tear down occur. " part into a
"Note" right after the code would prove enough ?

* tutorials/running_selected_tests.html

I find "We can list the available tests in a test executable with
--report_level set to detailed" a bit misleading as it doesn't just list
the tests but still primarily executes them.
Maybe rephrase it to something along the line of "Executed tests can be
listed with --report_level set to detailed when running a test executable" ?

Is there any reason why you didn't mention using
--run_test=hello_world_* first in order to filter on the test case names
instead of jumping into test suites ?
I personally rarely use test suites because I usually use a naming
convention like the one you introduced in a previous tutorial.

* guide/compilation/minimal_header.html

In the "Meaning" column I fear that for instance "the test fails and
test execution terminates" may leave a (perhaps non English native)
reader wondering whether it aborts the current test or the test
application ?

* guide/test_case_design.html

Maybe add a link to http://en.wikipedia.org/wiki/Test-driven_development
to "(...), using test-driven development" ?

The Visual Studio configuration snapshot is a bit big in my opinion :)

Under "Goal: Repeatable Test" a bullet point list doesn't display properly.

* guide/testing_file_io.html

Just for the record I found writing tests with files to be one of the
recurring pitfalls where manipulating actual files does indeed help,
because when abstracting the file system things like re-entrance and
file locking can be easily remain untested.

Any reason for the "extern" when declaring text_files ?

I believe the static for defining ARBITRARY_DIRECTORY_NAME and all isn't
needed. Also the upper case names are usually reserved for macros (in
Boost).

* guide/mocking_collaborators.html

Being the author of turtle I certainly appreciate the spotlight here,
but isn't this part a bit out of Boost.Test documentation scope ?

* guide/testing__main_.html

Again, any reason for the "extern" ? I'm just thinking it might confuse
beginners.

* reference/test_case/boost_test_case_template.html

The fact that 'signed_types' is defined elsewhere is a bit odd
especially since this page comes first when browsing the reference pages
using the "next" button.
Also 'unsigned_types' doesn't show up anywhere as it's simply not included.
Maybe move both to the BOOST_TEST_CASE_TEMPLATE page or add them to both
relevant pages ?

* reference/test_suite/boost_test_suite.html

manual_registration.cpp starts with a "using namespace
boost::unit_test;" which doesn't appear in the documentation making it
difficult to understand where the functions come from.
Note that the same issue appears in
guide/manually_registering_test_cases_and_suites.html which includes the
same code.

* reference/assertion.html

There is a typo in the note e.g. "be defiend" instead of "be defined".

* reference/assertion/*

None of the pages provide links to the source code files, is it on purpose ?

* reference/assertion/_boost____level___close_.html and
reference/assertion/_boost____level___close_fraction_.html

Both have the exact same description and example code. An explanation on
how they differ would certainly prove useful as this is probably one of
the most asked question about Boost.Test on the mailing lists.
Also a note about when to use BOOST_level_SMALL instead would be nice.

* reference/assertion/boost_level_equal.html

In example_equal_custom_compare there's an extra space in the first
initialization which might be confusing.
Also I don't get how the comment is relevant given that operator== uses
std::string to make the comparison ?

* reference/assertion/_boost____level___equal_collections_.html

I believe "static" to be deprecated at a namespace level ?

I would personally rather use
BOOST_REQUIRE_EQUAL_COLLECTIONS(
boost::begin( expected ), boost::end( expected ),
boost::begin( actual ), boost::end( actual ));
which has the clear advantage to cover all the cases, but I understand
the need for more "raw" examples.

* reference/assertion/boost_level_exception.html,
reference/assertion/boost_level_message.html and
reference/assertion/_boost____level___predicate_.html

The "static" aren't really needed, I believe.

* reference/assertion/_boost____level___small_.html

Maybe explain the unit of the tolerance for completeness and coherence
with BOOST_level_CLOSE* ?


Thanks for doing this !

MAT.

[Klaim - Joël Lamotte]

I read more than half the documentation and so far I'm surprised at the
simplicity of the documentation.
It's efficient, even with a few strange problems that others pointed
already.

I'm considering using Boost.Test in one of my OSS projects, see how it goes.

[Richard]

[Please do not mail me a copy of your followup]

Thank you Mathieu for your detailed feedback.

Mathieu Champlon <m.cha...@free.fr> spake the secret code
<52CA700F...@free.fr> thusly:

>* source code links
>
>When clicking on a source code link (any of the "Example source code" of
>the tutorials section for instance) my browser (firefox) downloads the
>file and attempts to open it using Visual Studio.
>I would rather be able to display the source code as html for a quick
>look instead.

I think when the code gets deployed to the web site, you end up seeing
a browser-ified version of the source code. For instance, the
existing links to source files appear this way on the web site, but I
cannot find any indiciation that the quickbook source for other
libraries, or the existing non-quicbook documentation for boost.test,
does anything special to provide this.

>* tutorials/testing_with_fixtures.html
>
>The first example has a constructor and destructor for the fixture.
>While it's explained a little further that the default ones are fine
>here (and it should be obvious to a seasoned C++ developer), I fear that
>beginners who skim through the documentation will simply copy-paste this
>into their fixtures cluttering their code needlessly. Ideally having it
>the other way around (start with a simple fixture without
>constructor/destructor, then complicating things further) would be
>better, but maybe just moving the "In most cases, the setup will be a
>little more involved than just creating an instance variable and there
>will be some work done in the constructor of the fixture. We could
>simply use the default constructor and destructor of our fixture in this
>case. We've written the constructor and destructor explicitly to
>emphasize that this is where setup and tear down occur. " part into a
>"Note" right after the code would prove enough ?

So if I might paraphrase -- you're saying to display best practices
first and make a note of where the setup/tear down code would appear?

>* tutorials/running_selected_tests.html
>
>I find "We can list the available tests in a test executable with
>--report_level set to detailed" a bit misleading as it doesn't just list
>the tests but still primarily executes them.

Yes, the trunk version of boost.test has a specific option for listing
the tests called --list_content and not executing them. I'm not a fan
of the name of this argument, but it is present on trunk and not on
release. When I originally wrote this documentation it was based on
trunk but I rebased it on release in order to get it moving forward.

>Maybe rephrase it to something along the line of "Executed tests can be
>listed with --report_level set to detailed when running a test executable" ?

I'll reword this to say that the tests are executed as well as test
names shown.

>Is there any reason why you didn't mention using
>--run_test=hello_world_* first in order to filter on the test case names
>instead of jumping into test suites ?

I was using this as a way to introduce test suites :-).

>I personally rarely use test suites because I usually use a naming
>convention like the one you introduced in a previous tutorial.

If you prefer that style of test organization, was the explanation in
this tutorial good enough for you to see how to use that style?

>* guide/compilation/minimal_header.html
>
>In the "Meaning" column I fear that for instance "the test fails and
>test execution terminates" may leave a (perhaps non English native)
>reader wondering whether it aborts the current test or the test
>application ?

I can reword this for more clarification.

>* guide/test_case_design.html
>
>Maybe add a link to http://en.wikipedia.org/wiki/Test-driven_development
>to "(...), using test-driven development" ?

Good idea. More linking never hurts.

>The Visual Studio configuration snapshot is a bit big in my opinion :)

It is native size. Were you thinking that providing a smaller,
filtered version of the screen shot with the image being a link to the
native size version?

>Under "Goal: Repeatable Test" a bullet point list doesn't display properly.

Ah. Good catch.

>* guide/testing_file_io.html
>
>Just for the record I found writing tests with files to be one of the
>recurring pitfalls where manipulating actual files does indeed help,
>because when abstracting the file system things like re-entrance and
>file locking can be easily remain untested.

This is why I talk about acceptance testing being an integral part of
any plan for automated testing. Unit tests alone are not enough
because they exercise pieces of the system in isolation; you still
need some sort of automated tests that bring all the pieces together
and exercise them as a whole.

>Any reason for the "extern" when declaring text_files ?

It is a function with external linkage. I don't want to declare it as
a local function, but one that is provided externally.

Do you feel that this is a leftover "C"-ism?

>I believe the static for defining ARBITRARY_DIRECTORY_NAME and all isn't
>needed. Also the upper case names are usually reserved for macros (in
>Boost).

That may simply be a matter of style -- I picked this up from "Modern
C++ Programming with Test-Driven Development" where he advises that
using named constants for arbitrary inputs helps make it clear that
the actual value used in the test isn't important so much as how that
value appears in inputs and expected outputs.

Regarding the casing of the identifier, I was falling back on
constants being all caps, but I'll consult the boost style guidelines
and adjust per their recommendations.

>* guide/mocking_collaborators.html
>
>Being the author of turtle I certainly appreciate the spotlight here,
>but isn't this part a bit out of Boost.Test documentation scope ?

It is, but mocking objects is too important to be left unmentioned.
Since Turtle is designed to work directly with Boost.Test I wanted to
give it a mention.

>* guide/testing__main_.html
>
>Again, any reason for the "extern" ? I'm just thinking it might confuse
>beginners.

Same as above.

>* reference/test_case/boost_test_case_template.html
>
>The fact that 'signed_types' is defined elsewhere is a bit odd
>especially since this page comes first when browsing the reference pages
>using the "next" button.
>Also 'unsigned_types' doesn't show up anywhere as it's simply not included.
>Maybe move both to the BOOST_TEST_CASE_TEMPLATE page or add them to both
>relevant pages ?

OK, I'll expand this example to be more complete when viewed
standalone.

>* reference/test_suite/boost_test_suite.html
>
>manual_registration.cpp starts with a "using namespace
>boost::unit_test;" which doesn't appear in the documentation making it
>difficult to understand where the functions come from.

OK. I'll revise the examples to avoid using namespaces in order to
make it more obvious where functions originate.

>* reference/assertion.html
>
>There is a typo in the note e.g. "be defiend" instead of "be defined".

Good catch. I will run a spell checker on everything to make sure I
find any other typos like this.

>* reference/assertion/*
>
>None of the pages provide links to the source code files, is it on purpose ?

I thought the example snippets as they stood would be sufficient. I
can certainly link to the example source file.

>* reference/assertion/_boost____level___close_.html and
>reference/assertion/_boost____level___close_fraction_.html
>
>Both have the exact same description and example code. An explanation on
>how they differ would certainly prove useful as this is probably one of
>the most asked question about Boost.Test on the mailing lists.
>Also a note about when to use BOOST_level_SMALL instead would be nice.

The floating-point comparison is the one area where I still need to
write something real instead of just a placeholder. This is the last
area that I have yet to finish.

>* reference/assertion/boost_level_equal.html
>
>In example_equal_custom_compare there's an extra space in the first
>initialization which might be confusing.

Which space are you referring to?

The space between the definition of the string s and the first assertion?

>Also I don't get how the comment is relevant given that operator== uses
>std::string to make the comparison ?

The standard library provides operator== and operator<< for
std::string, so BOOST_REQUIRE_EQUAL can be used without requiring you
to define these.

For your own custom type you'll have to define them.

>* reference/assertion/_boost____level___equal_collections_.html
>
>I believe "static" to be deprecated at a namespace level ?

Static linkage to a function means that it is only visible within that
single compilation unit. However, since generate_list is the system
under test, this may be a little confusing. I could separate it into
another compilation unit and give it a head, but then the whole
example becomes much more complicated being spread across three source
files, a new static library and an executable. For the purposes of
making a small illustrative example I felt that was overkill.

>I would personally rather use
> BOOST_REQUIRE_EQUAL_COLLECTIONS(
> boost::begin( expected ), boost::end( expected ),
> boost::begin( actual ), boost::end( actual ));
>which has the clear advantage to cover all the cases, but I understand
>the need for more "raw" examples.

There is a feature request from 4 years ago(!) for exactly this:
<https://svn.boost.org/trac/boost/ticket/3871>

I recently tried to read the Boost.Range documentation and I confess
that I didn't exaclty find it accessible at a quick read. In this
documentation I wanted it to be as standalone as possible and not drag
the reader unnecessarily into other libraries. In this case I think
that the vast majority of C++ programmers are not going to have a
problem with container.begin() or container.end() whereas many may not
have seen boost::begin(container) and boost::end(container).

Is the motivation for this suggestion to guide C++ programmers more
towards C++11 style usage where std::begin() and std::end() are
available? I can see how their use for fixed length arrays is
preferable over the old "sizeof(array)/sizeof(array[0])" business, but
how exactly is it preferable for containers where we have begin() and
end() member functions?

>* reference/assertion/boost_level_exception.html,
>reference/assertion/boost_level_message.html and
>reference/assertion/_boost____level___predicate_.html
>
>The "static" aren't really needed, I believe.

See above. They are given static linkage to intentionally limit the
accessibility of their names.

>* reference/assertion/_boost____level___small_.html
>
>Maybe explain the unit of the tolerance for completeness and coherence
>with BOOST_level_CLOSE* ?

This is part of the remaining area that needs to be done.

--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
The Terminals Wiki <http://terminals.classiccmp.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

[Mathieu Champlon]

On 06/01/2014 20:07, Richard wrote:
>> * tutorials/testing_with_fixtures.html
>>
>> (...)

> So if I might paraphrase -- you're saying to display best practices
> first and make a note of where the setup/tear down code would appear?

Yes.

>> * tutorials/running_selected_tests.html
>> (...)

>> I personally rarely use test suites because I usually use a naming
>> convention like the one you introduced in a previous tutorial.
> If you prefer that style of test organization, was the explanation in
> this tutorial good enough for you to see how to use that style?

The idea of using wildcards only appears at about 2/3 of the page and it
looks almost as if it were a feature of test suites because the two
sentences are on the same line:

"The names of the enclosing suites are separated from each other and
from the test case name with a slash (/). The names supplied to
--run_test also allow for wildcard matching (...)"

To me both test suites and wildcards are equally important and maybe
they could be introduced at the same time, for instance rephrasing

"We can solve this problem by using test suites, which organize tests
into a named group"

to incorporate wildcards as well, e.g. something along

"We can solve this problem by either using wildcards with --run_test
when test names follow a regular naming pattern:
<example>
or by using test suites, which organize tests into a named group (...)"

>> * guide/test_case_design.html
>>
>> (...)

>> The Visual Studio configuration snapshot is a bit big in my opinion :)
> It is native size. Were you thinking that providing a smaller,
> filtered version of the screen shot with the image being a link to the
> native size version?

It's just the image is quite big and the only useful information are the
two lines at the top.
But I know the window cannot be resized and downsizing the screen shot
itself will probably reduce the quality.
No big deal, never mind.

>> * guide/testing_file_io.html
>>
>> (...)

>>
>> Any reason for the "extern" when declaring text_files ?
> It is a function with external linkage. I don't want to declare it as
> a local function, but one that is provided externally.
>
> Do you feel that this is a leftover "C"-ism?

I don't think there is any reason to use the extern keyword at all, see
for instance
http://stackoverflow.com/questions/11712707/extern-functions-in-c-vs-c

>> * reference/assertion/boost_level_equal.html
>>
>> In example_equal_custom_compare there's an extra space in the first
>> initialization which might be confusing.
> Which space are you referring to?

symbol const s1 = { "var" , 1 };
symbol const s2 = { "var", 1 };

There is an extra space at the right of "var" for s1 compared to s2.

>> Also I don't get how the comment is relevant given that operator== uses
>> std::string to make the comparison ?
> The standard library provides operator== and operator<< for
> std::string, so BOOST_REQUIRE_EQUAL can be used without requiring you
> to define these.
>
> For your own custom type you'll have to define them.

I meant about the comment in:

symbol const s1 = { "var" , 1 };
symbol const s2 = { "var", 1 };
// If the compiler doesn't collapse multiple constants, then:
// s1.name != s2.name;
BOOST_REQUIRE_EQUAL(s1, s2);

While true, I don't see how that's relevant.
Or maybe you wanted to explain why operator== uses std::string instead
of comparing 'name' pointers?

static bool operator==(symbol const& lhs, symbol const& rhs)
{
return lhs.value == rhs.value
&& std::string(lhs.name) == std::string(rhs.name);
}

in that case maybe move the comment there?

>> I would personally rather use
>> BOOST_REQUIRE_EQUAL_COLLECTIONS(
>> boost::begin( expected ), boost::end( expected ),
>> boost::begin( actual ), boost::end( actual ));
>> which has the clear advantage to cover all the cases, but I understand
>> the need for more "raw" examples.
> There is a feature request from 4 years ago(!) for exactly this:
> <https://svn.boost.org/trac/boost/ticket/3871>
>

> (...)

>
> Is the motivation for this suggestion to guide C++ programmers more
> towards C++11 style usage where std::begin() and std::end() are
> available?
>

Yes, but a more proper solution may be to introduce a
BOOST_CHECK_EQUAL_RANGE similar to the one described in the ticket you
linked.

> I can see how their use for fixed length arrays is
> preferable over the old "sizeof(array)/sizeof(array[0])" business, but
> how exactly is it preferable for containers where we have begin() and
> end() member functions?
>

For the sake of uniformity I suppose, but don't bother the examples are
fine as they are.

Thanks !
MAT.

[Beman Dawes]

On Fri, Jan 3, 2014 at 12:13 PM, Richard
<legaliz...@mail.xmission.com>wrote:

> I'd rather document the current 1.55 release and move forward. It's
> already taken over 6 months to get to this point, I see no point in
> delaying any further.
>

With all reviews in favor and none against, it seems clear you should move
forward with this.

Please open an admin issue asking for Boost.Test push + pull permission.

See https://svn.boost.org/trac/boost/wiki/ModAdmin

Thanks for taking this on!

--Beman

[Rob Stewart]

On Jan 7, 2014, at 8:19 AM, Mathieu Champlon <m.cha...@free.fr> wrote:

> On 06/01/2014 20:07, Richard wrote:
>
>>> * guide/test_case_design.html
>>>
>>> (...)
>>> The Visual Studio configuration snapshot is a bit big in my opinion :)
>> It is native size. Were you thinking that providing a smaller, filtered version of the screen shot with the image being a link to the native size version?
>
> It's just the image is quite big and the only useful information are the two lines at the top.
> But I know the window cannot be resized and downsizing the screen shot itself will probably reduce the quality.

Perhaps cropping the bottom from it would help?

___
Rob

(Sent from my portable computation engine)

[Richard]

[Please do not mail me a copy of your followup]

bo...@lists.boost.org spake the secret code

<24E237EE-C80E-40F9...@comcast.net> thusly:

>On Jan 7, 2014, at 8:19 AM, Mathieu Champlon <m.cha...@free.fr> wrote:
>> It's just the image is quite big and the only useful information are
>the two lines at the top.
>> But I know the window cannot be resized and downsizing the screen shot
>itself will probably reduce the quality.
>
>Perhaps cropping the bottom from it would help?

I know how to take care of this; it will be reflected in the next
update after I incorporate the feedback on this thread.

--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
The Terminals Wiki <http://terminals.classiccmp.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

[Gennadiy Rozental]

Richard <legalize+jeeves <at> mail.xmission.com> writes:

Hi Richard,

Unfortunately I am very busy with moving to a new position to review this
properly, but I feel I need to make few comments. We already kinda went
through this couple times, so these should not be a surprise.

I generally do not mind your style or tools used even though both are not my
favorite. Here are the things that I find I am NOT ok with:

1. First and foremost: it is pretty much completely irrelevant
Trunk version of boost.test will require us to rewrite 90% of what you did.
I keep asking you to collaborate on something which we can actually release.
I am not sure why you keep ignoring me.

2. This is not a Boost.Test documentation - this is is boost unit test
framework documentation

You are missing the whole original point of layering in boost.test design.
You are missing description of all the other independent layers. In general
I think you are missing quite a few other things as well.

3. You eliminated the examples. I liked original examples much better with
their output etc.

4. You made quite a lot of description more terse

Take runtime parameters for example. Where I had few pages you left only few
paragraphs. I find new description very lacking.

5. I understand you have your own favorite mocking library, but it is not
part of Boost.Test (not yet at least). In fact Boost.Test does have mocking
components, which just never been properly documented. The library you
promote in fact is an example of mocking approach, which I am strongly
against. That said I do not mind pages with tutorial on using any kind of
mocking library, but I would probably start with gmock instead.

6. I think quickbook navigation is seriously lacking. If you do not like my
take, I would like to see something along the lines of what preprocessor
library is using.

These are just from top of my head without properly reading the pages. I
once again would like you to consider collaboration. I may not have enough
time at the moment to finish this myself, but together (and probably with
help of some other people) I am sure we can make it happen.

Regards,
Gennadiy

[Gennadiy Rozental]

Beman Dawes <bdawes <at> acm.org> writes:

>
> On Fri, Jan 3, 2014 at 12:13 PM, Richard

> <legalize+jeeves <at> mail.xmission.com>wrote:

>
> > I'd rather document the current 1.55 release and move forward. It's
> > already taken over 6 months to get to this point, I see no point in
> > delaying any further.
> >
>
> With all reviews in favor and none against, it seems clear you should move
> forward with this.

I hoped I still have some say in this. I outlined my concerns in another
post. I would prefer not to make any movement until we get to some consensus
on these.

Regards,
Gennadiy

[Beman Dawes]

On Thu, Jan 9, 2014 at 3:25 AM, Gennadiy Rozental <rog...@gmail.com> wrote:

> Beman Dawes <bdawes <at> acm.org> writes:
>
> >
> > On Fri, Jan 3, 2014 at 12:13 PM, Richard
> > <legalize+jeeves <at> mail.xmission.com>wrote:
> >
> > > I'd rather document the current 1.55 release and move forward. It's
> > > already taken over 6 months to get to this point, I see no point in
> > > delaying any further.
> > >
> >
> > With all reviews in favor and none against, it seems clear you should
> move
> > forward with this.
>
> I hoped I still have some say in this. I outlined my concerns in another
> post. I would prefer not to make any movement until we get to some
> consensus
> on these.
>

Gennadiy,

1) The library hasn't had much attention for several years, so I'm
personally against further delay. While normally we wouldn't think of going
forward with a major docs rewrite against the wishes of the library
maintainer, you just haven't been responsive to documentation complaints
for what seems like eons of time.

2) One of the objectives of modularizing boost was so that we could
encourage wider participation, yet avoid giving boost-wide write
authorization to new contributors. It seems to me the work Richard is doing
is a very useful example of that.

3) Some of your complains, such as concerns about poor coverage of
layering, may be valid but shouldn't prevent moving Richard's stuff into
develop. We can make sure any issues are resolved before merging to master.

Since you have objected, I will poll the other release managers privately
before we move ahead.

--Beman

[Alexander Lamaison]

Gennadiy Rozental <rog...@gmail.com> writes:

> Richard <legalize+jeeves <at> mail.xmission.com> writes:
>
> Unfortunately I am very busy with moving to a new position to review this
> properly, but I feel I need to make few comments. We already kinda went
> through this couple times, so these should not be a surprise.
>
> I generally do not mind your style or tools used even though both are not my
> favorite. Here are the things that I find I am NOT ok with:
>
> 1. First and foremost: it is pretty much completely irrelevant
> Trunk version of boost.test will require us to rewrite 90% of what you did.
> I keep asking you to collaborate on something which we can actually release.
> I am not sure why you keep ignoring me.

Richard didn't ignore you. He explained that he documented the version
of Boost.Test that users are actually using. The version that ends up
in the releases. It's no use to the users if the documentation
describes what may happen at some unspecified point in the future, but
doesn't match the code in front of them.

> 2. This is not a Boost.Test documentation - this is is boost unit test
> framework documentation
>
> You are missing the whole original point of layering in boost.test design.
> You are missing description of all the other independent layers. In general
> I think you are missing quite a few other things as well.

The problem with the original documentation was that so much of it was
about how the UTF was implemented. No-one cares. Users only want to
know how to use it to write unit test with Boost.Test. Back in the day
I had to learn that from Richard's blog.

I could respond to each of your criticism but I think I'll just end by
saying that, if a team have scrutinised this documentation for months,
finding it greatly improved, and the user reception is overwhelmingly
positive, perhaps Richard has some skills in the documentation area.

Alex

--
Swish - Easy SFTP for Windows Explorer (http://www.swish-sftp.org)

[Richard]

[Please do not mail me a copy of your followup]

bo...@lists.boost.org spake the secret code

<loom.2014010...@post.gmane.org> thusly:

>Richard <legalize+jeeves <at> mail.xmission.com> writes:
>
>1. First and foremost: it is pretty much completely irrelevant
>Trunk version of boost.test will require us to rewrite 90% of what you did.

I disagree. I wrote 99% of everything here based on trunk which still
has all the stuff I documented -- and must have all of that in order
to be backwards compatible with release.

What you have done in trunk is add an unknown quantity of new,
untested features -- untested in the sense that noone is using them so
they have no user feedback as to the quality or usefulness of their
implementation. You haven't documented them and you haven't solicited
anyone to perform any pre-release evaluation of those features.

I don't feel compelled to document something for which there appears
to be no design documentation, no statement of completeness, no
rationale for existing and has unknown amounts of bugs in it because
noone is really using it.

>I keep asking you to collaborate on something which we can actually release.
>I am not sure why you keep ignoring me.

Funny, I could say the same thing. I wrote my tutorial blog posts 4
years ago:

<http://article.gmane.org/gmane.comp.lib.boost.user/49092>
<http://article.gmane.org/gmane.comp.lib.boost.user/49110>
<http://article.gmane.org/gmane.comp.lib.boost.user/49111>
<http://article.gmane.org/gmane.comp.lib.boost.user/49113>
<http://article.gmane.org/gmane.comp.lib.boost.user/49114>

You responded saying that you would include a link to those tutorials
in the documentation:
<http://article.gmane.org/gmane.comp.lib.boost.user/49151>

That never happened.

On May 1st, 2013 I asked for early reviewers of this rewrite:
<http://article.gmane.org/gmane.comp.lib.boost.devel/240855>

You never replied to that, either in the list/newsgroup or in email.
All the people that replied were added to my list of reviewers and
have been getting updated snapshots along the way: Tom Kent, Alexander
Lamaison, Paul A. Bristow and Julian Gonggrijp (apologies if I
misspelled any of your names). I went through at least 7 iterations
with those early reviewers, incorporating their feedback along the
way.

There was ample time for you to engage me at any step along the way.

In general, I have found you unresponsive to either questions from
myself or from others on the mailing list. I would post questions
to the mailing list via the newsgroup on gmane.org and not get any
answer from you. I have also sent you email to which you never
responded.

I found documentation bugs in the bug tracker that had been sitting
there for 4 years with no movement. (I fixed them all in my rewrite.)

A good rule of thumb when you attempt to explain a technology is to
make sure that you can answer any questions that other people have
about that technology. This is the approach I took for multiple years
with regards to Direct3D while I was writing my (free) book on the
subject. I have experience in explaining APIs to a programmer audience.
<http://legalizeadulthood.wordpress.com/the-direct3d-graphics-pipeline/>

Questions get posted about Boost.Test on the user or developer mailing
lists and go unanswered by you and given the current state of affairs,
you were pretty much the only one that could answer them.

Lately, I have been answering questions about Boost.Test on the
mailing list, which has helped me to make sure that I have the
appropriate information in the documentation. Ideally the
documentation should be good enough that people don't need to ask a
quesiton on the mailing list because they found the answer in the
documentation.

Examples:
<http://article.gmane.org/gmane.comp.lib.boost.devel/247897>
<http://article.gmane.org/gmane.comp.lib.boost.user/80578>
<http://article.gmane.org/gmane.comp.lib.boost.user/80577>
<http://article.gmane.org/gmane.comp.lib.boost.user/80575>
<http://article.gmane.org/gmane.comp.lib.boost.user/80574>
<http://article.gmane.org/gmane.comp.lib.boost.user/80341>
<http://article.gmane.org/gmane.comp.lib.boost.user/79220>
<http://thread.gmane.org/gmane.comp.lib.boost.user/78884/focus=78927>
<http://article.gmane.org/gmane.comp.lib.boost.user/78791>
...and more if you keep searching.

>2. This is not a Boost.Test documentation - this is is boost unit test
>framework documentation

I find this to be a distinction that is both confusing and unnecessary
to users of Boost.Test. Similarly, I removed all reference to "test
tools" because this terminology is non-informative (tool is too
generic) and not consistent with the terminology generally in use --
the industry calls these programming constructs assertions and I see no
reason why Boost.Test should deviate from industry standard terminology.

>You are missing the whole original point of layering in boost.test design.

In 4+ years of using Boost.Test, I haven't found that I ever cared
about this. When I talk to other people about the existing
documentation, they found all this discussion of these layers to be
distracting and irrelevant. So I treated them as an internal
implementation detail that users didn't need to know about.

>You are missing description of all the other independent layers. In general
>I think you are missing quite a few other things as well.

<shrug> In my opinion, what remains is what is important and
necessary in order to use this library for unit testing.

>3. You eliminated the examples. I liked original examples much better with
>their output etc.

I didn't eliminate examples, I replaced them. Everything that you
need to do as a user of this library has an example. For cases where
the output is important, the output is present in the documentation.
I don't see much utility in demonstrating the kind of message you get
from every single assertion when it fails. I could be convinced
otherwise, but as a user of this library for 4+ years I haven't needed
it. You simply look at the failed message and you know what to do
without reading documentation.

>4. You made quite a lot of description more terse

This was intentional. The existing documentation used pages to
describe what could be explained in a paragraph or two, particularly
for the reference section.

>Take runtime parameters for example. Where I had few pages you left only few
>paragraphs. I find new description very lacking.

Yet other reviewers have not commented on this at all. That tells me
that the extra detail simply isn't needed.

>5. I understand you have your own favorite mocking library, but it is not
>part of Boost.Test (not yet at least).

Correct, but once you start practicing TDD in C++ you quickly get to
the point where you need a mocking library. Turtle is designed to
work seamlessly with Boost.Test and is proceeding towards being
submitted to Boost. There are other mocking libraries out there and
I'm happy to link to them as well from the documentation, but the ones
that I've used in the past required more boiler-plate than Turtle.
gmock can no longer be used with Boost.Test because using gmock now
requires the use of gtest. MockCpp requires more boilerplate and
hasn't been updated since Oct 2011. There are probably others; while
I have a personal preference for Turtle that doesn't mean I'm adverse
to linking to any other mock libraries.

>6. I think quickbook navigation is seriously lacking.

Specific suggestions are welcome.

--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
The Terminals Wiki <http://terminals.classiccmp.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

[Gennadiy Rozental]

Alexander Lamaison <awl03 <at> doc.ic.ac.uk> writes:

> > I keep asking you to collaborate on something which we
> > can actually release.
> > I am not sure why you keep ignoring me.

> Richard didn't ignore you. He explained that he documented the version
> of Boost.Test that users are actually using. The version that ends up
> in the releases.

The new version that needs to be releases IS the one that will appear in a
release and the one which is significantly different.

> > It's no use to the users if the documentation
> describes what may happen at some unspecified point in the future, but
> doesn't match the code in front of them.

It is not some unspecified time in a future. New version is ready and just
need to be fitted with new documentation.

> > 2. This is not a Boost.Test documentation - this is is boost unit test
> > framework documentation
> >
> > You are missing the whole original point of layering in boost.test
design.
> > You are missing description of all the other independent layers. In
general
> > I think you are missing quite a few other things as well.
>
> The problem with the original documentation was that so much of it was
> about how the UTF was implemented. No-one cares. Users only want to
> know how to use it to write unit test with Boost.Test. Back in the day
> I had to learn that from Richard's blog.

You can probably have number of complains about existing implementation.
This is NOT one of them. If anything it was lacking reference part and
Richard's version improve on that. Would you care to show an example?

> I could respond to each of your criticism but I think I'll just end by
> saying that, if a team have scrutinised this documentation for months,
> finding it greatly improved, and the user reception is overwhelmingly
> positive, perhaps Richard has some skills in the documentation area.

I am not argue his skills. I just want to apply them in a right direction.

Gennadiy

[Gennadiy Rozental]

Richard <legalize+jeeves <at> mail.xmission.com> writes:

>
> [Please do not mail me a copy of your followup]
>

> boost <at> lists.boost.org spake the secret code
> <loom.20140109T090205-364 <at> post.gmane.org> thusly:

>
> >Richard <legalize+jeeves <at> mail.xmission.com> writes:
> >
> >1. First and foremost: it is pretty much completely irrelevant
> >Trunk version of boost.test will require us to rewrite 90% of what you
did.
>
> I disagree. I wrote 99% of everything here based on trunk

No. It is not.

> which still has all the stuff I documented -- and must have all of that
> in order to be backwards compatible with release.

Sure it will. But this is not what we want users to use. New version has
much better tools and interfaces, more runtime parameters, new subsystems
etc.

> What you have done in trunk is add an unknown quantity of new,
> untested features -- untested in the sense that noone is using them so
> they have no user feedback as to the quality or usefulness of their
> implementation. You haven't documented them and you haven't solicited
> anyone to perform any pre-release evaluation of those features.

The plan was to write docs and ask for review. We can't ask anyone to review
new library features without some form of documentation. Otherwise all the
features are tested - I have unit tests for them.

> I don't feel compelled to document something for which there appears
> to be no design documentation, no statement of completeness, no
> rationale for existing and has unknown amounts of bugs in it because
> noone is really using it.

You did not feel compelled to ask the question. Would you care to point me
to a single boost library developer, who wrote design document before
implementing library changes? And do you think if I would be one like this,
I would not finish whole docs by now?

The procedure is to make the changes, test them, write some docs. Ask for
review if changes are significant enough. I hoped we can collaborate on last
part parts. Before library is out there, there obviously will be no one

using it.

> >I keep asking you to collaborate on something which we can actually
release.
> >I am not sure why you keep ignoring me.
>
> Funny, I could say the same thing. I wrote my tutorial blog posts 4

> You responded saying that you would include a link to those tutorials
> in the documentation:
> <http://article.gmane.org/gmane.comp.lib.boost.user/49151>
>
> That never happened.

You are not the only one. I have a long list of links with references to
various blogs, web pages etc. I still plan to have a separate page in docs,
with all the "web wisdom" mentioned. That said, documenting new features is
still a priority.

> On May 1st, 2013 I asked for early reviewers of this rewrite:
> <http://article.gmane.org/gmane.comp.lib.boost.devel/240855>
>
> You never replied to that, either in the list/newsgroup or in email.
> All the people that replied were added to my list of reviewers and
> have been getting updated snapshots along the way: Tom Kent, Alexander
> Lamaison, Paul A. Bristow and Julian Gonggrijp (apologies if I
> misspelled any of your names). I went through at least 7 iterations
> with those early reviewers, incorporating their feedback along the
> way.

I did not reply because in our previous conversion (few month before that) I
ask for you to work on a new version instead of spending time on old one and
you ignored me. I did not see a reason to waste time reviewing something
which is no help for new release.

> I have also sent you email to which you never responded.

Where did you send it? and when? I obviously expressed interest in talking
with you. All people who are interested, can reach me easily.

> I found documentation bugs in the bug tracker that had been sitting
> there for 4 years with no movement. (I fixed them all in my rewrite.)

Most of them are already fixed in a trunk version of Boost.Test docs.

> Questions get posted about Boost.Test on the user or developer mailing
> lists and go unanswered by you and given the current state of affairs,
> you were pretty much the only one that could answer them.

There are few threads from last several month in my "to answer" bin indeed.
I am very busy and not always can get to it right away. If question is
answered fine by someone else, I am not going to chip in second time.

> >2. This is not a Boost.Test documentation - this is is boost unit test
> >framework documentation
>
> I find this to be a distinction that is both confusing and unnecessary
> to users of Boost.Test. Similarly, I removed all reference to "test

I do not think you can make this decision.

> tools" because this terminology is non-informative (tool is too
> generic) and not consistent with the terminology generally in use --
> the industry calls these programming constructs assertions and I see no
> reason why Boost.Test should deviate from industry standard terminology.

This is fine. Especially since in a new version most of it will come to a
single type of assertion.

> >You are missing the whole original point of layering in boost.test
design.
>
> In 4+ years of using Boost.Test, I haven't found that I ever cared
> about this.

Well, you are not the only user of the library. There are lot more people
using these features than you think. UTF is obviously the most used part of
Boost.Test, but not the only one.

> When I talk to other people about the existing
> documentation, they found all this discussion of these layers to be
> distracting and irrelevant.

Those 5 people you were talking to are not the only users of the library.

> So I treated them as an internal
> implementation detail that users didn't need to know about.

It is not. It is part of public interface and needs to be properly
documented.

> >You are missing description of all the other independent layers. In
general
> >I think you are missing quite a few other things as well.
>
> <shrug> In my opinion, what remains is what is important and
> necessary in order to use this library for unit testing.

In my opinion - it's not.

> >3. You eliminated the examples. I liked original examples much better
with
> >their output etc.
>
> I didn't eliminate examples, I replaced them. Everything that you
> need to do as a user of this library has an example. For cases where
> the output is important, the output is present in the documentation.
> I don't see much utility in demonstrating the kind of message you get
> from every single assertion when it fails. I could be convinced
> otherwise, but as a user of this library for 4+ years I haven't needed
> it. You simply look at the failed message and you know what to do
> without reading documentation.

In my experience of supporting this library (which is much longer)expected
output is always a good thing. I would rather keep all of these.

> >4. You made quite a lot of description more terse
>
> This was intentional. The existing documentation used pages to
> describe what could be explained in a paragraph or two, particularly
> for the reference section.

Well, this is a matter of opinion. I am willing to discuss this.

> >Take runtime parameters for example. Where I had few pages you left only
few
> >paragraphs. I find new description very lacking.
>
> Yet other reviewers have not commented on this at all. That tells me
> that the extra detail simply isn't needed.

You reviewers are already experienced users of the library. They may not
have an unbiased view on this.

> >5. I understand you have your own favorite mocking library, but it is not
> >part of Boost.Test (not yet at least).
>
> Correct, but once you start practicing TDD in C++ you quickly get to
> the point where you need a mocking library. Turtle is designed to
> work seamlessly with Boost.Test and is proceeding towards being
> submitted to Boost. There are other mocking libraries out there and
> I'm happy to link to them as well from the documentation, but the ones
> that I've used in the past required more boiler-plate than Turtle.
> gmock can no longer be used with Boost.Test because using gmock now
> requires the use of gtest. MockCpp requires more boilerplate and
> hasn't been updated since Oct 2011. There are probably others; while
> I have a personal preference for Turtle that doesn't mean I'm adverse
> to linking to any other mock libraries.

Let's review this separately. I have a preference to what I already have on
Boost.Test obviously. In any case We should mention all possible candidates
(including older version of gmock if if is applicable).

> >6. I think quickbook navigation is seriously lacking.
>
> Specific suggestions are welcome.

Didn't you just delete one: let's consider something along the lines of
Boost.Preprocessor.

Regards,
Gennadiy