Adding your own command line options

[Mateusz Łoskot]

Hi,

Is there any mechanism to pass arguments to the code under test?

For example, what's the canonical way to pass arguments like
a database connection string?

I'm reading the own-main.md [1] and there is a section announcing
the custom command line options feature.
It is not available yet, is it?

[1] https://github.com/philsquared/Catch/blob/master/docs/own-main.md

Best regards,
--
Mateusz  Loskot, http://mateusz.loskot.net

[Mateusz Loskot]

bump

> --
> You received this message because you are subscribed to the Google Groups
> "CATCH" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to catch-forum...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Phil Nash]

On Wednesday, 22 April 2015 11:24:25 UTC+1, Mateusz Łoskot wrote:

Hi,

Is there any mechanism to pass arguments to the code under test?

For example, what's the canonical way to pass arguments like
a database connection string?

I'm reading the own-main.md [1] and there is a section announcing
the custom command line options feature.
It is not available yet, is it?

You can do it but it's not as simple as I'd like just yet - which is why the documentation trails off a bit there.

There's actually two ways you could do it - both involving you supplying your own main(), as in that doc page:

(1)  Before you hand off to Catch parse the args however you like and extract the args you're interested in. Pass the rest into Catch. You could pass them all into Catch at this point but it will warn about unknown args.

(2) There is a way, which I don't have the details of to hand right now (can get back to you with this if you're interested) where you can pass the arguments into Catch at a lower level and tell it to return unused args to you, instead of complaining. You'll get them back in a pre-parsed form (this is actually Clara - the self-contained argument parser). You can then feed these back into Clara with your own arg spec (which also needs documenting - but you can probably piece together from the examples at https://github.com/philsquared/Clara

Sorry that's a bit vague - but may give you enough of an idea of which direction you want to go in from here...

[Mateusz Loskot]

On 5 May 2015 at 07:31, Phil Nash <pan.e...@gmail.com> wrote:
> On Wednesday, 22 April 2015 11:24:25 UTC+1, Mateusz Łoskot wrote:
>>
>> For example, what's the canonical way to pass arguments like
>> a database connection string?
>

> (1) Before you hand off to Catch parse the args however you like and
> extract the args you're interested in. Pass the rest into Catch. You could
> pass them all into Catch at this point but it will warn about unknown args.

The warning is not a problem.
I'm more concerned about nice and efficient way of making such arguments,
or custom configuration, available from inside the tests.

> (2) There is a way, which I don't have the details of to hand right now (can
> get back to you with this if you're interested) where you can pass the
> arguments into Catch at a lower level and tell it to return unused args to
> you, instead of complaining. You'll get them back in a pre-parsed form (this
> is actually Clara - the self-contained argument parser). You can then feed
> these back into Clara with your own arg spec

This sounds good, though how to access them inside the tests?
I can't see how I could feed Catch::Session with custom Config and ConfigData.
The only way would be to have own singleton object available for all tests,
then no need to parse the arguments by Catch itself

I realised, my scenario is more complicated than just arguments parsing.
It feels more like dynamic configuration of test cases

0. Two test cases

TEST_CASE("Oracle")
{
auto connection = ... // #1 passed as CATCH arg
soci::session sql(soci::oracle, connection);
...
}
TEST_CASE("DB2")
{
auto connection = ... // #2 passed as CATCH arg
soci::session sql(soci::db2, connection);
...
}

1. Built as single executable
2. Run: my_tests --oracle "Oracle connection" --db2 "DB2 connection string"
3. Test cases use --oracle argument in #1 and --db2 argument in #2, but how?

What would be a recommended solution for such scenario?

[Marv L]

On Tuesday, 5 May 2015 13:00:06 UTC+1, Mateusz Łoskot wrote:

On 5 May 2015 at 07:31, Phil Nash <pan.e...@gmail.com> wrote:
> On Wednesday, 22 April 2015 11:24:25 UTC+1, Mateusz Łoskot wrote:
>>
>> For example, what's the canonical way to pass arguments like
>> a database connection string?
>
>

I realised, my scenario is more complicated than just arguments parsing.
It feels more like dynamic configuration of test cases

0. Two test cases

TEST_CASE("Oracle")
{
    auto connection = ... // #1 passed as CATCH arg
    soci::session sql(soci::oracle, connection);
    ...
}
TEST_CASE("DB2")
{
    auto connection = ... // #2 passed as CATCH arg
    soci::session sql(soci::db2, connection);
    ...

} n

1. Built as single executable
2. Run: my_tests --oracle "Oracle connection" --db2 "DB2 connection string"
3. Test cases use --oracle argument in #1 and --db2 argument in #2, but how?

What would be a recommended solution for such scenario?

 

Good question. I don't know of a standard solution...

Have you considered using a text file? A test fixture could read the file before the tests run and populate const variables with the necessary strings. These could then be passed to the tests as required.

Another possibility would be to create a fixture with the necessary strings as maybe static constants. This could be used in whichever tests needed the information.

If you wish to emulate argument parsing, you could use either of these ideas to populate an argv array and set argc appropriately.

I guess it comes down to what your reasons are for using command line arguments passed to the catch test host.

Marv

[Mateusz Loskot]

Yes, I considered INI, YAML, JSON or any other format files, but that
would require
a parser, so first I'm looking for out-of-box solution, if possible.
Second, running tests in my case would require to manage
generating/removing of config file.

> Another possibility would be to create a fixture with the necessary strings
> as maybe static constants. This could be used in whichever tests needed the
> information.

The strings are dynamic.

> I guess it comes down to what your reasons are for using command line
> arguments passed to the catch test host.

Main reason is to run tests (using CMake/CTest) remotely on a CI server
where connection strings are not always known upfront, but generated
during CMake run.

The idea with using a config file is feasible, but command line arguments
fit my needs more naturally.

[Phil Nash]

So there's two things going on here:

1. Getting the command line arguments in the first place. You want (a) some help getting them at all - which is where Clara can help and (b) some co-operation with Catch so you get what's left after Catch has taken what it recognises (or vice-versa).

2. Having got, and parsed, those arguments: make them available from within your test cases.

Your follow-up question was about (2). Here I think you're on your own. I think some sort of Singleton config is justified here (I don't normally like to justify Singletons - and yet Catch uses a number of them itself). In theory Catch could be modified to hold your config object and pass it into each test case but (a) I don't see this as Catch's responsibility, (b) it would complicate the interface for everyone else, regardless of whether they used the feature and (c) I don't think a Singleton is  big deal here.

In fact I've done something almost identical to this in some (not unit) tests in my day job, which involve database access.

[Mateusz Łoskot]

On Wednesday, 22 April 2015 12:24:25 UTC+2, Mateusz Łoskot wrote:

Hi,

Is there any mechanism to pass arguments to the code under test?

For example, what's the canonical way to pass arguments like
a database connection string?

It has been long time...
Meanwhile, I have managed to apply some of the ideas to make
the custom command line arguments work for my case:

Anyone looking for similar solution, check what I have
applied to the nanodbc library tests:

https://github.com/lexicalunit/nanodbc/blob/f1b50965afa6c489670f17ac6924b5dbbb4f23f1/test/main.cpp