Associate setup/startup functions with particular tests

[Stefan Chipilov]

Hello,

I was wondering if it is possible to associate setup* and/or startup* functions with particular tests.

In my experience, this is possible in other languages, since the startup/setup methods are usually defined per test class, which contains multiple tests.

Is it possible to do something similar in pgTAP?

[Stefan Chipilov]

I guess it's NOT possible yet, since I just found this: https://github.com/theory/pgtap/issues/96

[kevin]

--

Why not just make a "higher lever" test? Maybe something like:

hl_testcase1 (which calls):
    setup_testcase1
    testcase1
    teardown_testcase1

Or else at the top of testcase1, just put all of the setup required; it is SQL. Then again, I don't use runtest but call pg_prove on my test files so perhaps that makes a difference.

HTH,
Kevin

[Kyle Jensen]

Stefan - I do what Kevin suggested and it works great for me.

Sincerely, Kyle

[Stefan Chipilov]

Kayle, Kevin - thanks for the suggestion.

Indeed, this approach is a decent workaround for associating setup functions with tests.

However, this would be a bit harder to achieve for startup functions, since they are supposed to be called only once before all tests (as opposed to being called before each test).

I guess one way to achieve this would be to call the startup function ONLY in the first (alphabetically) test which matches a pattern. However, this would prevent running the tests in parallel (since you cannot ensure that the test which calls the startup code will be run first) and even for serial running, it would be a bit error-prone since you must ensure that you don't accidentally introduce another test with the same pattern which comes before (alphabetically) the test which calls the startup function.

Regards,

Stefan

[kevin]

Then I'd probably make the startup/setup tests/functions have a number prefix and run those serially.
Then have all of the rest of the tests with a letter prefix and run those in parallel, whether all at once or by letter, depending on your needs.
All controlled by some script, of course.

It sounds to me like you really have a dependency problem more than anything, something that should be easily solvable by a "make", "maven", "ant", or some tool like that. If you do that, then you don't need the "controlling script" as this tool would do that job.

Hopefully that gives you some ideas... :)

HTH,
Kevin

--
You received this message because you are subscribed to the Google Groups "pgTAP Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pgtap-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pgtap-users/88640c97-b7ef-4363-a315-34449ff71095n%40googlegroups.com.

[Stefan Chipilov]

Kevin, I am not sure why you think there is a dependency problem - this a pretty common feature (e.g. TestNG, Jest, etc) and I think it has its valid use-cases (of course, you can always abuse it, but that's true for almost anything).

For example, you might have two sets of tests - A and B - where tests in A expect that you are offline while tests in B rely that you are online. It would be great if you don't need to fiddle with network settings before each test which would make them much slower. For pgTAP, maybe a more realistic case would be tests in A expecting certain data in the DB while tests in B expecting different data.

You are right that I can probably hack together a way to achieve this using various scripts but, in a small way, that starts to defeats the purpose of pgTAP - in my view, one of the the main purposes for something like pgTAP (and other testing frameworks) is convenience.

Anyway, thanks for your input, I will keep it mind going forward.

Regards,

Stefan

[kevin]

Stefan,

It sounded like a dependency problem because you mentioned running things in specific orders, running things only once, doing things in serial and parallel; but you know your problem better than me. For our stuff, there are no dependencies and I could run all of my pgTap tests at once if I had enough CPUs. :) Instead I use "GNU parallel" to keep 10 tests running at all times until all tests are complete.

I think some of the disconnect between us is how we view pgTap. :) I read your statements as you're looking at it as a larger framework sort of thing, while I view it as a much lower level tool (generally not a framework or at most a very simple framework). To me, it's a tool to test basic sanity like are my tables correct, are all the pieces/parts that I'm expecting there, etc; I suppose I could use it to also test sql function output but I've never bothered as I have code unit tests to do that because I have to manufacture the data. I'm not saying pgTap can't be used like a full framework, but if that's where you're trying to go then I'll bow out here and let others help. If I've mischaracterized your goal, my apologies.

HTH,
Kevin

To view this discussion on the web visit https://groups.google.com/d/msgid/pgtap-users/091b0b35-5bc4-4d97-afbc-c8a1d748dd7fn%40googlegroups.com.

[Nasby, Jim]

If your specific concern is the availability of specific test data in your database, the way I’ve handled that in the past was to create a small framework that made it fairly easy to create test data that had a named identifier, and then allowed you to easily retrieve that test data later.

 

Unfortuanetly this was developed internally at a company, so I don’t have code I could share, but the interface looked something like this:

 

Test_data__create(

  Data_name text
   , table anyelement

  , creation_command

) RETURNS SETOF anyelement

 

Test_data__get(

  Data_name text

  , table anyelement

) RETURNS SETOF anyelement

 

The fun part here is `anyelement`. That pseudotype is used to tell these function what table the test data is in, *and* because they’re polymorphic allow the functions to *return* data from any table in the database. The easiest way to specify the table was to do `NULL::schema_name.table_name`; that creates a NULL that is cast as the table’s composite type. With some catalog trolling, that can be converted into the regclass for the table.

 

Test_data__create() ran `creation_command` in order to generate test data. That command had to be written in such a way that it did the equivalent of `INSERT INTO table … RETURNING *`. The data returned was stored in an internal table by doing something along the lines of

 

INSERT INTO test_data VALUES( data_name, regclass of table, array( <results of creation_command casted to table> )::text );

 

Test_data__get would then return

SELECT unnest( SELECT data::<composite array type of table> FROM test_data WHERE (data_name, table) = (…) )

 

Note that I had this setup in such a way that test_data__create would not re-create data that already existed. That made it safe to call those functions as many times as needed. That meant that in each of my test scripts, I could simply do a `\i` to include whataver data creation scripts I needed, regardless of the order tests were running in.

 

From: <pgtap...@googlegroups.com> on behalf of kevin <kbra...@pwhome.com>
Date: Wednesday, January 5, 2022 at 2:29 PM
To: "pgtap...@googlegroups.com" <pgtap...@googlegroups.com>
Subject: RE: [EXTERNAL] [pgtap-users] Re: Associate setup/startup functions with particular tests

 

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/pgtap-users/ac2d6557-5a96-25b1-0e65-fb30ff6bce8a%40pwhome.com.

[Stefan Chipilov]

Thanks for sharing this approach, Jim. 

To be honest, I am not very familiar with some of the details in your description (e.g. the usage of anyelement and regclass) so I am not sure I understand completely. Nevertheless, if I gloss over these details, I think the most important thing that I don't really understand is how the production code that is being tested will use this data during test. 

For example, let's say that you have a PL/pgSQL function called verify_subscription_pyaments() which is triggered every X hours and does 3 things:

 - retrieves all records from a table called sales_order where the payment_date field is NULL (e.g. SELECT * INTO unpaid_orders FROM sales_order WHERE payment_date IS NULL);

 - for each retrieved record above, try to find a new payment for that order (e.g. SELECT * INTO order_payment FROM payments WHERE sales_order_id = unpaid_order_id);

 - for each sales_order where a payment is found, set the payment_date to be the date of the payment record;

 - for each sales_order which is still unpaid and whose creation timestamp is more than X days, revoke access from the relevant user who has NOT paid their order;

My understanding is that in order to test various scenarios for this function, I would need to populate the sales_order and payments tables with appropriate data (e.g. orders with and without payment) and after the function completes, check that it has marked the relevant orders and users, appropriately.

How would the function get data using the functions you mention above? Are you somehow mocking the SELECT calls to go to your Test_data__get functions instead of the actual tables?