Testing without writing in the database
27 views
Skip to first unread message
jdeel
Nov 27, 2016, 6:37:17 AM11/27/16
to Sidekiq
My Rails 5 application is configured to use Postgresql as database.
I would need to test a few Sidekiq workers that fetch data from external websites with Nokogiri and write them to Postgresql.
Testing that the workers are effectively queued with a fake test is useful.
However it would be even more useful if I could test that the workers do their jobs.
As far as I have understood both the inline mode and the mode with harness disabled actually perform their jobs *writing* to the development database: is that correct?
I suppose they are useful to test Sidekiq and Redis configuration and communication, but they seem to actually do their jobs.
Moreover, I guess that, once these tests are successfully completed, they should be deleted or disabled, because every time the complete test suite were ran, unlike all other test unit tests, they would keep writing data to the dev database.
Is there any way to test Sidekiq workers (queueing, that their jobs are persisted to Redis and that the jobs are effectively performed) without writing to the application dev database?
I apologise for any misconception due to inexperience
I would need to test a few Sidekiq workers that fetch data from external websites with Nokogiri and write them to Postgresql.
Testing that the workers are effectively queued with a fake test is useful.
However it would be even more useful if I could test that the workers do their jobs.
As far as I have understood both the inline mode and the mode with harness disabled actually perform their jobs *writing* to the development database: is that correct?
I suppose they are useful to test Sidekiq and Redis configuration and communication, but they seem to actually do their jobs.
Moreover, I guess that, once these tests are successfully completed, they should be deleted or disabled, because every time the complete test suite were ran, unlike all other test unit tests, they would keep writing data to the dev database.
Is there any way to test Sidekiq workers (queueing, that their jobs are persisted to Redis and that the jobs are effectively performed) without writing to the application dev database?
I apologise for any misconception due to inexperience
Mike Perham
Nov 27, 2016, 3:03:31 PM11/27/16
to sid...@googlegroups.com
1. Workers are designed to be easy to test: MyWorker.new.perform(args).
2. Sidekiq does not use a SQL database ever, only Redis.
3. Rails5 does require a database connection even if a job does not use ActiveRecord. IMO this is a Rails5 bug; I’d advise you to open an issue with Rails if this is a problem for you. Specifically the Rails5 Executor executes an AR callback to clear out any transient query_cache data after every job finishes, which requires a connection. Backtrace here:
Only your application code (or Rails’s migrations) will ever write to the database.
Mike
--
You received this message because you are subscribed to the Google Groups "Sidekiq" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sidekiq+unsubscribe@googlegroups.com.
To post to this group, send email to sid...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sidekiq/4af4e501-b3f9-47ce-94eb-95d0c49b57c4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Mike Perham – CEO, Contributed Systems
Smart, effective open source infrastructure for your apps.
jdeel
Nov 28, 2016, 5:27:27 AM11/28/16
to Sidekiq
I gave my Sidekiq workers the task of updating attributes of ActiveRecord model instances once a week.
At the beginning of the year I need to populate my database, and at the end of the year I need to reset everything.
So my jobs would use ActiveRecord.
Rails has a dedicated test database for tests.
I am mainly concerned about my tests using the dedicated test database.
To my understanding, the 'disable' mode isn't meant for running tests, as I found in a codeship tutorial.
Would not this mean that in 'disable' mode the code in my Sidekiq workers would actually be executed writing changes in the development database?
Since testing all the process of queueing a job, pushing the job to Redis and performing the job would represent a complete test, I would use the 'disable' mode if it was meant to use the Rails test database.
What happens in 'inline' mode? Would the tests use the test database?
At the beginning of the year I need to populate my database, and at the end of the year I need to reset everything.
So my jobs would use ActiveRecord.
Rails has a dedicated test database for tests.
I am mainly concerned about my tests using the dedicated test database.
To my understanding, the 'disable' mode isn't meant for running tests, as I found in a codeship tutorial.
Would not this mean that in 'disable' mode the code in my Sidekiq workers would actually be executed writing changes in the development database?
Since testing all the process of queueing a job, pushing the job to Redis and performing the job would represent a complete test, I would use the 'disable' mode if it was meant to use the Rails test database.
What happens in 'inline' mode? Would the tests use the test database?
Mike Perham
Nov 28, 2016, 3:52:30 PM11/28/16
to sid...@googlegroups.com
Ah, ok, now I understand. Ignore everything I said before.
When running tests, ActiveRecord will only use the test database — models shouldn’t touch your dev db at all. `disable` means that Sidekiq will create jobs in Redis instead of creating jobs in-memory. Those jobs are not actually executed, that’s what `inline` does.
Please read the Testing wiki page for detail: https://github.com/mperham/sidekiq/wiki/Testing
--
You received this message because you are subscribed to the Google Groups "Sidekiq" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sidekiq+unsubscribe@googlegroups.com.
To post to this group, send email to sid...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sidekiq/7e478508-6361-4c52-b7b7-73c526cc39d1%40googlegroups.com.
jdeel
Nov 29, 2016, 4:31:58 AM11/29/16
to Sidekiq
Now I've got it!
Many thanks indeed for your help
Many thanks indeed for your help
Reply all
Reply to author
Forward
0 new messages