Too frequent deploys on performance test server.

[Mateusz Kwiatkowski]

Hi everyone,

we've noticed that Jenkins queue is often full of performance builds. It appears that Performance Setver Deploy job is run after any of regular Test Server Deploy jobs and after that Performance jobs for all services are started.

We've added a 1h throttle for Performance Server Deploy but it is still a lot of jobs started at the same time. We think that it could be enough to run those tests on one specific moment of the day.

What do you think of that change?

Best Regards,

Mateusz

SolDevelo Sp. z o.o. [LLC] / www.soldevelo.com
Al. Zwycięstwa 96/98, 81-451, Gdynia, Poland
Phone: +48 58 782 45 40 / Fax: +48 58 782 45 41

[Paweł Gesek]

Is it very useful for us to have the performance feedback tied to a specific commit?

If not - lets just run daily.

If yes - how about just running the job appropriate for the affected pipeline - only requisition performance tests if this is a push to requisitions, but I am not sure how much of an effort such a rework of our CI jobs this will take.

Regards,

Paweł

--
You received this message because you are subscribed to the Google Groups "OpenLMIS Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev+unsubscribe@googlegroups.com.
To post to this group, send email to openlm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/CAHq-FDOmKocpedrHhw%2Byht7vsKf5MCQa2FJU%3DXqHD5_CkzVJ1Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

Paweł Gesek
Technical Project Manager
pge...@soldevelo.com / +48 690 020 875

[Łukasz Lewczyński]

But still you can change something in one service that modify response time of other services. The best example here is the reference data service. I think endpoints from this service are used everywhere so if we change something here we need to verify that other endpoints will still work fine.

I would say make it run once per day.

Regards,

Lukasz

Łukasz Lewczyński
Software Developer
llewc...@soldevelo.com

To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/CADt-Nu1C8%2BJ6GE-v5BhdgUx8w9K2%2Bnk%2BF6KCT-9HyRJ8Sxky8A%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Sebastian Brudziński]

I believe the reason to run all of them, rather than just the one related to the same service, was that the change in one service can affect other services too. Eg. a change in referencedata can affect performance of most/all of our services currently.

Anyways, I'd agree that running them daily, rather than a part of the pipeline, would be enough. We would need to think of a clear alert when they start failing too, since the author of the breaking commit wouldn't be notified anymore (unless we notified all people that sent any code changes during last day). We could set up the same notification that we have with functional tests (using @here/@channel Slack mention on #build).

Best regards,
Sebastian.

To unsubscribe from this group and stop receiving emails from it, send an email to openlmis-dev...@googlegroups.com.

To post to this group, send email to openlm...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openlmis-dev/CADt-Nu1C8%2BJ6GE-v5BhdgUx8w9K2%2Bnk%2BF6KCT-9HyRJ8Sxky8A%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Sebastian Brudziński
Senior Software Developer / Team Leader
sbrud...@soldevelo.com

[Sebastian Brudziński]

FYI everyone - we have discussed this at the tech committee call today and decided to convert those builds to happen daily as suggested. Here's the ticket: https://openlmis.atlassian.net/browse/OLMIS-4252

Thanks for suggesting this!

Best regards,
Sebastian.

[josh....@openlmis.org]

Did someone change the deploy-to-perftest job to not refresh perftest!?  The freshest images in there were from 9 days ago and I look at the job config and it's been "gutted" - no build steps.  I thought I'd re-deployed perftest but I suppose not.

[josh....@openlmis.org]

Just to followup on this, this was picked up and finished.  It redeploys and tests at sometime (when there's the least load) between 0300-0700 UTC, which I hope is the best time for most teams:  https://www.worldtimebuddy.com/?qm=1&lid=5809844,12,100&h=5809844&date=2018-3-8&sln=19-24

Best,
Josh

On Tuesday, March 6, 2018 at 9:04:12 AM UTC-8, Sebastian Brudziński wrote: