Jenkins global metrics

[gotvi...@gmail.com]

I'm looking to generate a report, as a job, which will provide the global metrics of my Jenkins instance.  I need details in the report like (1) How many jobs ran? (2) # of failed and successful jobs and every possible metrics possible in one report.  I need to send this report every week to my management, so would like to know if it's possible to schedule a job to run this report every Friday and send the metrics by email to my managers?  I found these two plugins - 'Global Build Status' and 'Build Metrics'.  But not able to use these as part of a job, download the report or email a link to the report or nothing of that sort.  Please help.

[gotvi...@gmail.com]

Any help on this.  Little urgent.

[R. Tyler Croy]

(replies inline)

On Wed, 26 Sep 2018, gotvi...@gmail.com wrote:

> I'm looking to generate a report, as a job, which will provide the global
> metrics of my Jenkins instance. I need details in the report like (1) How
> many jobs ran? (2) # of failed and successful jobs and every possible
> metrics possible in one report. I need to send this report every week to
> my management, so would like to know if it's possible to schedule a job to
> run this report every Friday and send the metrics by email to my managers?

> I found these two plugins - '*Global Build Status*' and '*Build Metrics*'.

> But not able to use these as part of a job, download the report or email a
> link to the report or nothing of that sort. Please help.

I personally use Datadog and their plugin integration to get such metrics out
of our Jenkins instances. You can certainly script such metrics gathering in
Jenkins, but there's nothing ready-made out of the box.



Cheers

--
GitHub: https://github.com/rtyler
Twitter: https://twitter.com/agentdero

[gotvi...@gmail.com]

Thanks for the info Tyler.

[Vishal Raina]

We have been exploring Hygiene it not only helps with Jenkins metrics and also acts as a central dashboard for all your CI/CD tool set.

Take a look at here https://github.com/Hygieia/Hygieia.

HTH

[gotvi...@gmail.com]

Thanks a lot Vishal.  Will explore this.

[RAJENDRA PRASAD]

Hi ,

I did similar implementation for monitoring my Jenkins instance. But I never used a plugin to do that , instead of plugins I used groovy script to prepare mail body content and sending this mail body to intended recipient list.

Using groovy it is possible to get any information of Jenkins using Jenkins API calls.

Let me know you need further help in implementing this procedure.

Thanks,

Rajendra

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/b90c4f44-cae0-4c29-bdb1-2d9ceb0e1736%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[gotvi...@gmail.com]

Hi Rajendra.  Thank you so much for this news.  Would really appreciate help on implementing this.  Could you please guide me through?

[RAJENDRA PRASAD]

I am on the way to office , after reaching to office I can provide you sample groovy script and required configuration I can share with you.

With that info you can implement in time.

Thanks,

rajendra

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/42f25477-6bdc-485e-95a9-466833637c9f%40googlegroups.com.

[RAJENDRA PRASAD]

Here is the Ggroovy Script:

import javax.mail.*

import javax.mail.internet.*

import jenkins.model.*

import java.text.SimpleDateFormat

import java.util.*;

import java.util.Calendar;

import java.lang.System;

import org.apache.tools.ant.Project

import org.apache.tools.ant.ProjectHelper

def env="Dev_Jenkins_Server";

SimpleDateFormat sdf = new SimpleDateFormat("EEE MMM dd HH:mm:ss z yyyy");

sdf.setTimeZone(TimeZone.getTimeZone("EST"));

def  d=new Date();

def oneHourPrior=new Date(System.currentTimeMillis() - 3600 * 1000);

def currentTime=sdf.format(d);

def monitoringStartTime=trim(oneHourPrior)

def totalMonitoringJobs=0;

def testSuiteIndex=0;

def passedTestSuiteCount=0;

def failedTestSuiteCount=0;

def passedJobsList=[];

def failedJobsList=[];

def totalJobsList=[];

println "Monitoring Start Time:        "+monitoringStartTime ;

println  "Monitoring Current Time:      "+currentTime;

def matchedJobs = Jenkins.instance.items.findAll { job ->

    job.name =~ /Monitor/

}

println "*********************************************************************************"

       println "Total Monitoring Suites Count:"+matchedJobs.size();

       totalMonitoringJobs=matchedJobs.size();

println "***********************************Listing all Available Monitoring Projects**********************************************"

    def index=0;

     matchedJobs.each { job ->

               index=index+1;

    println index+". "+job.name;

                totalJobsList<<job.name+"\n";

}

    println "*********************************************************************************\n\n"

println "********************************Todays Execution Summary Results***************************************"

 matchedJobs.each { job ->

   def numbuilds = job.builds.size()

  if (numbuilds == 0) {

       println '  -> no build'

       return

  }

   def lastbuild = job.builds[0]

       Calendar cal1 = Calendar.getInstance();

        Calendar cal2 = Calendar.getInstance();

        cal1.setTime(monitoringStartTime);

        cal2.setTime(lastbuild.getTime());

if (cal2.after(cal1)) {

           testSuiteIndex=testSuiteIndex+1

            def rightPadding=50;

String jobName= job.name;

int requiredPadding=50-(jobName.size());

while(requiredPadding>0){

       jobName=jobName+" "

       requiredPadding--;

} 

def jobInfo=jobName+'\tlastbuild: ' + lastbuild.displayName + ' = ' + lastbuild.result + ', time: ' + lastbuild.getTime();

String latestbuildStatus=lastbuild.result;

     if( latestbuildStatus.equals("SUCCESS")){

                 passedTestSuiteCount=passedTestSuiteCount+1

                 //println "Found a Success Project"

                 jobInfo="\t\t"+ passedTestSuiteCount+" . "+jobInfo

                passedJobsList.add(jobInfo)

        }

     if( latestbuildStatus.equals("FAILURE")){

                 failedTestSuiteCount=failedTestSuiteCount+1

                   jobInfo="\t\t"+ failedTestSuiteCount+" . "+jobInfo

                 failedJobsList.add(jobInfo)

        }

    }

}

String passedJobsInfo= passedJobsList.join("\n")

String failedJobsInfo= failedJobsList.join("\n")

def jobsList ="";

def jobIndex=0;

totalJobsList.each{

     jobIndex=jobIndex+1;

     jobsList=jobIndex+". "+jobsList+it

}

String cumulativeStatus="PASS"

if(failedJobsList.size()>0){

    cumulativeStatus="FAIL"

}

String  emailBody=""" 

     *******************************Todays Cumulative Test Execution Summary Report*****************

     \tMonitoring Start Time:\t\t${monitoringStartTime} 

     \tReport Genretated Time:\t${currentTime}

     \tTotal Monitoring Jobs Count Scheduled to run in Monitoring Window:  ${totalMonitoringJobs}

     \tTotal Jobs Executed Today(As of Now): ${testSuiteIndex}

     \tTotal Jobs Passed Today(As of Now):    ${passedJobsList.size()}

     \tTotal Jobs Failed Today:(As of Now):     ${failedJobsList.size()}

     \tCumulative Result: ${cumulativeStatus}

    ***********************************************************************"""+"\n\n\t----->>>>>Failed Jobs Count: ${failedJobsList.size()} \n"+failedJobsInfo+"\n\n\t----->>>>>Passed Jobs Count: ${passedJobsList.size()}\n"+passedJobsInfo+"\n\n"

println emailBody;

String subject="${env} Jenkins Server:[ ${cumulativeStatus}]  ${currentTime}:  Cumulative Test Execution Summary Report";

 public static Date trim(Date date) {

        Calendar calendar = Calendar.getInstance();

        calendar.setTime(date);

        calendar.add(Calendar.SECOND, 0);

        calendar.add(Calendar.MINUTE, -60);

        calendar.add(Calendar.HOUR_OF_DAY,0);

        return calendar.getTime();

    }

def padRight(String s, int n) {

     String.format("%1-" + n + "s", s);  

}

//You have Following use email service and send it :

def emailSubject=subject

def emailRecipientsList="email1,email2"

def email_subject= subject

def email_body= emailBody

Above groovy script collects the pass failed jobs info and composes a  mail subject and mail body.

To use above groovy script:

1. Install Groovy plugin from plugin manager

2. Create a Freestyle project

3. Good to 'Build. section of the free style job (Please look at attached pic -1)

4. Select Option  :  Execute system Groovy script  (Please look at attached pic -1)

        Note: Unless you install Groovy plugin above option will not display build options

5. Select option Groovy Command 

6. Copy the above groovy Script in the next text field of build area config

7. If you wan to run this script periodically every day define a cron entry  build triggers section  similar to the attached pic-3

I am filtering jobs based on the job name, i.e if job name contains 'Monitor' token then those jobs only checked  and consolidated report will be generated.

You can also test above groovy script from script console of jenkins  

Note: I have not added the email sending part in this, there are sevceral ways to do that nased on the email service that you use 

Let me know if you need help in that also 

Thanks and Regards,
Rajendra Prasad Reddy Penumalli

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/42f25477-6bdc-485e-95a9-466833637c9f%40googlegroups.com.

[gotvi...@gmail.com]

Rajendra.  Thank you so much for the information.  It is very descriptive.  But, need some more help.  Sorry to bother, I'm new to Groovy.  You mentioned you are filtering the job name by 'Monitor'.  Just wanted to check how it works for this scenario

I need details of all jobs, irrespective of which project/folder they belong to, from Saturday to Friday.  How does it work in this case?

Also, I need this report to be emailed.  If you can suggest a method for that, will be helpful.  Thanks so much for the help.

[gtho...@gmail.com]

I have used autojenkins, it’s command line and api is neat to expose job details 

Sent from my iPhone

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAMrg02TjKqHeQsftXX7i%2BakzVOspKuh2yj%2BtYwwRwH%2BRiFCX8w%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

<pic_1_Build Options.png>

<pic_2_Build_Secition _Free Style_Job.jpg>

<pic_3_BuildTriggers.png>

[RAJENDRA PRASAD]

Hi ,

Happy to help you,

If you need all the jobs ( instead of filtered ones based in job name) , small chance needs to done in script, I can can update the script send you tomorrow, today I am fully tired, if it is utter udgent, let me know I can work today itself.

Regarding email, I am using ant script for sending email, that also I can explain you, using Jenkins you can install ant plugin and trigger the ant build.xml via groovy.

Also are you using Microsoft exchange server for for email service in your company or Gmail ?

Let me know will send you a small script to a send email via ant within groovy script.

Thanks,

Rajendra

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/9fea91d6-dc7d-4720-b361-d3411df8c91c%40googlegroups.com.

[RAJENDRA PRASAD]

Following snippet can collect all jobs into a list,

Just modify the script and please remove the filtering logic from previous script,

And add the following snippet, that's all you will get all the jobs.

def jobsList=[];

Jenkins.instance.projects.collect { 

String currentJob=it.name; 

jobsList.add(currentJob)

}

Thanks,

Rajendra

On Fri 28 Sep, 2018, 8:03 PM , <gotvi...@gmail.com> wrote:

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/9fea91d6-dc7d-4720-b361-d3411df8c91c%40googlegroups.com.

[Pavel Novák]

Hi, for that probably the best fits custom pipeline script

I have the similar

- reading total number of builds and the builds in the given period

Feel free to customize it or use as starting point if interested) - in attachment 

Dne středa 26. září 2018 21:29:23 UTC+2 gotvi...@gmail.com napsal(a):

[gotvi...@gmail.com]

Sorry for the delayed response Rajendra.  Will try this and let you know.  Got stuck with another issue.