Questions on Jenkin

[avineet]

Hi,

We wanted to implement Jenkin in our project which require some kind of automation tool which support testing around Abnitio.

I have heard that Jenkin works with Abnitio,Could you please confirm me the same.
What would be the Procurement cost of the product.

Also,I drafted few high level cases which we want the tool should support.Could anyone please have a look and let me know if these cases works with Jenkin.

    Compare input and output record counts
    Compare input values and output values
    Validates transformations
    Compares checksums created against the input and output (e.g. The sum of input column 1 = sum of output column 1)
    Compare input with a filter record count with output record count
    Supports files and tables as either the input or output (File to Table, Table to Table, Table to File)
    Supports the validation of a test against a static value, e.g. Sum of column 1 = value x
    Supports embedded SQL to use for the validation routine, e.g. Output of SQL 1 is equal to the output of SQL 2 or Output of SQL 1 is equal to this

[Mandeville, Rob]

Do you mean Ab Intio?  BTW, it’s Jenkins, not Jenkin.

 

The procurement cost of Jenkins is zero.  Just go to http://www.jenkins-ci.org and download.  Cloudbees will sell you an enterprise edition with extra plugins and a support contract, or sell you a service where you run Jenkins on their cloud.  I’d start with the free copy to see if it even meets your needs, then consider the possibility of going to Cloudbees (Truth in advertising: my company just bought their enterprise edition for the support contract).

 

I have found nothing that specifically connects Jenkins to Ab Inito, Jenkins can run with a bunch of different tools because it supports running tasks as Windows batch scripts or Unix shell scripts.  This is how it connects to all but a few general development tools.

 

While Jenkins is useful for automating running of tests, I don’t think that it directly supports the sorts of tests that you show below.  There are plugins to read *Unit style logs, or to read certain patterns (such as “TEST PASSED” or “TEST FAILED”) as pass/fail criteria, but doesn’t have the logic to compare results.  Basically, you would have to write the tests yourself, generate simple pass-or-fail strings, and have Jenkins parse the logs for those pass-or-fail strings.  The nice thing about Jenkins is that it will do so unattended, either on a schedule or whenever you submit a change to your source control system.

 

--Rob

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

The information in this message is for the intended recipient(s) only and may be the proprietary and/or confidential property of Litle & Co., LLC, and thus protected from disclosure. If you are not the intended recipient(s), or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any use, dissemination, distribution or copying of this communication is prohibited. If you have received this communication in error, please notify Litle & Co. immediately by replying to this message and then promptly deleting it and your reply permanently from your computer.

[avineet gupta]

Thanks Rob for the information.Yes Its Abinitio.

Do you know some information/docs which can explain the procedure to work with Abnitio.

Also what exactly the Cloudbees cost.

--
Thanks and Regards..
Avineet

[Mandeville, Rob]

All I know about Abinitio is what I got in five minutes of web searching, so I can’t help you much there.  Wikipedia says that Abinitio graphs compile down into Korn shell scripts or Windows Batch files, either of which Jenkins can run out of the box.  You can set Jenkins to poll your source control system every few minutes, and run builds when somebody checks in a change.

 

If you need Jenkins to compile Abinitio, I don’t know how to do that—if there is a command-line tool that will do that, Jenkins can run it.  If you need to test Abinitio graphs after you compile them, you’ll have to write or otherwise get a test harness for it in such a way that it produces “test passed” and “test failed” strings that Jenkins can parse.  Jenkins is not a test harness itself, but it can run them.

 

I don’t know Cloudbees’ support pricing structure, but I do know that it depends on the number of executors that you set up on your Jenkins server.  Very roughly speaking, an executor is the ability to do one thing at a time, so if you need to run ten builds at once, you need ten executors.  But I wouldn’t worry about Cloudbees support until you try the free version out.  Download a copy of jenkins.war, run “java –jar Jenkins.war”, and it’ll show up on port 8080 on the machine you run it on.  Point a browser at it and start playing.

 

In short: if you need to automate running a test harness, Jenkins can help you.  If you need a test harness, Jenkins doesn’t provide that.  Figure out your test harness some other way, and then Jenkins can run that for you.

[avineet gupta]

Thanks a lot Rob.It helps a lot.Just have few questions

1.What would be skills set required and effort required to implement the tool.

2.Does it supports embedded SQL to use for the validation routine

On Fri, Apr 26, 2013 at 4:35 PM, Mandeville, Rob <rmand...@litle.com> wrote:

[avineet gupta]

Just to elaborate in details:

1.I need to understand how much effort is required for the initial setup so that we can map it to cost figures. For example, if we need 2 Java developers and 1 System admin working for a month to set the tool up, we know we are talking about 3 FTEs worth of effort which we can convert to cost figures easily.

2.These are the high level requirement.We want our tool should help us to execute the below scenerio's example:

Compare input and output record counts Compare input values and output values Validates transformations Compares checksums created against the input and output (e.g. The sum of input column 1 = sum of output column 1)  Compare input with a filter record count with output record count  Supports files and tables as either the input or output (File to Table, Table to Table, Table to File)  Supports the validation of a test against a static value, e.g. Sum of column 1 = value x

Supports embedded SQL to use for the validation routine, e.g. Output of SQL 1 is equal to the output of SQL 2 or Output of SQL 1 is equal to this expected value

Compares checksums created against the input and output

Compare input with a filter record count with output record count

Supports files and tables as either the input or output (File to Table, Table to Table, Table to File)

Supports the validation of a test against a static value, e.g. Sum of column 1 = value x

Supports embedded SQL to use for the validation routine

[Mandeville, Rob]

The high level requirements you are giving (compare record counts, run two SQL commands and compare the results, compare test results from static values) are requirements for a test harness, not a continuous integration server.  If you don’t have a test harness, Jenkins cannot help you.  If you have a test harness, Jenkins can run it for you whenever somebody checks a change in to your source control.

 

If you have:

1: A source control system supported by Jenkins

2: A command-line build process to create your program (I believe they’re called “Graphs” in Ab Initio)

3: A command-line test harness that Jenkins understands (there are plugins for several, and the Build Log Parser plugin can be used to read results like “TEST PASSED” and “TEST FAILED”)

 

Then a release engineer can make something demoable in a matter of days, not weeks.  Basically, they’ll be able to get Jenkins to poll the source control system every n minutes, and when something changes, it can check out your code, run your command-line-based build procedure and then your test harness.

 

If you’re asking for a tool that will compare record counts, expected vs. actual results, or SQL validation, you need a test harness, and Jenkins is not a test harness.  Jenkins runs test harnesses for a living, but you need such a harness first.

 

--Rob

 

From: jenkins...@googlegroups.com [mailto:jenkins...@googlegroups.com] On Behalf Of avineet gupta
Sent: Tuesday, April 30, 2013 6:27 AM
To: jenkins...@googlegroups.com
Subject: Re: Questions on Jenkin

 

Just to elaborate in details:

[avineet gupta]

Thanks Rob!!
What are the source control systems that Jenkins supports? Ab Initio has its own source control as part of the EME which does include a command line interface but can Jenkins be used to access it?

[Mandeville, Rob]

You can see the whole list of plugins at https://wiki.jenkins-ci.org/display/JENKINS/plugins.  While there are over 20 source code management plugins alone, none of them appear to be for EME.  That being said, there are some other ways to trigger builds.

 

1: You can trigger a job to launch at certain times, such as “every five minutes”.  If that job can ask EME if the source has changed, it can then launch your actual build and test cycle.

2: The “File System SCM” and “URL SCM” plugins (there are several URL plugins, actually) allow you to poll certain files in a file system or the contents of a Web page every few minutes.  If you can get EME to even change the timestamp on a file somewhere whenever it receives new code, for instance, the File System SCM can detect that and use it to trigger new builds.

[Jennifer Christa]

Someone help me with CI/CD automation through Jenkins on ab initio application

How to connect EME repo through jenkins? What is the source code management used to connect EME repo to cloudbees jenkins?
How to do deployment on Ab initio applications using Jenkins?

Please suggest me a solution and help me on this.

[saurabh patra]

Hi,

Option 1: You can write your own custom script (shell script) to poll the eme and when you see any changes invoke Jenkins.

Option 2: Ab Initio has a utility to poll the EME, check with support on this.

Option 3: There a whole lot in co-op 3.3.5 for CI/CD, worth checking with Ab Initio if you want to build an e2e pipeline using Jenkins.

Cheers,

Saurabh