Support for git plug-in?

[drdt...@gmail.com]

Is the Git plug-in still supported?  I filed a bug issues.jenkins-ci.org last week and haven't seen any movement on it, or on any of the issues filed in the last six months against the Git component.  There are about fifty such issues, most of them assigned to Nicholas de Loof (the original author, I think), and none of them with any response or acknowledgement from him (or anyone else).

[nicolas de loof]

yes it is, but filling a bug (which one ?) doesn't mean someone will jump on it to fix. This is a community effort :P

issues are by default assigned to plugin maintainer (me) so the status on jira. I don't really monitor it (my bad), as I mostly focus on my own issues with this plugin and contributors pull-requests.

2013/2/13 <drdt...@gmail.com>

Is the Git plug-in still supported?  I filed a bug issues.jenkins-ci.org last week and haven't seen any movement on it, or on any of the issues filed in the last six months against the Git component.  There are about fifty such issues, most of them assigned to Nicholas de Loof (the original author, I think), and none of them with any response or acknowledgement from him (or anyone else).

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Don Ross]

I am sorry, this is my first foray into trying to get support for a community-maintained tool.  We are using Jenkins in an enterprise environment and haven't had any issues we couldn't solve ourselves since I started working here.  Thanks for getting back to me directly.

My issue is reported as https://issues.jenkins-ci.org/browse/JENKINS-16693; basically, we are getting heap errors since upgrading to the latest plug-in version, but only in our production environment.  We can't reproduce it in the supposedly-identical test environment or we wouldn't have upgraded.  I think it is due to the enhancements made in version 1.1.23 to changelog parsing for merge targets, but that is just speculation on my part.

Any debugging suggestions you have would be welcome.

Thanks,
Don

[nicolas de loof]

This is not a trivial one to debug.

Could help if you can confirm previous git plugin version was ok in same environment, and then maybe to confirm your hypothesis to build a custom git plugin that reverts the "merge targets" support

as a general answer to the issue you see logged in Jira, I'd like to refactor the plugin to use JGit in replacement for git cli ran as a system command, so that it don't rely on an external process, fragile output parsing, etc

2013/2/13 Don Ross <drdt...@gmail.com>

[Stephen Connolly]

On 13 February 2013 13:36, Don Ross <drdt...@gmail.com> wrote:

I am sorry, this is my first foray into trying to get support for a community-maintained tool.  We are using Jenkins in an enterprise environment

[My & Nicolas' employer's hat]

If you were/are a paying customer of My & Nicolas' employer's you could/can apply whips and chains to Nicolas to get the bug fixed 

(Nicolas, no cheating and assigning to me ;-) )

[/My & Nicolas' employer's hat]

Sorry, I couldn't help myself there... I did remove the name of My & Nicolas' employer, though it isn't exactly a secret.

[Mark Waite]

I concur with Stephen and Nicolas about this being a community effort and commend them for their contribution and for the help of their employer.

I think the community may have already discovered a variant this problem and has already provided a fix for some of the cases.  That fix has already been pulled into the source code of the git-plugin through a community contribution.  It has not yet been released in a new version of the git plugin.

If your git configuration for this job does not define any exclusion regionss, does not define any excluded committers, and does not define any inclusion regions, then the next release of the git plugin (whenever that happens) will resolve your issue.

Refer to https://github.com/jenkinsci/git-plugin/commit/81f432ef0961149b13ff82d7803a897587d8c887 for the details of the commit.

Thanks,

Mark Waite

I agree with Stephen and Nicolas that enterprise support is available from their employer (I suspect that is CloudBees).

There were a number of interesting discussions about the inherent problems in the "git show" command and its use in the git plugin.  As far as I can tell, this is the one and only use of the "git show" command, and it can be avoided in most cases (with the change mentioned above).

---

From: Stephen Connolly <stephen.al...@gmail.com>
To: "jenkin...@googlegroups.com" <jenkin...@googlegroups.com>
Sent: Wednesday, February 13, 2013 7:18 AM
Subject: Re: Support for git plug-in?

[Don Ross]

Thanks for the info, Mark.  I will roll my installation back to the previous version (1.1.21) and wait for an update.  I am a little concerned because we do, in fact, have exclusion regions, and have since before we started using Git.  Does that mean that this fix (when available) may not actually solve my problem?

My company did have CloudBees out to give a sales pitch on Enterprise Jenkins this fall, but decided against it.

[Don Ross]

I can confirm that our build jobs, and our build environment, has not changed in several weeks.  We saw the behavior almost immediately after upgrading both the Jenkins server to 1.480 (from 1.1478) and the Git plug-in to 1.1.26 (from 1.1.21), and nothing else changed at that time.  Today we are rolling back the plug-in and, if the problem goes away, that will confirm the plug-in is to blame. 

Unfortunately, I don't have the infrastructure, nor the expertise, to build my own version of the plug-in from source, and my hypothesis is pure speculation in case the suggestion rang a bell with you.  We could try to apply incremental upgrades of the plug-in over time, but since this is in a production environment, it will take a week to vet each upgrade.  We are trying to reproduce the problem in a test environment, which will give us more flexibility.

[nicolas de loof]

2013/2/14 Don Ross <drdt...@gmail.com>

I can confirm that our build jobs, and our build environment, has not changed in several weeks.  We saw the behavior almost immediately after upgrading both the Jenkins server to 1.480 (from 1.1478) and the Git plug-in to 1.1.26 (from 1.1.21), and nothing else changed at that time.  Today we are rolling back the plug-in and, if the problem goes away, that will confirm the plug-in is to blame. 

Which version of git plugin are you running now ?

 

Unfortunately, I don't have the infrastructure, nor the expertise, to build my own version of the plug-in from source, and my hypothesis is pure speculation in case the suggestion rang a bell with you.  We could try to apply incremental upgrades of the plug-in over time, but since this is in a production environment, it will take a week to vet each upgrade.  We are trying to reproduce the problem in a test environment, which will give us more flexibility.

Sure, first thing to do. Then I can help you to create snapshots of the plugin to isolate the commit to introduce this regression. 

[Mark Waite]

I am interested if the rollback of the git plugin version will change the behavior you are observing.  I'm pretty sure a plugin version rollback would not have resolved my issue, since my issue was due to a bug in the "git show" command and a change in my git repository which exercised that bug.

If you have excluded regions, or excluded users, then the submission I referenced is unlikely to help you.  The submission helps me because I have very few jobs that use excluded regions or users.  Most of the time the new git plugin will entirely avoid the call to "git show" in my usage.  If you have excluded regions or excluded users, I don't know of any way to avoid the calls to "git show", which probably means you will continue to see the problem.

The issue I encountered was that the repository that Jenkins was using seems to exercise a known bug in the "git show" command.  It seems that there are cases where the "git show" command may spin with 100% CPU use for hours.  I realize that is not the problem you encountered, but it was the original problem I encountered which caused me to investigate why the git plugin was calling "git show" at all.

The plugin calls "git show" within the "isRevExcluded()" method.  It uses "git show" to decide if a specific commit has been excluded or not.  Unfortunately, by using "git show", it also exercised the bug in "git show" which exposed the initial problem I saw.

Thanks,

Mark Waite

---

From: Don Ross <drdt...@gmail.com>
To: jenkin...@googlegroups.com
Sent: Thursday, February 14, 2013 8:40 AM

[Don Ross]

So far, rolling back the plugin version from 1.1.26 to 1.1.21 looks promising.  We were seeing the issue at least once a day, and after 24 hours it hasn't recurred.  In fact, a job which has been blocked for a week on this issue (we just didn't notice because it isn't a critical job and only runs once daily) ran automatically this morning.

Perhaps my issue isn't the same as yours, but it was definitely caused by some change in git plug-in behavior between 1.1.21 and 1.1.26.  I had never seen calls to 'git show' before the upgrade, so I had assumed that the 'git show' command was the new behavior.  Based on what you are telling me, though, it sounds like the parameters to git show changed, or perhaps the way the output is parsed...

What version of git are you using?  I am on 1.7.10.  Maybe that is the difference?

[Don Ross]

We are now back on 1.1.21 and the problem appears to have been eliminated.  Our test environment will be set up with 1.1.26 next week.

[nicolas de loof]

if you can get intermediate plugin release installed this might help. 1.1.21 .. 26 is a large range 

2013/2/15 Don Ross <drdt...@gmail.com>

[Mark Waite]

1.1.21 used "git whatchanged" to answer the "isRevExcluded" question, while 1.1.26 uses "git show" to answer the "isRevExcluded" question.  That might explain why 1.1.21 works for you and 1.1.26 does not.

Have you attempted the exact "git show" command from within one of the problem Jenkins job repositories to see if it shows a similar problem outside of the Jenkins context?  If it does, that would be another indicator that the use of "git show" in the isRevExcluded() API is at the root of your issue.

Mark Waite

---

From: nicolas de loof <nicolas...@gmail.com>
To: jenkin...@googlegroups.com
Sent: Friday, February 15, 2013 10:37 AM

[nicolas de loof]

I need to read git doc in detail to understand how those two commands differ

2013/2/15 Mark Waite <mark...@yahoo.com>

[rdoh...@smugmug.com]

Related to support for this plugin (and the fix Mark linked to for `git show` problems), will there be a release soon? I'm waiting on that fix and the last release was 3 months ago.

Thanks!

-Ryan

[nicolas de loof]

I have a WIP on using JGit, would make sense to have a release just before. Will do asap

2013/2/15 <rdoh...@smugmug.com>

[Ryan Doherty]

Awesome, thanks!

nicolas de loof wrote:

I have a WIP on using JGit, would make sense to have a release just
before. Will do asap

2013/2/15 <rdoh...@smugmug.com <mailto:rdoh...@smugmug.com>>

    Related to support for this plugin (and the fix Mark linked to for
    `git show` problems), will there be a release soon? I'm waiting on
    that fix and the last release was 3 months ago.

    Thanks!
    -Ryan


    On Friday, February 15, 2013 10:45:13 AM UTC-8, nicolas de loof wrote:

        I need to read git doc in detail to understand how those two
        commands differ

        2013/2/15 Mark Waite <mark...@yahoo.com>

            1.1.21 used "git whatchanged" to answer the
            "isRevExcluded" question, while 1.1.26 uses "git show" to
            answer the "isRevExcluded" question.  That might explain
            why 1.1.21 works for you and 1.1.26 does not.

            Have you attempted the exact "git show" command from
            within one of the problem Jenkins job repositories to see
            if it shows a similar problem outside of the Jenkins
            context?  If it does, that would be another indicator that
            the use of "git show" in the isRevExcluded() API is at the
            root of your issue.

            Mark Waite

                ------------------------------------------------------------------------
                *From:* nicolas de loof <nicolas...@gmail.com>
                *To:* jenkin...@googlegroups.com
                *Sent:* Friday, February 15, 2013 10:37 AM

                *Subject:* Re: Support for git plug-in?

                            ------------------------------------------------------------------------
                            *From:* Don Ross <drdt...@gmail.com>
                            *To:* jenkin...@googlegroups.com
                            *Sent:* Thursday, February 14, 2013 8:40 AM

                            *Subject:* Re: Support for git plug-in?

                                https://github.com/__jenkinsci/git-plugin/commit/__81f432ef0961149b13ff82d7803a89__7587d8c887

                                <https://github.com/jenkinsci/git-plugin/commit/81f432ef0961149b13ff82d7803a897587d8c887>
                                for the details of the commit.

                                Thanks,
                                Mark Waite

                                I agree with Stephen and Nicolas that
                                enterprise support is available from
                                their employer (I suspect that is
                                CloudBees).

                                There were a number of interesting
                                discussions about the inherent
                                problems in the "git show" command and
                                its use in the git plugin.  As far as
                                I can tell, this is the one and only
                                use of the "git show" command, and it
                                can be avoided in most cases (with the
                                change mentioned above).

                                    ------------------------------------------------------------------------
                                    *From:* Stephen Connolly
                                    <stephen.al...@gmail.__com>
                                    *To:*
                                    "jenkin...@googlegroups.__com"
                                    <jenkin...@googlegroups.__com>
                                    *Sent:* Wednesday, February 13,
                                    2013 7:18 AM
                                    *Subject:* Re: Support for git

                                        https://issues.jenkins-ci.org/__browse/JENKINS-16693

                                                <http://issues.jenkins-ci.org/>

                                                jenkinsci-de...@__googlegroups.com.

                                                For more options,
                                                visit
                                                https://groups.google.com/__groups/opt_out
                                                <https://groups.google.com/groups/opt_out>.

                                            --
                                            You received this message
                                            because you are subscribed
                                            to the Google Groups
                                            "Jenkins Developers" group.
                                            To unsubscribe from this
                                            group and stop receiving
                                            emails from it, send an
                                            email to

                                            jenkinsci-de...@__googlegroups.com.

                                            For more options, visit
                                            https://groups.google.com/__groups/opt_out
                                            <https://groups.google.com/groups/opt_out>.

                                        --
                                        You received this message
                                        because you are subscribed to
                                        the Google Groups "Jenkins
                                        Developers" group.
                                        To unsubscribe from this group
                                        and stop receiving emails from
                                        it, send an email to

                                        jenkinsci-de...@__googlegroups.com.

                                        For more options, visit
                                        https://groups.google.com/__groups/opt_out
                                        <https://groups.google.com/groups/opt_out>.

                                    --
                                    You received this message because
                                    you are subscribed to the Google
                                    Groups "Jenkins Developers" group.
                                    To unsubscribe from this group and
                                    stop receiving emails from it,
                                    send an email to

                                    jenkinsci-de...@__googlegroups.com.

                                    For more options, visit
                                    https://groups.google.com/__groups/opt_out
                                    <https://groups.google.com/groups/opt_out>.

                                --
                                You received this message because you
                                are subscribed to the Google Groups
                                "Jenkins Developers" group.
                                To unsubscribe from this group and
                                stop receiving emails from it, send an
                                email to

                                jenkinsci-de...@__googlegroups.com.

                                For more options, visit
                                https://groups.google.com/__groups/opt_out
                                <https://groups.google.com/groups/opt_out>.

                            --
                            You received this message because you are
                            subscribed to the Google Groups "Jenkins
                            Developers" group.
                            To unsubscribe from this group and stop
                            receiving emails from it, send an email to

                            jenkinsci-de...@__googlegroups.com.

                            For more options, visit
                            https://groups.google.com/__groups/opt_out
                            <https://groups.google.com/groups/opt_out>.

                        --
                        You received this message because you are
                        subscribed to the Google Groups "Jenkins
                        Developers" group.
                        To unsubscribe from this group and stop
                        receiving emails from it, send an email to

                        jenkinsci-de...@__googlegroups.com.

                        For more options, visit
                        https://groups.google.com/__groups/opt_out
                        <https://groups.google.com/groups/opt_out>.

                    --
                    You received this message because you are
                    subscribed to the Google Groups "Jenkins
                    Developers" group.
                    To unsubscribe from this group and stop receiving
                    emails from it, send an email to

                    jenkinsci-de...@__googlegroups.com.

                    For more options, visit
                    https://groups.google.com/__groups/opt_out
                    <https://groups.google.com/groups/opt_out>.

                --
                You received this message because you are subscribed
                to the Google Groups "Jenkins Developers" group.
                To unsubscribe from this group and stop receiving
                emails from it, send an email to

                jenkinsci-de...@__googlegroups.com.

                For more options, visit
                https://groups.google.com/__groups/opt_out
                <https://groups.google.com/groups/opt_out>.

            --
            You received this message because you are subscribed to
            the Google Groups "Jenkins Developers" group.
            To unsubscribe from this group and stop receiving emails

            from it, send an email to jenkinsci-de...@__googlegroups.com.

            For more options, visit
            https://groups.google.com/__groups/opt_out
            <https://groups.google.com/groups/opt_out>.

    --
    You received this message because you are subscribed to the Google
    Groups "Jenkins Developers" group.
    To unsubscribe from this group and stop receiving emails from it,
    send an email to jenkinsci-de...@googlegroups.com

    <mailto:jenkinsci-dev%2Bunsu...@googlegroups.com>.

[drdt...@gmail.com]

Now that we are no longer blocked, this has dropped to the bottom of the queue for the RelEng department.  I am still pressuring them for resolution, because if this is not a real problem with the plug-in, then we must be using Git in some non-standard way that causes unexpected behavior and I would like to change that.

[Don Ross]

W have managed to replicate this issue in our test environment, and confirmed it still exists in the all-new version 1.3.0.
If you have suggestions for things I can try to further diagnose the issue, I am eager to once again engage in resolving this.

[Mark Waite]

Can you clarify what you intend by "this"?

If "this" is https://issues.jenkins-ci.org/browse/JENKINS-16693 (as mentioned in the beginning of the thread), then the problem is that the Git plugin isRevExcluded() method uses "git show" to decide if commits since the last build should be excluded or not.  The "git show" command can generate massive amounts of output and the isRevExcluded() method is not designed to handle the large volume output from "git show".  If you want to submit a pull request for an alternate implementation of isRevExcluded (preferably with many more tests to confirm it works in many more cases), I'm sure the plugin maintainer would be willing to consider your pull request.

Thanks!
Mark Waite

[Don Ross]

By 'this', I was referring to the email exchange I was having with Mr. Deloof about debugging JENKINS-16693 in my environment, in the hopes of helping him find a solution for this problem which seems to be effecting many users (as there are multiple similar bugs filed in the issues-jenkins-ci in addition to mine).

I am not in a position to submit a pull request as that would require expertise of which I am rather lacking, which is one of the reasons I prefer to use third-party tools over home grown.  However, if an effort was being made in the community to resolve the defect, I can participate by providing a production test environment.

Respectfully, Don.

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/82KcytLyMwA/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.