New plugin Bitbucket Server Branch Source

[Robin Müller]

Hi there,

I started implementing a new plugin "Bitbucket Server Branch Source" which provides the functionality to configure Bitbucket Server projects with the help of a SCMSourceNavigator.
I know there is already a plugin called "Bitbucket Branch Source" which has also support for Bitbucket Server. But I see several issues with this plugin. Bitbucket Cloud and Bitbucket Server are 2 different software artifacts that just sound similar but behave different in many ways. If you have a look to the code of the plugin you'll find a lot parts where it has to distinguish between Bitbucket Cloud and Bitbucket Server. This code would become obsolete if there are 2 plugins, one for handling Bitbucket Cloud and one for Bitbucket Server. One big difference between these 2 tools is that Bitbucket Cloud has built-in WebHooks and for Bitbucket Server you have to install an additional add-on. And there are several of them out there. To make everyone happy the Jenkins plugin should support all available Bitbucket Server WebHook add-on. In the new plugin I would try to reach this by providing an extension API which decouples the add-on specific code from the "Bitbucket Server Branch Source". To support a Bitbucket Server WebHook add-on it's just necessary to implement a separate plugin that implements this API. Doing this in the existing "Bitbucket Branch Source" plugin has 2 drawbacks:
1. it makes the already complex code of the plugin even more complex
2. the code base isn't very clean, if I would provide this change as a Pull Request it would take me weeks to get it working with the risk that the change will break something else

IMHO having 2 plugins "Bitbucket (Cloud) Branch Source" and "Bitbucket Server Branch Source" is the best we can do. And I'd assume that the "Bitbucket Branch Source" plugin would never have got support for Bitbucket Server if it would be still named Atlasian Stash.

If you have the same opinion it would be nice if somebody with the necessary permissions can create the following 2 git repositories in the jenkinsci organization and give me write access for them:
bitbucket-server-branch-source
bitbucket-server-webhook-api

Regards,
Robin

P.S. the current status of my plugin can be seen here https://github.com/coder-hugo/bitbucket-server-branch-plugin

[Stephen Connolly]

I do not think this is the right way to go at all.

There is a refactoring or two from my PoV to take place in the current plugin and that will result in much cleaner code without fracturing things.

While there are multiple webhook plugins available for BB server, from my understanding there is only one that has the events required to give efficient event notification for scm api (and I had to write the patch to get that too!)

I think it is much better to just highlight the best path and make that rock solid

My €0.02 anyway

- Stephen

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/b726aa13-db0c-422f-8ae2-903b52aa8233%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Sent from my phone

[Robin Müller]

Hi,

Am Sonntag, 12. März 2017 16:32:22 UTC+1 schrieb Stephen Connolly:

I do not think this is the right way to go at all.

Would you also implement the SCMSourceNavigator for GitLab within the GitHub Branch Source Plugin just as their names are similar? Bitbucket Cloud and Bitbucket Server are 2 different products so IMHO they should be handled by different plugins. Although Bitbucket Cloud and Bitbucket Server are both developed by Atlassian they don't even share the same code base. IIRC Bitbucket Cloud is written in Python whereas Bitbucket Server is written in Java. They have different APIs to interact with them so I can't see any good reason why they should be handled by a single plugin. IMHO the current solution doesn't follow the idea of a pluggable system.

 

There is a refactoring or two from my PoV to take place in the current plugin and that will result in much cleaner code without fracturing things.

While there are multiple webhook plugins available for BB server, from my understanding there is only one that has the events required to give efficient event notification for scm api (and I had to write the patch to get that too!)

This is wrong. Most of the WebHook addons I saw provide the necessary information to give efficient event notification for the scm api. Additionally not everybody has the chance to install exactly the add-on that is supported by the Bitbucket Branch Source Plugin. Depending on the amount of users that have access to a Bitbucket Server you have to setup some policies for add-on installations to not slow down the whole server by installing each and every add-on. Btw. this is the case in the company where I work.

Regards,

Robin 

[Stephen Connolly]

On Sun 12 Mar 2017 at 18:23, 'Robin Müller' via Jenkins Developers <jenkin...@googlegroups.com> wrote:

Hi,

Am Sonntag, 12. März 2017 16:32:22 UTC+1 schrieb Stephen Connolly:

I do not think this is the right way to go at all.

Would you also implement the SCMSourceNavigator for GitLab within the GitHub Branch Source Plugin just as their names are similar? Bitbucket Cloud and Bitbucket Server are 2 different products so IMHO they should be handled by different plugins.

That is a false argument IMHO

The biggest part of implementing the SCM-API contract for CDVCS systems like Gogs, GitHub, Bitbucket, GitLab etc is the logic for how to handle PRs.

I am exploring if I can provide some common base classes that apply generally across all of the products, but in the absence of that, the two Bitbucket CDVCS are most similar. We would just be duplicating the code and doubling the maintenance effort. I do not see an advantage in that.

Although Bitbucket Cloud and Bitbucket Server are both developed by Atlassian they don't even share the same code base. IIRC Bitbucket Cloud is written in Python whereas Bitbucket Server is written in Java. They have different APIs to interact with them so I can't see any good reason why they should be handled by a single plugin. IMHO the current solution doesn't follow the idea of a pluggable system.

The issues I have with the current impl is that the client "API" is not as cleanly designed as I'd like. There are changes I have planned. The net result is that the we should end up with a client API not a client "API" and the differences between the two can be isolated to the API. If at that point the API gets split into three plugins: API, Cloud & Server... and then Bitbucket Branch Source depends on these... that's fine... if IMHO overkill.

 

There is a refactoring or two from my PoV to take place in the current plugin and that will result in much cleaner code without fracturing things.

While there are multiple webhook plugins available for BB server, from my understanding there is only one that has the events required to give efficient event notification for scm api (and I had to write the patch to get that too!)

This is wrong. Most of the WebHook addons I saw provide the necessary information to give efficient event notification for the scm api.

Cool if the others provide the required details. The reality is SCMEvent handling can be extracted trivially, so I don't see the need to split if all you want is separate event pathways. 

Eg I wrote an SCMEvent handler plugin for ASF's GitPubSub in 20 minutes last night.

Most of this is stuff that can be handled in under 120 Lines of Java

Additionally not everybody has the chance to install exactly the add-on that is supported by the Bitbucket Branch Source Plugin. Depending on the amount of users that have access to a Bitbucket Server you have to setup some policies for add-on installations to not slow down the whole server by installing each and every add-on. Btw. this is the case in the company where I work.

There is also the issue of support for the  Jenkins plugin behaviour.

Not everyone in the OSS community has the luxury of somebody who can pay to get licenses for these and set up multiple Bitbucket server instances to allow testing the different server plugins.

If we add support for more we effectively push others in the OSS community out of the ability to contribute.

Anyway, I am just giving my opinion. I don not see value in splitting the implementation. If anything I see it causing more confusion for users.

Regards,

Robin 

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/1ee88bbb-14aa-4a02-9687-14651b6b284c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Robin Müller]

Hi

Am Sonntag, 12. März 2017 21:50:01 UTC+1 schrieb Stephen Connolly:

On Sun 12 Mar 2017 at 18:23, 'Robin Müller' via Jenkins Developers <jenkin...@googlegroups.com> wrote: 

...   

Not everyone in the OSS community has the luxury of somebody who can pay to get licenses for these and set up multiple Bitbucket server instances to allow testing the different server plugins.

If we add support for more we effectively push others in the OSS community out of the ability to contribute.

This wouldn't be an issue if the add-on specific code is moved to a different Jenkins plugin. Just have a look at this, how it can work.

If we leave the support for both Bitbucket Cloud and Bitbucket Server within the "Bitbucket Branch Source" plugin I'd suggest the following changes to improve the maintainability:

1. implement a BitbucketCloudSCMSourceNavigator and a BitbucketServerSCMSourceNavigator that share the same base class but have the tool specific implemented in itself (e.g. retrieving the correct client implementation). With this you don't need to distinguish based on the URL or some other user provided input whether to talk to Bitbucket Cloud or Bitbucket Server. You'll always know this information depending on the context.

2. Do the same for the SCMSource

3. Replace the current "REST client implementation" by using JAX-RS (see BitbucketServerAPI.java, BitbucketServerClientService.java). The current implementation is error-prone. E.g. URLs get generated using String.fromat which means you have to care for proper URL encoding of path parameters yourself. You have to map status codes of error responses manually to exceptions. JAX-RS does this out-of-the-box. And last but not least. JAX-RS provides an easy way to read the whole request/response to add it the the log (see LoggingFilter.java) which is very useful if some request failed.

I know that 1. and 2. will lead to some migration effort but this can be done automatically by the plugin within an initialisier. I did something similar already within the gitlab-plugin.

I don't know how your time schedule looks like but if you like I can support you reaching the above points. But as not everything of the plugin is covered by tests it could take some time to be sure I don't break any existing functionality.

Nevertheless I'll install my plugin for the moment in our Jenkins as this is working with our Bitbucket Server and will save the whole department a lot of time.

Regards,

Robin

[Patrick Wolf]

In hindsight it might have better to split this into two separate plugins but at the time it was our understanding that Atlassian was moving to consolidate both the branding and API specification of Cloud and Server. The implementation would still be separate but the specification was supposed to merge to be much more similar. These things obviously take longer than expected but afaik that is still their plan. I would love it if Bitbucket Server would provide webhooks out of the box like Bitbucket Cloud does.

Given where we are now I think splitting the plugins will take more work, create more overlapping code, and cause more user confusion than is necessary. If all of their jobs suddenly stopped working users would be pretty pissed. 

I think the refactor suggestions above are fine and leave it to you and Stephen to figure out. However, I think we need a default webhook plugin to test with and include in the bitbucket-branch-source plugin. We can create an extension point to register other webhook providers but we shouldn't break anything for existing users. It would be nice we either consolidate on one webhook source or we agreed on the minimum JSON payload that would be provided by each.

--

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/85b5ac9b-b3b5-4063-8676-0418fb784607%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Patrick Wolf

Product Director - Jenkins 

CloudBees