Multi Build Systems / Multi Repo Builds

37 views
Skip to first unread message

Fredrik Ekholdt

unread,
Oct 11, 2013, 5:12:22 AM10/11/13
to adep...@googlegroups.com
Posted orginally on github issues (#26):

Hi,

Say if I have project A in SBT, project B on Maven, C on Gradle, D on Buildr, ... I while the projects are hosted on GIT, Hg, Bazaar, etc. It will be nice to be able to create a projects say SuperProject which uses all this by pointing to the relevant repos' hash / tag / etc. then get the build script evoke the build tool and cache the artifacts without having to build separately or use output builds of the subprojects.

Also generate build scripts for SuperProject to integrated as a sub project into other build systems by generating the Maven, Gradle, SBT files and plugins.

Suminda

Fredrik Ekholdt

unread,
Oct 11, 2013, 5:13:12 AM10/11/13
to adep...@googlegroups.com
Replies where:
From me:

I am not a 100% sure I understand so let me try to rephrase it and you can tell me if I understood correctly: 

If one has multiple projects in different build tools and they all use Adept, it should be easy to create a super project in a build tool that that uses the published artifacts (and required dependencies) if that build tool also uses Adept. 


Then from Suminda:

BTW, I do not think the pure dependency management route will work. You will need some build and task management capabilities also with the ability to work with existing build management / dependency management / CI / Configuration Management systems. Look at Gradle, Buildr, Ivy, Ant, Maven, SBT, SBuild, Psake and try to get some of the best features into Adept with regard to build tools. Adept will need a way to represent a task or build management idiom in its own way and the ability to delegate this to an embedded instance of a build management system. Also ability to talk with different CIs need to be considered. Again so that the user need not learn different idioms Adept can has its own way to represent this.

With regard to interop Adept must be able to generate build scripts targeted to the relevant build management system and any plugins which will execute tasks which cannot be handled by the particular build management system.

Another interesting application is that Aspt can be a Glue where you can use multiple build tools to play at the strength of each tool. You can run some tasks in Buildr, some in Gradle, some in something else.

Also ability to integrate with different version control systems for automations and also ability to have dependencies being checked out and build in the fly without having to rely on published artifacts.

Fredrik Ekholdt

unread,
Oct 11, 2013, 5:30:43 AM10/11/13
to adep...@googlegroups.com, sirina...@gmail.com


On Friday, 11 October 2013 11:13:12 UTC+2, Fredrik Ekholdt wrote:
Replies where:
From me:

I am not a 100% sure I understand so let me try to rephrase it and you can tell me if I understood correctly: 

If one has multiple projects in different build tools and they all use Adept, it should be easy to create a super project in a build tool that that uses the published artifacts (and required dependencies) if that build tool also uses Adept. 


Then from Suminda:

BTW, I do not think the pure dependency management route will work. You will need some build and task management capabilities also with the ability to work with existing build management / dependency management / CI / Configuration Management systems. Look at Gradle, Buildr, Ivy, Ant, Maven, SBT, SBuild, Psake and try to get some of the best features into Adept with regard to build tools. Adept will need a way to represent a task or build management idiom in its own way and the ability to delegate this to an embedded instance of a build management system. Also ability to talk with different CIs need to be considered. Again so that the user need not learn different idioms Adept can has its own way to represent this. 

With regard to interop Adept must be able to generate build scripts targeted to the relevant build management system and any plugins which will execute tasks which cannot be handled by the particular build management system.

Perhaps you could be more specific why you think this will not work? I cannot see any reasons for it? Note: the it is the build tool and tool authors that will have to manage the different idioms/conventions, the user should not need to. So Adept will provide a set of conventions ("version" attributes have the same name, etc etc). These will be encoded in a library to promote them and to make sure we can change/move/deprecate conventions. The build tools should use these conventions.
I believe very strongly that we must do everything to avoid falling into the trap of where you mix concerns: building projects and dependency management. For me these MUST be 2 different things, which is what Adept does. Ivy does this too.

Another interesting application is that Aspt can be a Glue where you can use multiple build tools to play at the strength of each tool. You can run some tasks in Buildr, some in Gradle, some in something else.

Yes, that would be cool! Which is exactly why Adept must not be a build tool :)

 

Also ability to integrate with different version control systems for automations and also ability to have dependencies being checked out and build in the fly without having to rely on published artifacts.

If a build tool can consume dependencies (via subprojects) directly (sbt can do this, I think gradle also) you will not really need a dependency manager. If you want to use a dependency manager you will have to publish the artifacts. That being said, Adept will make it is easier and safer to publish: a "local" repository behaves in the exact same way as a "real" repository: you do not need repository managers, you do not have to worry about caching issues, ... To summarise: it is easier to publish locally (built in the model). When you cannot / do not want to consume dependencies directly (you have multiple build tools), you can easily and safely do this with Adept.

Suminda Dharmasena

unread,
Oct 11, 2013, 6:07:08 AM10/11/13
to Fredrik Ekholdt, adep...@googlegroups.com


--
Suminda Sirinath Salpitikorala Dharmasena, B.Sc. Comp. & I.S. (Hon.) Lond., P.G.Dip. Ind. Maths. J'Pura, MIEEE, MACM ▣ Address: 6G • 1st Lane • Pagoda Road • Nugegoda 10250 • Sri Lanka. ▣ Mobile+94-(0)711007945 ▣ Office+94-(0) 11 2 199766 ▣ Home Office: +94-(0)11-5 875614 ▣ Home+94-(0)11-5 864614 / 2 825908 ▣ Webhttp://www.sakrio.com 
--


On 11 October 2013 15:00, Fredrik Ekholdt <fre...@gmail.com> wrote:


On Friday, 11 October 2013 11:13:12 UTC+2, Fredrik Ekholdt wrote:
Replies where:
From me:

I am not a 100% sure I understand so let me try to rephrase it and you can tell me if I understood correctly: 

If one has multiple projects in different build tools and they all use Adept, it should be easy to create a super project in a build tool that that uses the published artifacts (and required dependencies) if that build tool also uses Adept. 


Then from Suminda:

BTW, I do not think the pure dependency management route will work. You will need some build and task management capabilities also with the ability to work with existing build management / dependency management / CI / Configuration Management systems. Look at Gradle, Buildr, Ivy, Ant, Maven, SBT, SBuild, Psake and try to get some of the best features into Adept with regard to build tools. Adept will need a way to represent a task or build management idiom in its own way and the ability to delegate this to an embedded instance of a build management system. Also ability to talk with different CIs need to be considered. Again so that the user need not learn different idioms Adept can has its own way to represent this. 

With regard to interop Adept must be able to generate build scripts targeted to the relevant build management system and any plugins which will execute tasks which cannot be handled by the particular build management system.

Perhaps you could be more specific why you think this will not work? I cannot see any reasons for it? Note: the it is the build tool and tool authors that will have to manage the different idioms/conventions, the user should not need to. So Adept will provide a set of conventions ("version" attributes have the same name, etc etc). These will be encoded in a library to promote them and to make sure we can change/move/deprecate conventions. The build tools should use these conventions.
I believe very strongly that we must do everything to avoid falling into the trap of where you mix concerns: building projects and dependency management. For me these MUST be 2 different things, which is what Adept does. Ivy does this too.


It need strong interop with other tools. It cannot stand on itself as a standalone isolated tool. It needs to be like a Glue among many tools. So it needs to be able to work with the features of many tools. Ideally to bring down the learning curve if it can have an Adept specific way to achive this it would be great (like mobile frameworks targeting multiple OSs.)
 

Another interesting application is that Aspt can be a Glue where you can use multiple build tools to play at the strength of each tool. You can run some tasks in Buildr, some in Gradle, some in something else.

Yes, that would be cool! Which is exactly why Adept must not be a build tool :)

 

Also ability to integrate with different version control systems for automations and also ability to have dependencies being checked out and build in the fly without having to rely on published artifacts.

If a build tool can consume dependencies (via subprojects) directly (sbt can do this, I think gradle also) you will not really need a dependency manager. If you want to use a dependency manager you will have to publish the artifacts. That being said, Adept will make it is easier and safer to publish: a "local" repository behaves in the exact same way as a "real" repository: you do not need repository managers, you do not have to worry about caching issues, ... To summarise: it is easier to publish locally (built in the model). When you cannot / do not want to consume dependencies directly (you have multiple build tools), you can easily and safely do this with Adept.

Many build tools play well with sub projects dependent projects if they use the same build tools. So project level dependencies need to be handled. Also say I want to specify X project on Hg, and Y project on Git as sub projects. X and Y need not have the same build management system. So you will need an embedded build tool to build the projects and perhaps a way to intercept the dependencies used through a transformation / special plugin / special task.
Reply all
Reply to author
Forward
0 new messages