how to Joomla! website development and GIT source control on a distributed development team

[Edric Navarro (enav)]

Hello Joomla! ppl

How a team of developers can work on a Joomla! website on their local machines then commit the changes to their local GIT repositories and each day push and merge all the changes to a remote repository. I know this can be done easily with files but what about the database.

Please don't get con confused this is not a extension development project, this is website development where 2 developers will do lots of changes to the template files and add lots of articles like menu items, articles, modules and so forth for the same project but using a local server on their respective computers.

GIT and source control for files is easy but what about the Joomla! database?... each developer have a working copy of the Joomla! website with it file and also a copy of the database on their local mySQL servers.

What if one user add a menu item and the next record key is "42" but the other developer also adds another menu items and the next record key is also "42" at the time of database merge there will be a collision of 2 records with the same key but different menu types.

The only workaround for Joomla! website development is to use GIT for all the files (no big deal) but create a development copy of the database in a remote server so both developer can work on the same DB same time and avoid this record keys collisions or any other problem.

This seems to be one solution but is clear that is not robust enough plus the DB have no source control, and there is no way to save ourselves in case some one deletes stuff that was not meant to be deleted.

I made this question at #PHP and some one told me that database source control for Joomla! is not possible because the database architecture, nonetheless they mention this following keywords to google about them: liquibase, flyway, h2, hsqldb, doctrine.

Any help suggestion or documentation to read about this topic will be greatly appreciated.

[Chad Windnagle]

Hi : 

This is a great question; and one that I personally have asked to other Joomla devs. 

The problem is not just _joomla_. The problem is databases! They just don't lend themselves well to version controlling of content or schemas on their own. 

I've done loads of reading and research on this topic and there aren't any perfect solutions (that I've found so far!) There's work arounds, and there's procedures that help. 

Here's some different approaches I've learned about:

1) http://dbv.vizuina.com/ (not used, but read about)

2)  .git based implementation of schema changes with a custom post-git-commit hook to dump structures to a .sql in the project folder and compare them. 

3) more complex versions of #2 but expanded to include content and rows (pretty annoying with large databases imho)

4) Shared database for all developers 

 - a single, remotely accessible database which all developers have access to, thus everyone works' on the same database

5) Dev databases with only sample content and a proper schema as a .sql which is under .git version control. Any changes are committed to the file. (basically like #1)

6) Someone I talked to actually had an idea (not sure if it ever got implemented) where they wrote a plugin for Joomla which grabbed into the onSave event hook, and basically grabbed whichever query was being run and dumped it into a file as a logger. This is great for changing menus and content and all that. Then you can take those queries and run them on a DB to get up to date.

This is probably the best way to do it if you really don't want to run remote databases. Unfortunately there's no off the shelf plugin that I know of, and any component's not using the event system will not get tracked. 

What I do:

I actually do a hybrid of the shared database for live sites, while version controlling the .sql schema (not the content).

Generally its two databases, a production one and a dev one. Dev's work on new features. commit .sql changes to a file and when we launch the feature we manually run the queries to update the production database (after proper backups, of course). 

This is basically the solution you already identified I think. It's not pretty or perfect, but its usable. 

--

The Doctrine / ORM way:

If you've used Symfony / Doctrine or Laravel's Eloquent there's a file-based version control for schema (and content if you build that out). It's reasonably good for developers when building out apps. 

Both systems ahve a form of migrations - which is basically a set of sql queries, in code (version control) which allow you to take the database through every schema it's ever been in. The only annoying thing about this is you have to kind of write those queries yourself for anything complex. 

And again, not great for tracking database content, but actually wonderfully good for schemas. I wish joomla worked nicely with an ORM. Apparently there are some people using Joomla + Doctrine, which is probably wonderful when it works. 

Hopefully some of this helps; I've spent a lot of time thinking about this problem and trying out different solutions. If you think of something better come find me! 

Regards,
Chad Windnagle

--
You received this message because you are subscribed to the Google Groups "Joomla! CMS Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to joomla-dev-cm...@googlegroups.com.
To post to this group, send email to joomla-...@googlegroups.com.
Visit this group at http://groups.google.com/group/joomla-dev-cms.
For more options, visit https://groups.google.com/d/optout.

[C Hamilton]

Hi Chad,

With a team of only two developers, who communicate closely so as not to be creating conflicts, Chad's idea #2 works well.

I once wrote a functional specification for a program similar to idea #6, but my coding skills are not yet up to this task. I think the merges can be partially automated, but human input will be needed to resolve certain types of conflicts.

Christina Hamilton

[Edric Navarro (enav)]

Thanks for your very nice answer chad.

I show this thread to the #joomla IRC channel and they say the same there is no standar official or defined way to version the joomla db along the code

[Joe Palmer]

There are some deployment tools which apparently handle database changes specifically with Joomla support that you may find useful:

http://www.pharaohtools.com/solutions/drupal-joomla-wordpress

On Fri, 6 Nov 2015 13:14 Edric Navarro (enav) <digitalco...@gmail.com> wrote:

Thanks for your very nice answer chad.

I show this thread to the #joomla IRC channel and they say the same there is no standar official or defined way to version the joomla db along the code

--
You received this message because you are subscribed to the Google Groups "Joomla! CMS Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to joomla-dev-cm...@googlegroups.com.

To post to this group, send an email to joomla-...@googlegroups.com.