Re: [jgen] How we corehacked Joomla to introduce different deployment environments

[Donald Gilbert]

I like the idea of being able to set config based on environment - very good implementation you have there. What we've done in the past is remove the configuration.php from version control, and use separate ones in our production and staging environments. Your approach is really good, I could see something like this making it into core and being very useful.

On Sat, Jan 26, 2013 at 3:11 PM, eiirc <eiirc...@gmail.com> wrote:

I guess this is a kind of tech blog post. When we chose Joomla for development of our website, we met a few obstacles. I thought I would share one of them, and how we proceeded to solve it. Maybe it could be taken into consideration for future releases?
------
I'm one of the developers of http://smil-norge.no, a Norwegian website. We average on 1 million page views a month, 2400 unique users a day, etc. The site has about 4.000 registered members, most of them logging in more than once every 24 hours.

Our site is so big we cannot afford security issues or major bugs in production. The way Joomla is built, it encourages changes (especially configuration changes) to be done directly in the production environment, which, for us, is not an option. We therefore made a core hack in the configuration management, so that we could override configuration options from one environment to the next.

What we did was add a constructor to JConfig:

<?php
class JConfig {
    public function __construct() {
        if (isset($_SERVER['JOOMLA_ENVIRONMENT'])) {
            $joomla_environment = $_SERVER['JOOMLA_ENVIRONMENT'];
            $config_environment = parse_ini_file($joomla_environment.'.ini');
            foreach ($config_environment as $key => $value) {
                $this->{$key} = $value;
            }
        }
    }

    // Default configuration here
    ...
}
?>

With this hack, we can override default configuration in "local.ini", "staging_area.ini", "production.ini" etc. Typically, we want to specify different database configuration, turn debug mode on/off, set different base paths, etc. for every environment. The environment is configured in Apache.

The cost of core hacks is that we no longer can upgrade Joomla versions easily. We also lost the ability to use the configuration control panel, and now have to specify all configuration manually/in the code. Another problem (more difficult to solve) is that configuration for all the components is stored in the database. To fix this, we had to introduce database versioning for the configuration tables, which, I promise, is a pain.

What we would definitely want to see in future versions of Joomla, is file-based configuration for everything (eventually cached in the database), and possibilities for overriding default configuration values in different environments!

--
You received this message because you are subscribed to the Google Groups "Joomla! General Development" group.
To post to this group, send an email to joomla-de...@googlegroups.com.
To unsubscribe from this group, send email to joomla-dev-gene...@googlegroups.com.
Visit this group at http://groups.google.com/group/joomla-dev-general?hl=en-GB.
For more options, visit https://groups.google.com/groups/opt_out.

[Olivier Nolbert]

Hi,

I'm not sure to see the benefit of using an outside configuration file for different environment as for me this can be done directly in the configuration.php.

Everything is in the configuration.php, you just have to have a different version for every environment as Donald wrote.

@eiirc, why did you have to hack the JConfig class ? What i mean is why can't you what you do in Apache environment in the Joomla! backend ? Let's say you want to change the db configuration. Your approach is to change a variable in the Apache environment and then it will be implemented in Joomla!. Why can't you do this directly from the Joomla! backend ?

Do i miss something important ?

2013/1/27 Donald Gilbert <dilber...@gmail.com>

--

Olivier Nolbert, Jiliko

14 passage Lecroisey 76600 Le Havre, France

(+33)6 48 09 46 25

SIREN : 517 564 365

www.jiliko.net - Newsletter subscription - Twitter - Google + - Facebook

[Naouak]

I agree with you. New extension addition in joomla is a pain in production environment. There is actually no simple way to export and reimport config parameters in joomla.

General config is not the most problematic. It's extension configuration. In my company, we always get goosebumps when we have to put something new in production that was not developed by us because of that.

[Nick Weavers]

Yes, in my experience any updates to a busy live site are hairy because even the backup you took (files & db) are quickly out of date so restoring becomes progressively less of an option compared to fixing any bug. Bugs may not be discovered immediately, in fact sometimes not for hours or days. Credit to the "core" Joomla development and testing folks that this rarely seems to occur in Joomla updates. Sounds like extensions can be considered higher risk due to the variable nature of their dev/test/support capability/resources (one developer/hobbyist when time allows, to commercial development team with unit testing). Still a case of Caveat Emptor I guess, but IMHO any risk reduction techniques - such as those suggested - that could be developed into Joomla core would be a good thing, even at the expense of increased complexity.

[eiirc]

Sorry for the double post. About the extensions' configurations - that's a much more difficult issue to deal with. I wanted to write something about it in the initial post, but the post would be too long.

Keeping track of all the extensions' configurations in the database is a hassle, since it makes it quite impossible to restore earlier versions of the website. At least as long as you don't put your database in version control. Which is technically an option, but who puts their Joomla site's database in version control (e.g. use Liquibase) just to make safer configuration changes? What we do is change the configuration locally in the admin panel (so it applies to the developer's simplified copy of the database), test it, and then do the same change in every other environment including production.

Actually, we tried "versioning" our database (we dumped our configuration tables into sql files, and checked in these files into Git), but even when it was automated, it was such a hassle we gave it up.

What we would love to have is file-based configuration for every extension, with a possibility to override default settings from environment to environment. So that changing a confiuration setting is a matter of doing it locally, testing it, checking in the changed file, verifying, and releasing. Such a solution would also scale down to the sites where you just want to do your changes right in production, without the complexity. But the backwards-compatibility would be a pain.

[Olivier Nolbert]

Wouldn't be possible to extend/override the Joomla! classes that handle the global configuration and extension configuration ?

I haven't checked this solution yet.

2013/1/28 eiirc <eiirc...@gmail.com>

--

You received this message because you are subscribed to the Google Groups "Joomla! General Development" group.

To post to this group, send an email to joomla-de...@googlegroups.com.
To unsubscribe from this group, send email to joomla-dev-gene...@googlegroups.com.
Visit this group at http://groups.google.com/group/joomla-dev-general?hl=en-GB.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[eiirc]

That should certainly be possible! Though it would be a more invasive approach (more core hacking).

I posted this suggestion as an idea here: http://ideas.joomla.org/forums/84261-joomla-idea-pool/suggestions/3620614-file-based-configuration