complexed setup - first time to version system
Bipul Agarwal
Please forgive me if this is not the right place for my question.
We have a comparatively large website and works on a
sub-domain and a main domain. Also, the website is in Drupal + Core PHP scripts
+ Two languages (templates) + mutiple databases (tens of tables under each
database).
We have recently got the contract to maintain and enhance this website but
surprisingly there was neither a staging server for it and nor any versioning
software. This is one of the biggest projects for my small company and we have
no experience with versioning softwares.
Development server(not sure about it), Staging server and Production server needs to be connected with a mercurial depository. We have taken a hosted Mercurial repository with Beanstalkapp.com and your feedback on this will be very helpful too.
I have gone through several start up guides but would appreciate suggestions for best practices – I don’t want to mess with the project by experimenting. For compatibility reasons, I would prefer to have staging environment on the production server itself (may be on a different domain?). The website is hosted in a dedicated server (linux) and we can create different cPanel account for it.
Thanks, Bipul
Greg Ward
> Please forgive me if this is not the right place for my question.
>
>
>
> We have a comparatively large website and works on a sub-domain and a main
> domain. Also, the website is in Drupal + Core PHP scripts + Two languages
> (templates) + mutiple databases (tens of tables under each database).
notice Mercurial -- it's plenty fast enough. Above that, you notice
things like "hey, checking out 20,000 files takes a minute or two" and
"transferring 500 MB over a slow network takes a while". Another key
metric: size of the largest file. If you have no individual files
larger than 10 MB or so, don't worry about it. Just do the obvious
thing and forget about performance tricks.
> We have recently got the contract to maintain and enhance this website but
> surprisingly there was neither a staging server for it and nor any
> versioning software. This is one of the biggest projects for my small
> company and we have no experience with versioning softwares.
priority should be to get the damn thing under version control and set
up a procedure for development/staging/production. I suggest starting
here:
https://www.google.com/?q=web+site+version+control+best+practices
> I have gone through several start up guides but would appreciate suggestions
The best thing about distributed version control systems is that you
*can* experiment in a local clone. If you screw up, no big deal. Throw
it away, make a new clone, and start over. You might lose an hour or
two of work, but the learning you gain from that is priceless.
And the best thing about using a development/staging/production
workflow for web development... same thing. Screw around on your
development server all you like. If it breaks, try again. No big deal.
> For compatibility reasons, I would prefer to have staging environment on the
> production server itself (may be on a different domain?).
question about PHP and whatever web server you're using (Apache,
nginx, whatever). This is *not* the right forum.
Anyways, you didn't really ask any specific questions, so I'm not
surprised you didn't get any replies. First, get the existing site
into a Mercurial repo and then make a clone that's far from
production, so you can play around. Try something and see if it works.
If not, try something else. If you get confused, ask here for help.
Greg
_______________________________________________
Mercurial mailing list
Merc...@selenic.com
http://selenic.com/mailman/listinfo/mercurial
Bipul Agarwal
On 19 August 2012, Bipul Agarwal said:
> Please forgive me if this is not the right place for my question.
> We have a comparatively large website and works on a sub-domain and a main
> domain. Also, the website is in Drupal + Core PHP scripts + Two languages
> (templates) + mutiple databases (tens of tables under each database).
How large? Up to (say) 5,000 files and 100 MB of data, you hardly
notice Mercurial -- it's plenty fast enough. Above that, you notice
things like "hey, checking out 20,000 files takes a minute or two" and
"transferring 500 MB over a slow network takes a while". Another key
metric: size of the largest file. If you have no individual files
larger than 10 MB or so, don't worry about it. Just do the obvious
thing and forget about performance tricks.
other than database, no individual file is more than 1MB.
> We have recently got the contract to maintain and enhance this website but
> surprisingly there was neither a staging server for it and nor any
> versioning software. This is one of the biggest projects for my small
> company and we have no experience with versioning softwares.
Yikes. If it wasn't already obvious to you, I suggest that your first
priority should be to get the damn thing under version control and set
up a procedure for development/staging/production. I suggest starting
here:
https://www.google.com/?q=web+site+version+control+best+practices
(using BeansTalkapp)
> I have gone through several start up guides but would appreciate
suggestions
> for best practices - I don't want to mess with the project by
experimenting.
The best thing about distributed version control systems is that you
*can* experiment in a local clone. If you screw up, no big deal. Throw
it away, make a new clone, and start over. You might lose an hour or
two of work, but the learning you gain from that is priceless.
And the best thing about using a development/staging/production
workflow for web development... same thing. Screw around on your
development server all you like. If it breaks, try again. No big deal.
we upload something directly on production server. And client is
disappointed each time that we are messing up with the live site - and he
is right too.
> For compatibility reasons, I would prefer to have staging environment on
the
> production server itself (may be on a different domain?).
I'm sure there are lots of ways to do that, but that's really a
question about PHP and whatever web server you're using (Apache,
nginx, whatever). This is *not* the right forum.
Anyways, you didn't really ask any specific questions, so I'm not
surprised you didn't get any replies. First, get the existing site
into a Mercurial repo and then make a clone that's far from
production, so you can play around. Try something and see if it works.
If not, try something else. If you get confused, ask here for help.
1) the website is partially done in Drupal for CMS purposes and the
anchor links in CMS are mostly Static - giving full URL path. So, we are not
sure how to deal with it, yet. If we move the clone to staging server,
we will manually need to change all the Static URL's to new domain. And even
if we do it, how do we move back and forth between staging and
production environment then?
2) the data is very sensitive/confidential and huge. Client is not
agreeing for letting us copy the database to staging server - we are allowed
only to copy the database structure. In this case, it would become almost
impossible to test SQL queries for reporting and other similar purposes. How
do we deal with it?
3) if we do structural changes in database tables
(adding/editing/deleting database tables or fields), would it also be
reflected on the production server? Again, if a database field in
staging server has only 10 records and we delete this field, what will
happen to database field on production server with thousands of
records?
4) I think we will know with our first experiment - do we keep
versions at each environment? On production server too? We will have many
many dump/useless files on development environment. If the
infrastructure is kept identical at all places, will we have similar files
on staging and production environment?
I might sound over cautious, but just that project is so big and I
am utmost concerned.
Many thanks, Bipul
Kevin Bullock
> Thanks for your reply Greg. My answers below yours:
>
> On 19 August 2012, Bipul Agarwal said:
>> Please forgive me if this is not the right place for my question.
>> We have a comparatively large website and works on a sub-domain and a main
>> domain. Also, the website is in Drupal + Core PHP scripts + Two languages
>> (templates) + mutiple databases (tens of tables under each database).
>
> How large? Up to (say) 5,000 files and 100 MB of data, you hardly
> notice Mercurial -- it's plenty fast enough. Above that, you notice
> things like "hey, checking out 20,000 files takes a minute or two" and
> "transferring 500 MB over a slow network takes a while". Another key
> metric: size of the largest file. If you have no individual files
> larger than 10 MB or so, don't worry about it. Just do the obvious
> thing and forget about performance tricks.
>
>
> Not more than 5000 files but data is approximately 1GB and above.
> other than database, no individual file is more than 1MB.
...and the database won't be stored in version control anyway.
> [...]
> Anyways, you didn't really ask any specific questions, so I'm not
> surprised you didn't get any replies. First, get the existing site
> into a Mercurial repo and then make a clone that's far from
> production, so you can play around. Try something and see if it works.
> If not, try something else. If you get confused, ask here for help.
>
> There are four specific questions that's keeping us worried.
...and all of them are beyond the scope of this list. You're looking for several things:
* a build system for multiple environments (dev/test/production),
* deployment automation (get the code onto the various servers, with the right config settings for each environment),
* database schema versioning (keeping the structure consistent across deployments and environments),
...etc. That should give you some key terms to google for. Since you're using Drupal, here's a link that might help to start with (found by googling "drupal deployment": http://stackoverflow.com/questions/730138/whats-best-drupal-deployment-strategy
For schema versioning, Drupal has its own stuff built in. Modules can hook into this.
For deployment automation, you might check out Fabric or Capistrano. Depends on your needs.
Again, this is all beyond the scope of this list. Hopefully this gives you a starting point for your own research.
pacem in terris / мир / शान्ति / سَلاَم / 平和
Kevin R. Bullock