Running new rails app *and* legacy app on top of the same database.

[Kevin Bedell]

I'm working on a project where we're building several new rails apps that are intended to be launched running along side a legacy app that will need to use the same database.

I believe it would be ideal to just cut-over cleanly and not have both the new and legacy apps running on the same db, but in this case it's simply not possible. The legacy app is too large and runs a pretty complex system that's the core of the business. The intention is to replace it a bit at a time.

But to do that will require that we launch rails apps each handling part of the application -- and sharing the database with the currently operating legacy app.

I've looked around quite a bit and haven't been able find any definitive rails docs describing what issues we might run into -- or even if it's possible. I know that Heroku now supports running multiple rails apps pointing at a single database. I've found lots of discussion on people doing this, but nothing that really pinpoints where the potential risks are. 

Anyone have experience or insight to share?

Thanks,

Kevin

[Andrew Kuklewicz]

Technically, there is not much difference between 2 different rails apps hitting the same database, and multiple instances of the same app (maybe even on multiple boxes) hitting the same db.  You also see folks have 2 different front-ends for the same app (old and new, current and beta) and allow switching back and forth - at least some of the time, that is the same db (or data services) underneath.

The issue is with the data.  If one of the apps is enforcing different validations, then you can have issues, as a simple example - one of them requires a title, the other does not - now what?

So now you basically get into the same issues whenever multiple apps share data.

Do the use the same possible values / taxonomies for things and data?

If the new app uses the same data as is, or with only additive fields, that works better, but what if you change the structure - something that used to be a many-to-one is now a many-to-many?  You will be limited in what changes you can make and maintain data consistency.

There may be other dangers lurking in your particular app, but that is what comes to mind.

- A

Andrew Kuklewicz

--
You received this message because you are subscribed to the Google Groups "Boston Ruby Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to boston-rubygro...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Patrick Robertson]

I think another problem is that if both apps can update the same record at the same time you have a race condition in which no one wins. Another potential issue is if you are using caching sweepers to throw out stale data you can easily miss updates from the other app.

- Patrick

[Andrew Kuklewicz]

The race condition stuff is pretty similar to 2 instances updating the same record - not sure this is a unique issue to this set-up.

Agreed on the caching - good one.

Andrew Kuklewicz

[Peter Degen-Portnoy]

Kudo's to you for moving forward and away from the older application; this is often a difficult decision to make and to convince folks that it is important to keep the infrastructure up to date.

Will you have any session based issues?  Something like wanting a session that starts on the new app to continue to be serviced by the new app?  Images, javascript files, AJAX requests, etc. likely fall into this category. If so, then you'll need to do some session based routing before you get to your webheads.

pdp

[Kevin Bedell]

Will you have any session based issues?  Something like wanting a session that starts on the new app to continue to be serviced by the new app?  Images, javascript files, AJAX requests, etc. likely fall into this category. If so, then you'll need to do some session based routing before you get to your webheads.

That's a good point. If the new application initially is responsible for servicing only part of the user's needs, then making sure both applications know the current state of that user's session variables may be an issue. It's certainly something to look for.

[Andrew Kuklewicz]

I have pulled it off before to share the session if the you have the same secret.

Andrew Kuklewicz

On Wed, May 8, 2013 at 3:32 PM, Kevin Bedell <ke...@kbedell.com> wrote:

Will you have any session based issues?  Something like wanting a session that starts on the new app to continue to be serviced by the new app?  Images, javascript files, AJAX requests, etc. likely fall into this category. If so, then you'll need to do some session based routing before you get to your webheads.

That's a good point. If the new application initially is responsible for servicing only part of the user's needs, then making sure both applications know the current state of that user's session variables may be an issue. It's certainly something to look for.

[Andrew Kuklewicz]

…and set the session to be a cookie on the top level domain, so all sub domains can access.

Andrew Kuklewicz

[Daniel Choi]

Whatever happens, Kevin, it would be very interesting to read a future blog post by you about this experience. You have my HN upvote. Good luck on this semi-perilous undertaking

-- Dan

[Robert Thau]

Andrew Kuklewicz writes:
> The issue is with the data. If one of the apps is enforcing different
> validations, then you can have issues, as a simple example - one of them
> requires a title, the other does not - now what?
>
> So now you basically get into the same issues whenever multiple apps share
> data.
> Do the use the same possible values / taxonomies for things and data?

To restate, the danger here is that one app puts stuff in the database
that the other(s) gag on. One way to try to deal with that is to put
a level of validation inside the schema itself, using check constraints,
foreign key constraints, and whatever else your particular DB happens
to support...

rst

[Johnny Boursiquot]

I read a relevant blog post today that may help as well: http://onstartups.com/tabid/3339/bid/97052/Screw-You-Joel-Spolsky-We-re-Rewriting-It-From-Scratch.aspx

Good luck.

On Wednesday, May 8, 2013 2:42:51 PM UTC-4, Kevin Bedell wrote:

[James Tikalsky]

Unless there's a strong technical reason for the two applications to share a major component (the schema), treat the two applications as completely separate, with no shared database tables. Instead, build a data syncing process. This has a few advantages, IMHO:

* The new application's schema design won't be limited by the old one.

* The data syncing process can act as an audit, and raise alarms if the new application's data can't be moved to the old one, an vice versa.

* The old app can receive bug fixes and changes more readily, because only the syncing process will need modification.

* The new app can take bigger risks and move forward more quickly, because the business can always rely on the old app.

* The syncing process can help you populate test databases as well.

James

--

James Tikalsky

--

You received this message because you are subscribed to the Google Groups "Boston Ruby Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to boston-rubygro...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
James Tikalsky
3 Nathan Pratt Drive Unit 308
Concord, MA 01742

[Kevin Bedell]

Unless there's a strong technical reason for the two applications to share a major component (the schema), treat the two applications as completely separate, with no shared database tables. Instead, build a data syncing process. This has a few advantages, IMHO:

* The new application's schema design won't be limited by the old one.

* The data syncing process can act as an audit, and raise alarms if the new application's data can't be moved to the old one, an vice versa.

* The old app can receive bug fixes and changes more readily, because only the syncing process will need modification.

* The new app can take bigger risks and move forward more quickly, because the business can always rely on the old app.

* The syncing process can help you populate test databases as well.

James

--

James Tikalsky

This is a really interesting viewpoint. There's a lot of sense in the idea of letting the new application develop with its own schema. Especially since twisting rails and ActiveRecord to strap it onto a schema that breaks a lot of 'rails' conventions (non-standard column names, multi-column primary keys, etc) will present a lot of challenge and keep us from getting a lot of the benefits that rails brings to the table. 

But on the other hand, building a synch process that's bidirectional between the two apps (we can't retire the legacy app until we can do full-feature replacement of it) will be time consuming and could absorb most of the dev staff just on its own. 

But maybe this could work on a limited basis with subsets of the data to begin with. We could setup, for example, new tables for some for some of the business objects in the app that follow rails conventions and then synch that data to/from the legacy tables in the same schema relatively easily. Certainly it's worth considering.

Anyone ever handle migration of a large, legacy app this way?

-Kevin

[Kevin Bedell]

On Wed, May 8, 2013 at 10:03 PM, Johnny Boursiquot <jbour...@gmail.com> wrote:

> I read a relevant blog post today that may help as well: http://onstartups.com/tabid/3339/bid/97052/Screw-You-Joel-Spolsky-We-re-Rewriting-It-From-Scratch.aspx

Great article Johnny. Thanks so much for sharing it - 

-Kevin

[Kevin Bedell]

Andrew - 

So you stored the session info in the database and got different rails apps to 'share' the session objects across apps by using identical 'Application.config.secret_token' values?

Pretty ingenious.  

[Gary Pinkham]

I had inherited a similar setup at a previous job..  It was only one way but still we had numerous issues with constraints, data oddities, caching etc..  The sync system has to be rock solid and resilient..   For instance there was a constraint on the new system that wasn't in the old.. someone would add the data to the old and the sync would fail to copy it over to the new system..   and it would fail silently (uggg!)..    It would be days sometimes before someone noticed the new data wasn't in the new system..      At any rate..   I would agree with you that you might spend a lot of dev cycles building and maintaining a sync system and have little time for other business features..   It consumed a ton of our time..     But I'm not sure you'd spend less time with another answer..   just my two cents on the one I had experienced.. 

best of luck!

now I need to get back to my Java code (ugggg)

--
You received this message because you are subscribed to the Google Groups "Boston Ruby Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to boston-rubygro...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
"only the last turn of a bolt tightens it, the rest is just movement."

[Andrew Kuklewicz]

I think it should work in a cookie or persisted - b/c both apps use the same domain and secret, they can share the cookie/session.

I've done projects like this twice. Once where we synched the data, and ran in parallel.

That was a nightmare - the complexity of the transformations created many issues, and latency.

I am not saying don't go the synch route, but do be aware of the issues involved, and how much work it will be.  If it is a simple db, with only smaller differences in schemas, and simpler synch logic, this can work I believe, but ours did not have those characteristics.

More lately, moving a java app to rails, much of the point was to make major schema changes and clean/transform data in ways that would be very hard to synch in real time. We opted to not run in parallel, and migrate the data and flip the switch when the new app was ready.  Not in the cards for you in seems, but that was the best option for us.  We made all our migration scripts as ruby code, sql, and rake tasks, and wrote tests to make sure the data looked right after each step.

Now, if you do decide to ride on the existing db you have some excellent tools.

AR has a lot of flexibility with aliasing columns, setting the primary key column, and the table name, to allow you to operate even on non-default AR tables.  You can also add columns (with defaults or triggers to make the old app work) without too much pain, so if a surrogate key or type column is needed, not a big big deal.

 

For keeping the same db but making tables more rails-flavored, views can be helpful.

For example, you can use them to break up or combine tables, or rename columns.

Now, they are read-only (most of the time, depends on the dbms), so updates will need special handling - but you can end up hiding most of this complexity in your models and db (with views/stored procs), and leave the rest of your app clean.

Then, after you have closed down the old app, you can get rid of the tech debt by actually updating the old schema, and simplifying the models back to default behavior - or just leave it be, in cases where it is cosmetic, and stable (depends on how much of a purist you are, and how much time you have to throw at such idealism).

   

This is an old prez related to this, but then again, it is an old problem :)

http://www.slideshare.net/napcs/rails-and-legacy-databases-railsconf-2009

Hope that helps,

- Andrew

Andrew Kuklewicz

--

[Scott Schulthess]

As far as your long-term architecture - and not just your transition - whenever you have multiple apps that use the same database to me it's a huge code smell.  If they have that many shared concerns then they are probably too similar to be a separate rails app.  "Things that change together should stay to together".   

Having multiple smalls rails apps that share lots of loose dependencies is worse than having a big rails app that's slower to startup and whatnot.   Where do databases migrations go?   Deployments are no longer atomic.

I would however be fine with running a legacy app and a rewrite along side each other using the same database as a temporary thing as part of a transition though, though I'd try to only do it for a couple of weeks.

On Wed, May 8, 2013 at 2:42 PM, Kevin Bedell <ke...@kbedell.com> wrote:

--

You received this message because you are subscribed to the Google Groups "Boston Ruby Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to boston-rubygro...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Sincerely,

Scott Schulthess
Phone: (978) 376 0468
scottschulthess at gmail dot com

[David Rogers]

To add a sort of, "test your assumptions" viewpoint:

At a previous job, we had a fairly mangled codebase (our application.js, most css, application controller and most other controllers and views were a complete disaster and not touchable at all) and the only real answer, because we had so few dev resources, who hadn't really the experience to take part in a massive swap over from one app to another, ops-wise, was to start with new routes, controllers, js, css and views (most importantly layouts), using namespacing of controllers and routes quite a bit, to make it basically a new app, running on the same database connections and models as before.  The idea was that once the older views were deprecated, we could reclaim things we thought we should be using such as our application layout, application js, etc.  As someone mentioned could be a problem earlier, lucky for us, the business logic in our models hardly had to change to support the new controllers, views, js, css, etc., so we weren't faced with having to write a ton of complex business logic to support different use cases for different views.  Because we could namespace our routes, controllers, views, etc. so easily in this way, it almost felt like a greenfield app because we didn't have to deal with creating bugs in the pre-existing code and morale stayed fairly high.

I mention this to emphasize that it also depends on the team you have working for you.  We were rails devs not particularly experienced in running multiple apps side by side or the ops requirements that go into that, or used to massive data migrations as an expert db would have been able to do in his/her sleep.  For us, that sounded way too complex and would be a disaster waiting to happen.  That said, we had to keep knocking out new features and redesigns, etc, so it was easier to just start over in the way we did.

- Dave