Pratik suggested I send an email to this list to discuss making timestamped migrations optional. I have submitted a ticket<http://rails.lighthouseapp.com/projects/8994-ruby-on-rails/tickets/44...>which includes a patch. I am a bit confused why the ticket was marked as incomplete as the solution seems rather simple. IMHO, making this optional would have saved the effort of removing code.
So, here's my take:
From a single developer/small team perspective timestamped migrations were unnecessarily added as the standard format. The problem addressed by introducing timestamped migrations is a communication issue that doesn't apply to everyone using Rails. I disagree with making timestamped migrations mandatory instead of optional. I also think the default should be the original numbering scheme due to legacy applications, not timestamps as Pratik says in the ticket. If people really wanted timestamped migrations, they could have added the plugin.
It's the small things that annoy, here are a few examples: 1) I use Terminal a lot and sometimes copy a migration to start a new one. The timestamp numbering makes this a pain. 2) I have a project, Lockdown, that generates 5 migrations. I had to modify the next_migration_string method to add a sleep(1) call to prevent all migrations from having the same number. Big deal right? Well, timestamped migrations forced me to update Lockdown just because of this change. 3) My "old" 2.0.2 app that had a bunch of migrations were now mixed with timestamped migrations. My migrations are no longer ordered in my file view.
There are other minor issues, but don't think there's a need to get that picky, this should be example enough.
In the future, should I submit a ticket first or just post something to this list?
Thanks for all the work, Rails saved me from the Java world. :)
I hope timestamped migrations are made optional in the next release of Rails, but if not, I have a way around it. I'm writing this for other Rails users more than myself. -andy
> Pratik suggested I send an email to this list to discuss making timestamped > migrations optional. I have submitted a ticket which includes a patch. I > am a bit confused why the ticket was marked as incomplete as the solution > seems rather simple. IMHO, making this optional would have saved the effort > of removing code.
It's marked incomplete because you've got the defaults backwards. UTC migrations should be the default:
+ # Specify whether or not to use timestamps for migration numbers + cattr_accessor :timestamped_migrations , :instance_writer => false + @@timestamped_migrations = false
> There are other minor issues, but don't think there's a need to get that > picky, this should be example enough.
I like the idea of the configuration option, just flip the default and get some feedback from other potential users to make sure it makes sense to them.
> In the future, should I submit a ticket first or just post something to this > list?
Generally a ticket is fine if it's a bug fix or something 'obvious'. But if you want to add an option or get some ideas about a new API or have a change in mind that might be controvercial, then this list should be your first port of call.
> Thanks for all the work, Rails saved me from the Java world. :)
> I hope timestamped migrations are made optional in the next release of > Rails, but if not, I have a way around it. I'm writing this for other Rails > users more than myself. > -andy
> It's marked incomplete because you've got the defaults backwards. UTC > migrations should be the default:
> + # Specify whether or not to use timestamps for migration numbers > + cattr_accessor :timestamped_migrations , :instance_writer => false > + @@timestamped_migrations = false
If you don't mind, could you explain why the should be the default? Or is this explanation already out there on a blog or something, I'm obviously missing something because I don't get that reasoning.
> If you don't mind, could you explain why the should be the default? Or is > this explanation already out there on a blog or something, I'm obviously > missing something because I don't get that reasoning.
For all the reasons it was included in the first place, fewer conflicts on branches, etc. But most importantly, we don't want to change this behaviour back in a minor point release. Those who want to switch to integers again can do so easily enough, and we keep the timestamps for people who need them.
On Fri, Jul 11, 2008 at 9:54 AM, Michael Koziarski <mich...@koziarski.com> wrote:
> > If you don't mind, could you explain why the should be the default? Or > is > > this explanation already out there on a blog or something, I'm obviously > > missing something because I don't get that reasoning.
> For all the reasons it was included in the first place, fewer > conflicts on branches, etc. But most importantly, we don't want to > change this behaviour back in a minor point release. Those who want > to switch to integers again can do so easily enough, and we keep the > timestamps for people who need them.
I'm not familiar with what the "etc" represents and find it funny you mention "Those who want to switch to integers again can do so easily enough". When those who wanted timestamps could have done it easily enough without it being forced upon those who don't.
You guys made the logical push of removing features from core that were "edge use cases" to plugins but then pulled a plugin into core and made the standard, without an option. That is quite confusing.
I understand the notion of "convention over configuration" and "opinionated software" but this is a step beyond.
Every application that gets coded by more then one person has benefit with timestamped migrations. I think most Rails applications get developed by > 1 person, so timestamped migrations are the logical choice imo.
> On Fri, Jul 11, 2008 at 9:54 AM, Michael Koziarski <mich...@koziarski.com > > wrote:
> > If you don't mind, could you explain why the should be the > default? Or is > > this explanation already out there on a blog or something, I'm > obviously > > missing something because I don't get that reasoning.
> For all the reasons it was included in the first place, fewer > conflicts on branches, etc. But most importantly, we don't want to > change this behaviour back in a minor point release. Those who want > to switch to integers again can do so easily enough, and we keep the > timestamps for people who need them.
> I'm not familiar with what the "etc" represents and find it funny > you mention "Those who want to switch to integers again can do so > easily enough". When those who wanted timestamps could have done it > easily enough without it being forced upon those who don't.
> You guys made the logical push of removing features from core that > were "edge use cases" to plugins but then pulled a plugin into core > and made the standard, without an option. That is quite confusing.
> I understand the notion of "convention over configuration" and > "opinionated software" but this is a step beyond.
> Every application that gets coded by more then one person has benefit > with timestamped migrations. I think most Rails applications get > developed by > 1 person, so timestamped migrations are the logical > choice imo.
Thanks for the input Jan. I'm not saying timestamps don't have benefits, I just disagree with the way they were introduced and the fact they were made mandatory. Git has benefits for a team environment, but not everyone is forced to use it, it's an option.
> Every application that gets coded by more then one person has benefit > with timestamped migrations. I think most Rails applications get > developed by > 1 person, so timestamped migrations are the logical > choice imo.
> Thanks for the input Jan. I'm not saying timestamps don't have > benefits, I just disagree with the way they were introduced and the > fact they were made mandatory. Git has benefits for a team > environment, but not everyone is forced to use it, it's an option.
> I'm not familiar with what the "etc" represents and find it funny you > mention "Those who want to switch to integers again can do so easily > enough". When those who wanted timestamps could have done it easily enough > without it being forced upon those who don't.
The timestamped migrations are fully backwards compatible with the integer based ones, there are no practical downsides to switching to it beyond the Timestamp collision that you've mentioned. There are however some aesthetic considerations which I think would be nice to handle with an option like you've mentioned. I personally think they look hideous and am grateful that you've taken the time to whip up the option.
But we're not going to flip the default back just to satisfy those aesthetic concerns and won't take a patch which does that.
> The timestamped migrations are fully backwards compatible with the > integer based ones, there are no practical downsides to switching to > it beyond the Timestamp collision that you've mentioned. There are > however some aesthetic considerations which I think would be nice to > handle with an option like you've mentioned. I personally think they > look hideous and am grateful that you've taken the time to whip up the > option.
> But we're not going to flip the default back just to satisfy those > aesthetic concerns and won't take a patch which does that.
Since you replied while I was replying to Jan, I'll just put both here:
to Jan:
It's not about wanting numeric migrations "that bad" (I can get around it), it's more about the way the change was introduced and the lack of an option. You would have ended up with fewer code changes by making it an option so it's not like it was a great deal of work. My fix to this was simply a revert of the 2.1 commit with a few conditional lines of code added.
I still haven't gotten a good reason to make it mandatory other than the "working in a team environment benefit" and the sense it's a "because we said so" attitude. To me, this is similar to the semicolon in the path idea. By the way, that was reverted on a bug fix release: 1.2.4 so I don't get the argument about changing this on a minor release.
to Michael:
I expected it wouldn't be accepted with the default I proposed, I was just looking for some explanation for the initial change.
Keep in mind, it was Pratik's idea to start this thread, not mine. :) But I did enjoy the back-and-forth.
> I still haven't gotten a good reason to make it mandatory other than the > "working in a team environment benefit" and the sense it's a "because we > said so" attitude.
With your patch it won't be mandatory any more, which is the best of both worlds
> To me, this is similar to the semicolon in the path > idea. By the way, that was reverted on a bug fix release: 1.2.4 so I don't > get the argument about changing this on a minor release.
It's not really anything like the semi colon issue as it functions just fine. We have all the benefits of the old style of migrations, with some nice branch-friendly properties.
Beyond the ugly file names are there any downsides which I'm missing?
> to Michael:
> I expected it wouldn't be accepted with the default I proposed, I was just > looking for some explanation for the initial change.
> Keep in mind, it was Pratik's idea to start this thread, not mine. :) But > I did enjoy the back-and-forth.
> I'll whip up another patch when I have the time.
2008/7/11 Michael Koziarski <mich...@koziarski.com>:
> Beyond the ugly file names are there any downsides which I'm missing?
My understanding is that the problem that timestamped migrations is the solution to is that of multiple developers committing the same numbered migration file. This, as far as I can tell, is a problem best solved not by the framework, but with intra-team communication: "Hey everybody, I'm creating and checking in migration XXX now, update your code before you create another." it's slightly clumsy sure, but the benefits of increased team communication outweigh that awkwardness: "why are you committing a new migration?" "to add ABC to model XYZ" "oh, wow that's awesome" "yeah, I know".
The timestamp change "solves" the problem by making it much more unlikely that collisions will happen. Because the collision is less likely to happen I can see a (admittedly super pessimistic and extreme) scenario. There's a team with very little communication and when there is a collision they freak out because they don't know what to do what with collisions being an uncommon event).
As to a numeric or timestamped (or something entirely different) migration naming scheme being the default I don't really mind (both have advantages in different scenarios), but an option to choose the mechanism get my vote every time.
> From a single developer/small team perspective timestamped > migrations were > unnecessarily added as the standard format. The problem addressed by > introducing timestamped migrations is a communication issue that > doesn't > apply to everyone using Rails.
(Josh Susser, I copied you because this could be a good opportunity to slip migration concordance logic into core. I'd be glad to help.)
The funny thing about all of this, is that UTC migrations were introduced as a quick way to fix the problem of migration number clashes. It was easier than some of the other solutions, which would've involved changing the way schema_info works.
But then we changed schema_info anyway, to allow for interleaved migrations (UTC or not). Now that schema_migrations exists, one can thing of other possibilities.
For example, we could track, instead of the timestamp or number, the complete migration name (or a hash of it). That way, we could have multiple migrations with the same UTC or number, still have order, still have interleaved migration support, and never[*] have collisions.
The tricky part is to do this in a backwards compatible way. It's also important to note that many of the ideas listed here suffer from bitrot in the schema_migrations table, which might need garbage collection from time to time (as hashes change).
[*] Identical filenames would clash. But the only solution to that is to hash the entire migration file[&], which some people have suggested in the past (Josh, are you still reading?), to detect when migrations change, and what have you. Not that there's anything that can be done automatically in that case, but, at least the developer finds out of the discordant migrations, and can act on it.
[&] This can get tricky too. Differences in line endings (CR, LF, CRLF) could make life hell. And if you chose to ignore those, the day a change in line endings fixes a migration, you have an even trickier case of hell.
> 2008/7/11 Michael Koziarski <mich...@koziarski.com>:
>> Beyond the ugly file names are there any downsides which I'm missing?
> My understanding is that the problem that timestamped migrations is the > solution to is that of multiple developers committing the same numbered > migration file. This, as far as I can tell, is a problem best solved not by > the framework, but with intra-team communication: "Hey everybody, I'm > creating and checking in migration XXX now, update your code before you > create another." it's slightly clumsy sure, but the benefits of increased > team communication outweigh that awkwardness: "why are you committing a new > migration?" "to add ABC to model XYZ" "oh, wow that's awesome" "yeah, I > know".
This is a good example and one option, the other is maintaining a version file in the db/migrate directory that could be updated with the most current version. This "version" file could be committed (to notify team members) even if the migration is not ready to be committed. This is just a simple way to use a development practice to address this issue.
> The timestamp change "solves" the problem by making it much more unlikely > that collisions will happen. Because the collision is less likely to happen > I can see a (admittedly super pessimistic and extreme) scenario. There's a > team with very little communication and when there is a collision they freak > out because they don't know what to do what with collisions being an > uncommon event).
> As to a numeric or timestamped (or something entirely different) migration > naming scheme being the default I don't really mind (both have advantages in > different scenarios), but an option to choose the mechanism get my vote > every time.
> (Josh Susser, I copied you because this could be a good opportunity to slip > migration concordance logic into core. I'd be glad to help.)
> The funny thing about all of this, is that UTC migrations were introduced > as a quick way to fix the problem of migration number clashes. It was easier > than some of the other solutions, which would've involved changing the way > schema_info works.
> But then we changed schema_info anyway, to allow for interleaved migrations > (UTC or not). Now that schema_migrations exists, one can thing of other > possibilities.
I thought the ability to interleave migrations was a great addition and is partially responsible for my confusion for the timestamped requirement. I can now say, "I'm using migration 007, please use 008" and when 007 is committed the next db:migrate will pick it up. Nice.
> For example, we could track, instead of the timestamp or number, the > complete migration name (or a hash of it). That way, we could have multiple > migrations with the same UTC or number, still have order, still have > interleaved migration support, and never[*] have collisions.
> The tricky part is to do this in a backwards compatible way. It's also > important to note that many of the ideas listed here suffer from bitrot in > the schema_migrations table, which might need garbage collection from time > to time (as hashes change).
> [*] Identical filenames would clash. But the only solution to that is to > hash the entire migration file[&], which some people have suggested in the > past (Josh, are you still reading?), to detect when migrations change, and > what have you. Not that there's anything that can be done automatically in > that case, but, at least the developer finds out of the discordant > migrations, and can act on it.
> [&] This can get tricky too. Differences in line endings (CR, LF, CRLF) > could make life hell. And if you chose to ignore those, the day a change in > line endings fixes a migration, you have an even trickier case of hell.
I really haven't given much thought to a completely different approach because I thought the simple sequence worked well. However, maybe there is a better solution.?
Is there a way to address some of the less-than-perfect aesthetics of the UTC migrations without losing any of their (very useful) functionality?
Along the lines of what Jordi was saying, or perhaps setting the UTC stamp inside the migration itself rather than as part of the filename (I haven't looked at the code; would this require loading every migration every time you want to migrate?).
The only other problem (IMHO) with UTC migrations is the "reference problem".
I can't say to another developer on my team - "Oh yeah, migration 95 is failing on production", I have to say "that add_whatever_to_something" migration is failing.
On Sat, Jul 12, 2008 at 4:48 AM, Andrew Stone <stoneli...@gmail.com> wrote:
>> (Josh Susser, I copied you because this could be a good opportunity to >> slip migration concordance logic into core. I'd be glad to help.)
>> The funny thing about all of this, is that UTC migrations were introduced >> as a quick way to fix the problem of migration number clashes. It was easier >> than some of the other solutions, which would've involved changing the way >> schema_info works.
>> But then we changed schema_info anyway, to allow for interleaved >> migrations (UTC or not). Now that schema_migrations exists, one can thing of >> other possibilities.
> I thought the ability to interleave migrations was a great addition and is > partially responsible for my confusion for the timestamped requirement. I > can now say, "I'm using migration 007, please use 008" and when 007 is > committed the next db:migrate will pick it up. Nice.
>> For example, we could track, instead of the timestamp or number, the >> complete migration name (or a hash of it). That way, we could have multiple >> migrations with the same UTC or number, still have order, still have >> interleaved migration support, and never[*] have collisions.
>> The tricky part is to do this in a backwards compatible way. It's also >> important to note that many of the ideas listed here suffer from bitrot in >> the schema_migrations table, which might need garbage collection from time >> to time (as hashes change).
>> [*] Identical filenames would clash. But the only solution to that is to >> hash the entire migration file[&], which some people have suggested in the >> past (Josh, are you still reading?), to detect when migrations change, and >> what have you. Not that there's anything that can be done automatically in >> that case, but, at least the developer finds out of the discordant >> migrations, and can act on it.
>> [&] This can get tricky too. Differences in line endings (CR, LF, CRLF) >> could make life hell. And if you chose to ignore those, the day a change in >> line endings fixes a migration, you have an even trickier case of hell.
> I really haven't given much thought to a completely different approach > because I thought the simple sequence worked well. However, maybe there is > a better solution.?
The fact that there are two caveats to the theoretical approach are probably a testament as to how useful it will actually be.
I think the UTC timestamps are great. Someone earlier said this should be solved by communication, but then people are now saying that it's a problem that people have to communicate TOO MUCH by telling a person the name of the migration?
My own solution to this problem is/was branched_migrations, which allows you to have folders inside of db/migrate for each branch or developer or whatever. This is actually less optimal than the UTC timestamps in my opinion, but it does fix the problem of the timestamps being ugly. Maybe we could combine the approaches, having the UTC timestamps that can live in a folder? That way you could say "the migration in `jeremy` that `adds_x_to_y`". It'll cut down the project search a little. When migrating, the list could be globbed and ordered from the directories so they're kept in sync like they would be if they were in the root dir. So a folder structure like this:
On Fri, Jul 11, 2008 at 11:33 AM, Jordi Bunster <jo...@bunster.org> wrote:
> On Jul 11, 2008, at 9:24 AM, Andrew Stone wrote:
>> So, here's my take:
>> From a single developer/small team perspective timestamped migrations were >> unnecessarily added as the standard format. The problem addressed by >> introducing timestamped migrations is a communication issue that doesn't >> apply to everyone using Rails.
> (Josh Susser, I copied you because this could be a good opportunity to slip > migration concordance logic into core. I'd be glad to help.)
> The funny thing about all of this, is that UTC migrations were introduced as > a quick way to fix the problem of migration number clashes. It was easier > than some of the other solutions, which would've involved changing the way > schema_info works.
> But then we changed schema_info anyway, to allow for interleaved migrations > (UTC or not). Now that schema_migrations exists, one can thing of other > possibilities.
> For example, we could track, instead of the timestamp or number, the > complete migration name (or a hash of it). That way, we could have multiple > migrations with the same UTC or number, still have order, still have > interleaved migration support, and never[*] have collisions.
> The tricky part is to do this in a backwards compatible way. It's also > important to note that many of the ideas listed here suffer from bitrot in > the schema_migrations table, which might need garbage collection from time > to time (as hashes change).
> [*] Identical filenames would clash. But the only solution to that is to > hash the entire migration file[&], which some people have suggested in the > past (Josh, are you still reading?), to detect when migrations change, and > what have you. Not that there's anything that can be done automatically in > that case, but, at least the developer finds out of the discordant > migrations, and can act on it.
> [&] This can get tricky too. Differences in line endings (CR, LF, CRLF) > could make life hell. And if you chose to ignore those, the day a change in > line endings fixes a migration, you have an even trickier case of hell.
Would it be possible to just move the UTC timestamps to the first line of the file? Then you could number them incrementally (if you want) and use the timestamps to break the tie when there is a conflict.
On Wed, Jul 16, 2008 at 6:59 AM, James Wylder <jbob0...@gmail.com> wrote: > Would it be possible to just move the UTC timestamps to the first line of > the file? Then you could number them incrementally (if you want) and use > the timestamps to break the tie when there is a conflict.
> On Wed, Jul 16, 2008 at 6:59 AM, James Wylder <jbob0...@gmail.com> wrote: >> Would it be possible to just move the UTC timestamps to the first line of >> the file? Then you could number them incrementally (if you want) and use >> the timestamps to break the tie when there is a conflict.
>> On Wed, Jul 16, 2008 at 6:59 AM, James Wylder <jbob0...@gmail.com> wrote: >>> Would it be possible to just move the UTC timestamps to the first line of >>> the file? Then you could number them incrementally (if you want) and use >>> the timestamps to break the tie when there is a conflict.
We've gone back and forth so much on the file names for migrations I think people are beginning to lose sight of the fact that it's just not that important. There's a stack of other bugs and patches that could do with work...
As for why it's a bug, you and I have shared a project where we had to hack old migrations because they referred to classes which no longer existed. If we only required the migrations we needed to move forwards, then it wouldn't matter as often.
Finally, the backwards compatibility issues would far out weigh the aesthetically limited benefits in the change.
> We've gone back and forth so much on the file names for migrations I > think people are beginning to lose sight of the fact that it's just > not that important. There's a stack of other bugs and patches that > could do with work...
Fair enough. ;)
> As for why it's a bug, you and I have shared a project where we had > to hack old migrations because they referred to classes which no > longer existed. If we only required the migrations we needed to move > forwards, then it wouldn't matter as often.
I totally understand why that's a bug - I've actually run into again recently. This change wouldn't stop a fix being put into place for that bug, and I can come up with a separate diff/patch that fixes that.
> Finally, the backwards compatibility issues would far out weigh the > aesthetically limited benefits in the change.
The patch works fine with any migrations that aren't timestamped inside the class, it just ignores them.
Come to think of it though, the UTC migrations will end up generating weird next-migration numbers.
I _really_ like the idea of getting the pretty-looking numeric migrations back, but if the tradeoffs aren't worth it, oh well.
On Thu, Jul 17, 2008 at 1:59 AM, Nik Wakelin <munkywre...@gmail.com> wrote: > On Thu, Jul 17, 2008 at 1:44 AM, Michael Koziarski > <mich...@koziarski.com> wrote:
>>> Hey Pratik,
>>> Could I ask why?
>> We've gone back and forth so much on the file names for migrations I >> think people are beginning to lose sight of the fact that it's just >> not that important. There's a stack of other bugs and patches that >> could do with work...
> Fair enough. ;)
>> As for why it's a bug, you and I have shared a project where we had >> to hack old migrations because they referred to classes which no >> longer existed. If we only required the migrations we needed to move >> forwards, then it wouldn't matter as often.
> I totally understand why that's a bug - I've actually run into again > recently. This change wouldn't stop a fix being put into place for > that bug, and I can come up with a separate diff/patch that fixes > that.
>> Finally, the backwards compatibility issues would far out weigh the >> aesthetically limited benefits in the change.
> The patch works fine with any migrations that aren't timestamped > inside the class, it just ignores them.
> Come to think of it though, the UTC migrations will end up generating > weird next-migration numbers.
> I _really_ like the idea of getting the pretty-looking numeric > migrations back, but if the tradeoffs aren't worth it, oh well.
|
