timestamped migrations
Andrew Stone
Pratik suggested I send an email to this list to discuss making timestamped migrations optional. I have submitted a ticket which includes a patch. I am a bit confused why the ticket was marked as incomplete as the solution seems rather simple. IMHO, making this optional would have saved the effort of removing code.
So, here's my take:
From a single developer/small team perspective timestamped migrations were unnecessarily added as the standard format. The problem addressed by introducing timestamped migrations is a communication issue that doesn't apply to everyone using Rails. I disagree with making timestamped migrations mandatory instead of optional. I also think the default should be the original numbering scheme due to legacy applications, not timestamps as Pratik says in the ticket. If people really wanted timestamped migrations, they could have added the plugin.
It's the small things that annoy, here are a few examples:
1) I use Terminal a lot and sometimes copy a migration to start a new one. The timestamp numbering makes this a pain.
2) I have a project, Lockdown, that generates 5 migrations. I had to modify the next_migration_string method to add a sleep(1) call to prevent all migrations from having the same number. Big deal right? Well, timestamped migrations forced me to update Lockdown just because of this change.
3) My "old" 2.0.2 app that had a bunch of migrations were now mixed with timestamped migrations. My migrations are no longer ordered in my file view.
There are other minor issues, but don't think there's a need to get that picky, this should be example enough.
In the future, should I submit a ticket first or just post something to this list?
Thanks for all the work, Rails saved me from the Java world. :)
I hope timestamped migrations are made optional in the next release of Rails, but if not, I have a way around it. I'm writing this for other Rails users more than myself.
-andy
--
Andrew Stone
Michael Koziarski
> migrations optional. I have submitted a ticket which includes a patch. I
> am a bit confused why the ticket was marked as incomplete as the solution
> seems rather simple. IMHO, making this optional would have saved the effort
> of removing code.
It's marked incomplete because you've got the defaults backwards. UTC
migrations should be the default:
+ # Specify whether or not to use timestamps for migration numbers
+ cattr_accessor :timestamped_migrations , :instance_writer => false
+ @@timestamped_migrations = false
> There are other minor issues, but don't think there's a need to get that
> picky, this should be example enough.
I like the idea of the configuration option, just flip the default and
get some feedback from other potential users to make sure it makes
sense to them.
> In the future, should I submit a ticket first or just post something to this
> list?
Generally a ticket is fine if it's a bug fix or something 'obvious'.
But if you want to add an option or get some ideas about a new API or
have a change in mind that might be controvercial, then this list
should be your first port of call.
> Thanks for all the work, Rails saved me from the Java world. :)
>
> I hope timestamped migrations are made optional in the next release of
> Rails, but if not, I have a way around it. I'm writing this for other Rails
> users more than myself.
> -andy
>
> --
> Andrew Stone
> >
>
--
Cheers
Koz
Andrew Stone
It's marked incomplete because you've got the defaults backwards. UTC
migrations should be the default:
+ # Specify whether or not to use timestamps for migration numbers
+ cattr_accessor :timestamped_migrations , :instance_writer => false
+ @@timestamped_migrations = false
If you don't mind, could you explain why the should be the default? Or is this explanation already out there on a blog or something, I'm obviously missing something because I don't get that reasoning.
thanks,
andy
--
Andrew Stone
Michael Koziarski
> this explanation already out there on a blog or something, I'm obviously
> missing something because I don't get that reasoning.
For all the reasons it was included in the first place, fewer
conflicts on branches, etc. But most importantly, we don't want to
change this behaviour back in a minor point release. Those who want
to switch to integers again can do so easily enough, and we keep the
timestamps for people who need them.
--
Cheers
Koz
Andrew Stone
I'm not familiar with what the "etc" represents and find it funny you mention "Those who want to switch to integers again can do so easily enough". When those who wanted timestamps could have done it easily enough without it being forced upon those who don't.
You guys made the logical push of removing features from core that were "edge use cases" to plugins but then pulled a plugin into core and made the standard, without an option. That is quite confusing.
I understand the notion of "convention over configuration" and "opinionated software" but this is a step beyond.
Andrew Stone
Jan De Poorter
with timestamped migrations. I think most Rails applications get
developed by > 1 person, so timestamped migrations are the logical
choice imo.
regards,
Jan De Poorter
http://workswithruby.com
Andrew Stone
Every application that gets coded by more then one person has benefit
with timestamped migrations. I think most Rails applications get
developed by > 1 person, so timestamped migrations are the logical
choice imo.
Thanks for the input Jan. I'm not saying timestamps don't have benefits, I just disagree with the way they were introduced and the fact they were made mandatory. Git has benefits for a team environment, but not everyone is forced to use it, it's an option.
--
Andrew Stone
Jan De Poorter
migrations are a show-stopper.
To revert the question, why do you want numeric migrations that bad?
Michael Koziarski
> mention "Those who want to switch to integers again can do so easily
> enough". When those who wanted timestamps could have done it easily enough
> without it being forced upon those who don't.
The timestamped migrations are fully backwards compatible with the
integer based ones, there are no practical downsides to switching to
it beyond the Timestamp collision that you've mentioned. There are
however some aesthetic considerations which I think would be nice to
handle with an option like you've mentioned. I personally think they
look hideous and am grateful that you've taken the time to whip up the
option.
But we're not going to flip the default back just to satisfy those
aesthetic concerns and won't take a patch which does that.
--
Cheers
Koz
Andrew Stone
The timestamped migrations are fully backwards compatible with the
integer based ones, there are no practical downsides to switching to
it beyond the Timestamp collision that you've mentioned. There are
however some aesthetic considerations which I think would be nice to
handle with an option like you've mentioned. I personally think they
look hideous and am grateful that you've taken the time to whip up the
option.
But we're not going to flip the default back just to satisfy those
aesthetic concerns and won't take a patch which does that.
Since you replied while I was replying to Jan, I'll just put both here:
to Jan:
It's not about wanting numeric migrations "that bad" (I can get around it), it's more about the way the change was introduced and the lack of an option. You would have ended up with fewer code changes by making it an option so it's not like it was a great deal of work. My fix to this was simply a revert of the 2.1 commit with a few conditional lines of code added.
I still haven't gotten a good reason to make it mandatory other than the "working in a team environment benefit" and the sense it's a "because we said so" attitude. To me, this is similar to the semicolon in the path idea. By the way, that was reverted on a bug fix release: 1.2.4 so I don't get the argument about changing this on a minor release.
to Michael:
I expected it wouldn't be accepted with the default I proposed, I was just looking for some explanation for the initial change.
Keep in mind, it was Pratik's idea to start this thread, not mine. :) But I did enjoy the back-and-forth.
I'll whip up another patch when I have the time.
Michael Koziarski
> "working in a team environment benefit" and the sense it's a "because we
> said so" attitude.
With your patch it won't be mandatory any more, which is the best of both worlds
> To me, this is similar to the semicolon in the path
> idea. By the way, that was reverted on a bug fix release: 1.2.4 so I don't
> get the argument about changing this on a minor release.
It's not really anything like the semi colon issue as it functions
just fine. We have all the benefits of the old style of migrations,
with some nice branch-friendly properties.
Beyond the ugly file names are there any downsides which I'm missing?
> to Michael:
>
> I expected it wouldn't be accepted with the default I proposed, I was just
> looking for some explanation for the initial change.
>
> Keep in mind, it was Pratik's idea to start this thread, not mine. :) But
> I did enjoy the back-and-forth.
>
> I'll whip up another patch when I have the time.
Look forward to it.
> thanks,
> andy
> --
> Andrew Stone
> >
>
--
Cheers
Koz
Murray Steele
Beyond the ugly file names are there any downsides which I'm missing?
The timestamp change "solves" the problem by making it much more unlikely that collisions will happen. Because the collision is less likely to happen I can see a (admittedly super pessimistic and extreme) scenario. There's a team with very little communication and when there is a collision they freak out because they don't know what to do what with collisions being an uncommon event).
As to a numeric or timestamped (or something entirely different) migration naming scheme being the default I don't really mind (both have advantages in different scenarios), but an option to choose the mechanism get my vote every time.
Muz
Jordi Bunster
On Jul 11, 2008, at 9:24 AM, Andrew Stone wrote:
> So, here's my take:
>
> From a single developer/small team perspective timestamped
> migrations were
> unnecessarily added as the standard format. The problem addressed by
> introducing timestamped migrations is a communication issue that
> doesn't
> apply to everyone using Rails.
(Josh Susser, I copied you because this could be a good opportunity to
slip migration concordance logic into core. I'd be glad to help.)
The funny thing about all of this, is that UTC migrations were
introduced as a quick way to fix the problem of migration number
clashes. It was easier than some of the other solutions, which
would've involved changing the way schema_info works.
But then we changed schema_info anyway, to allow for interleaved
migrations (UTC or not). Now that schema_migrations exists, one can
thing of other possibilities.
For example, we could track, instead of the timestamp or number, the
complete migration name (or a hash of it). That way, we could have
multiple migrations with the same UTC or number, still have order,
still have interleaved migration support, and never[*] have collisions.
The tricky part is to do this in a backwards compatible way. It's also
important to note that many of the ideas listed here suffer from
bitrot in the schema_migrations table, which might need garbage
collection from time to time (as hashes change).
[*] Identical filenames would clash. But the only solution to that is
to hash the entire migration file[&], which some people have suggested
in the past (Josh, are you still reading?), to detect when migrations
change, and what have you. Not that there's anything that can be done
automatically in that case, but, at least the developer finds out of
the discordant migrations, and can act on it.
[&] This can get tricky too. Differences in line endings (CR, LF,
CRLF) could make life hell. And if you chose to ignore those, the day
a change in line endings fixes a migration, you have an even trickier
case of hell.
--
Jordi
Andrew Stone
2008/7/11 Michael Koziarski <mic...@koziarski.com>:Beyond the ugly file names are there any downsides which I'm missing?My understanding is that the problem that timestamped migrations is the solution to is that of multiple developers committing the same numbered migration file. This, as far as I can tell, is a problem best solved not by the framework, but with intra-team communication: "Hey everybody, I'm creating and checking in migration XXX now, update your code before you create another." it's slightly clumsy sure, but the benefits of increased team communication outweigh that awkwardness: "why are you committing a new migration?" "to add ABC to model XYZ" "oh, wow that's awesome" "yeah, I know".
This is a good example and one option, the other is maintaining a version file in the db/migrate directory that could be updated with the most current version. This "version" file could be committed (to notify team members) even if the migration is not ready to be committed. This is just a simple way to use a development practice to address this issue.
The timestamp change "solves" the problem by making it much more unlikely that collisions will happen. Because the collision is less likely to happen I can see a (admittedly super pessimistic and extreme) scenario. There's a team with very little communication and when there is a collision they freak out because they don't know what to do what with collisions being an uncommon event).
As to a numeric or timestamped (or something entirely different) migration naming scheme being the default I don't really mind (both have advantages in different scenarios), but an option to choose the mechanism get my vote every time.
Muz
Yes, options are nice.
--
Andrew Stone
Andrew Stone
(Josh Susser, I copied you because this could be a good opportunity to slip migration concordance logic into core. I'd be glad to help.)
The funny thing about all of this, is that UTC migrations were introduced as a quick way to fix the problem of migration number clashes. It was easier than some of the other solutions, which would've involved changing the way schema_info works.
But then we changed schema_info anyway, to allow for interleaved migrations (UTC or not). Now that schema_migrations exists, one can thing of other possibilities.
I thought the ability to interleave migrations was a great addition and is partially responsible for my confusion for the timestamped requirement. I can now say, "I'm using migration 007, please use 008" and when 007 is committed the next db:migrate will pick it up. Nice.
For example, we could track, instead of the timestamp or number, the complete migration name (or a hash of it). That way, we could have multiple migrations with the same UTC or number, still have order, still have interleaved migration support, and never[*] have collisions.
The tricky part is to do this in a backwards compatible way. It's also important to note that many of the ideas listed here suffer from bitrot in the schema_migrations table, which might need garbage collection from time to time (as hashes change).
[*] Identical filenames would clash. But the only solution to that is to hash the entire migration file[&], which some people have suggested in the past (Josh, are you still reading?), to detect when migrations change, and what have you. Not that there's anything that can be done automatically in that case, but, at least the developer finds out of the discordant migrations, and can act on it.
[&] This can get tricky too. Differences in line endings (CR, LF, CRLF) could make life hell. And if you chose to ignore those, the day a change in line endings fixes a migration, you have an even trickier case of hell.
I really haven't given much thought to a completely different approach because I thought the simple sequence worked well. However, maybe there is a better solution.?
--
Andrew Stone
Nik Wakelin
the UTC migrations without losing any of their (very useful)
functionality?
Along the lines of what Jordi was saying, or perhaps setting the UTC
stamp inside the migration itself rather than as part of the filename
(I haven't looked at the code; would this require loading every
migration every time you want to migrate?).
The only other problem (IMHO) with UTC migrations is the "reference problem".
I can't say to another developer on my team - "Oh yeah, migration 95
is failing on production", I have to say "that
add_whatever_to_something" migration is failing.
Its the same with Git commit hashes actually :(
--
Nik Wakelin
munky...@gmail.com
Jeremy McAnally
probably a testament as to how useful it will actually be.
I think the UTC timestamps are great. Someone earlier said this
should be solved by communication, but then people are now saying that
it's a problem that people have to communicate TOO MUCH by telling a
person the name of the migration?
My own solution to this problem is/was branched_migrations, which
allows you to have folders inside of db/migrate for each branch or
developer or whatever. This is actually less optimal than the UTC
timestamps in my opinion, but it does fix the problem of the
timestamps being ugly. Maybe we could combine the approaches, having
the UTC timestamps that can live in a folder? That way you could say
"the migration in `jeremy` that `adds_x_to_y`". It'll cut down the
project search a little. When migrating, the list could be globbed
and ordered from the directories so they're kept in sync like they
would be if they were in the root dir. So a folder structure like
this:
jeremy/
2008122108_add_things.rb
2008122216_add_things.rb
2008122503_add_things.rb
matt/
2008121914_add_things.rb
2008122108_add_things.rb
andy/
2008122113_add_things.rb
Would be normalized into this list...
2008121914_add_things.rb
2008122108_add_things.rb
2008122113_add_things.rb
2008122216_add_things.rb
2008122503_add_things.rb
...and then migrated.
I don't know if that would really solve much of the problem, but it is
a thought.
--Jeremy
--
http://jeremymcanally.com/
http://entp.com/
htpt://omgbloglol.com
My books:
http://manning.com/mcanally/
http://humblelittlerubybook.com/ (FREE!)
Chris Cruft
loaded for any migration..
James Wylder
Nik Wakelin
It's still sans-tests unfortunately :(
But give me time!
--
Nik Wakelin
+64 27 424 5433
* Blog: http://codetocustomer.com/blog
* WWR: http://workingwithrails.com/person/7401-nicholas-wakelin
* Company: http://codetocustomer.com
Pratik
> loaded for any migration..
That's a bug.
-1 on moving UTC timestamps inside the migration file.
--
Cheers!
- Pratik
http://m.onkey.org
Nik Wakelin
Could I ask why?
Cheers,
Nik
Michael Koziarski
>
> Could I ask why?
We've gone back and forth so much on the file names for migrations I
think people are beginning to lose sight of the fact that it's just
not that important. There's a stack of other bugs and patches that
could do with work...
As for why it's a bug, you and I have shared a project where we had
to hack old migrations because they referred to classes which no
longer existed. If we only required the migrations we needed to move
forwards, then it wouldn't matter as often.
Finally, the backwards compatibility issues would far out weigh the
aesthetically limited benefits in the change.
--
Cheers
Koz
Nik Wakelin
<mic...@koziarski.com> wrote:
>
>> Hey Pratik,
>>
>> Could I ask why?
>
> We've gone back and forth so much on the file names for migrations I
> think people are beginning to lose sight of the fact that it's just
> not that important. There's a stack of other bugs and patches that
> could do with work...
Fair enough. ;)
>
> As for why it's a bug, you and I have shared a project where we had
> to hack old migrations because they referred to classes which no
> longer existed. If we only required the migrations we needed to move
> forwards, then it wouldn't matter as often.
>
I totally understand why that's a bug - I've actually run into again
recently. This change wouldn't stop a fix being put into place for
that bug, and I can come up with a separate diff/patch that fixes
that.
> Finally, the backwards compatibility issues would far out weigh the
> aesthetically limited benefits in the change.
The patch works fine with any migrations that aren't timestamped
inside the class, it just ignores them.
Come to think of it though, the UTC migrations will end up generating
weird next-migration numbers.
I _really_ like the idea of getting the pretty-looking numeric
migrations back, but if the tradeoffs aren't worth it, oh well.
:D
Nik
>
>
>
> --
> Cheers
>
> Koz
Nik Wakelin
Ok, at Koz's request I'll stop arguing about what brand the bikeshed
paint should be. :p
I've updated the patch with a flipped default and a little documentation:
Cheers,
Nik
Pratik
Thanks.
--
Jordi Bunster
> Ok, at Koz's request I'll stop arguing about what brand the bikeshed
> paint should be. :p
>
> I've updated the patch with a flipped default and a little
> documentation:
>
> http://rails.lighthouseapp.com/projects/8994-ruby-on-rails/tickets/446-make-timestamped-migrations-optional#ticket-446-8
Won't this bite people who have both numbered and UTC migrations right
now? Their migration numbers go:
1, 2 ... 20080715053417, 20080715053428.
And then they move to the next Rails release, flip the non-timestamp
switch to true, and then what does their sequence look like?
1, 2, 3, 4 ... 20080715053417, 20080715053428.
Unless I'm missing something, that's not quite what they would expect,
is it?
--
Jordi
Michael Koziarski
Nope, that's why their next migration will be 20080715053429 :)
--
Cheers
Koz
Jordi Bunster
On Jul 17, 2008, at 11:32 AM, Michael Koziarski wrote:
> Nope, that's why their next migration will be 20080715053429 :)
Oh got it. So they have to explicitly renumber. Fun stuff.
--
Jordi
Andrew Stone
--
Andrew Stone
Michael Koziarski
> have not upgraded, this would save them the mess.
The default won't be changed, we've already shipped a release with the
timestamped migrations.
I'm really sorry that you feel so strongly about this issue and don't
seem to be able to get past it, but there's nothing more to discuss.
--
Cheers
Koz
Andrew Stone
I'm really sorry that you feel so strongly about this issue and don't
seem to be able to get past it, but there's nothing more to discuss.
I am was/am past it and would appreciate a better tone.
