seed_fu v. populate data from migration

[Thufir]

Why does this migration fail? Using seed_fu was suggested, which I don't
mind, as it looks quite convenient, but I'm scratching my head over the
error:

thufir@ARRAKIS:~/projects/rss2mysql$
thufir@ARRAKIS:~/projects/rss2mysql$ nl db/migrate/*
1 class CreateUrls < ActiveRecord::Migration
2 def self.up
3 create_table :urls do |t|
4 t.column :url, :string
5 end
6 # Url.create(:url => 'http://www.slashdot.org/index.rss')
7 # Url.create(:url => 'http://groups.google.ca/group/ruby-talk-
google/feed/rss_v2_0_msgs.xml')
8 end

9 def self.down
10 drop_table :urls
11 end
12 end


13 class CreateItems < ActiveRecord::Migration
14 def self.up
15 create_table :items do |t|
16 t.column :title, :string
17 t.column :content, :string
18 t.column :source, :string
19 t.column :url, :string
20 t.column :timestamp, :timestamp
21 t.column :keyword_id, :integer
22 t.column :guid, :string
23 end
24 end

25 def self.down
26 drop_table :items
27 end
28 end
29 class CreatePages < ActiveRecord::Migration
30 def self.up
31 create_table :pages do |t|
32 t.references :item
33 t.column :page, :text
34 end
35 end

36 def self.down
37 drop_table :pages
38 end
39 end
thufir@ARRAKIS:~/projects/rss2mysql$
thufir@ARRAKIS:~/projects/rss2mysql$ rake
(in /home/thufir/projects/rss2mysql)
== CreateUrls: migrating
=====================================================
-- create_table(:urls)
-> 0.0472s
== CreateUrls: migrated (0.0480s)
============================================

== CreateItems: migrating
====================================================
-- create_table(:items)
-> 0.0190s
== CreateItems: migrated (0.0198s)
===========================================

== CreatePages: migrating
====================================================
-- create_table(:pages)
-> 0.0255s
== CreatePages: migrated (0.0260s)
===========================================

thufir@ARRAKIS:~/projects/rss2mysql$
thufir@ARRAKIS:~/projects/rss2mysql$ rake VERSION=0
(in /home/thufir/projects/rss2mysql)
== CreatePages: reverting
====================================================
-- drop_table(:pages)
-> 0.0039s
== CreatePages: reverted (0.0047s)
===========================================

== CreateItems: reverting
====================================================
-- drop_table(:items)
-> 0.0031s
== CreateItems: reverted (0.0376s)
===========================================

== CreateUrls: reverting
=====================================================
-- drop_table(:urls)
-> 0.0070s
== CreateUrls: reverted (0.0077s)
============================================

thufir@ARRAKIS:~/projects/rss2mysql$
thufir@ARRAKIS:~/projects/rss2mysql$ nano db/migrate/001_create_urls.rb
thufir@ARRAKIS:~/projects/rss2mysql$
thufir@ARRAKIS:~/projects/rss2mysql$ nl db/migrate/001_create_urls.rb
1 class CreateUrls < ActiveRecord::Migration
2 def self.up
3 create_table :urls do |t|
4 t.column :url, :string
5 end
6 Url.create(:url => 'http://www.slashdot.org/index.rss')
7 Url.create(:url => 'http://groups.google.ca/group/ruby-talk-
google/feed/rss_v2_0_msgs.xml')
8 end

9 def self.down
10 drop_table :urls
11 end
12 end


thufir@ARRAKIS:~/projects/rss2mysql$
thufir@ARRAKIS:~/projects/rss2mysql$ rake
(in /home/thufir/projects/rss2mysql)
== CreateUrls: migrating
=====================================================
-- create_table(:urls)
-> 0.0417s
rake aborted!
An error has occurred, all later migrations canceled:

uninitialized constant CreateUrls::Url
/home/thufir/projects/rss2mysql/rakefile:9
(See full trace by running task with --trace)
thufir@ARRAKIS:~/projects/rss2mysql$





thanks,

Thufir

[Jeffrey L. Taylor]

Quoting Thufir <hawat....@gmail.com>:
> Why does this migration fail? Using seed_fu was suggested, which I don't
> mind, as it looks quite convenient, but I'm scratching my head over the
> error:
>

From http://rails.rubyonrails.org/

reset_column_information()

Resets all the cached information about columns, which will cause them to be reloaded on the next request.

The most common usage pattern for this method is probably in a migration, when just after creating a table you want to populate it with some default values, eg:

class CreateJobLevels < ActiveRecord::Migration
def self.up
create_table :job_levels do |t|
t.integer :id
t.string :name

t.timestamps
end

JobLevel.reset_column_information
%w{assistant executive manager director}.each do |type|
JobLevel.create(:name => type)
end
end

def self.down
drop_table :job_levels
end
end

[Thufir]

On Nov 21, 2:48 am, "Jeffrey L. Taylor" <r...@abluz.dyndns.org> wrote:

> Quoting Thufir <hawat.thu...@gmail.com>:
>
> > Why does this migration fail?  Using seed_fu was suggested, which I don't
> > mind, as it looks quite convenient, but I'm scratching my head over the
> > error:
>

> Fromhttp://rails.rubyonrails.org/

>
>  reset_column_information()
>
> Resets all the cached information about columns, which will cause them to be reloaded on the next request.

[...]

Which I'll try, but *why* would it fail, given that the table
information has been deleted with the "rake VERSION=0" command, which
dropped the table? There's no table info cached, is there? If so,
where?


-Thufir

[Thufir]

On Nov 21, 2:48 am, "Jeffrey L. Taylor" <r...@abluz.dyndns.org> wrote:
> Quoting Thufir <hawat.thu...@gmail.com>:
>

> > Why does this migration fail?  Using seed_fu was suggested, which I don't
> > mind, as it looks quite convenient, but I'm scratching my head over the
> > error:
>

> Fromhttp://rails.rubyonrails.org/

>
>  reset_column_information()
>
> Resets all the cached information about columns, which will cause them to be reloaded on the next request.

[Frederick Cheung]

On Nov 21, 6:23 am, Thufir <hawat.thu...@gmail.com> wrote:
> (in /home/thufir/projects/rss2mysql)
> ==  CreateUrls: migrating
> =====================================================
> -- create_table(:urls)
>    -> 0.0417s
> rake aborted!
> An error has occurred, all later migrations canceled:
>
> uninitialized constant CreateUrls::Url
> /home/thufir/projects/rss2mysql/rakefile:9

Do you have a Url model ?

Fred

[Jeffrey L. Taylor]

The table information has been deleted from the database and the schema
changed, the table classes need to be recreated from the new DB schema. Rails
does not try and modify the model class to to match the new schema, it just
re-creates when you signal that you are done modifying the schema AND that you
want to use the new schema/model.

So the short answer is, contrary to your expectation of how Rails works, the
DB info is effectively cached in the model class's existence.

Jeffrey

[Jeffrey L. Taylor]

If you separated the migration and the population with seed data into two
migrations, the reset_column_info call would not be needed.

Jeffrey

[Thufir]

On Nov 21, 5:01 pm, Frederick Cheung <frederick.che...@gmail.com>
wrote:
[...]

> > uninitialized constant CreateUrls::Url
> > /home/thufir/projects/rss2mysql/rakefile:9
>
> Do you have a Url model ?

[...]

Certainly. The rake migration, or migration to version, worked fine
providing no data was written to the db from the migration. In any
event, the models:

thufir@ARRAKIS:~/projects/rss2mysql$ nl url.rb item.rb page.rb
1 require 'rubygems'
2 require 'activerecord'

3 class Url < ActiveRecord::Base
4 has_many :items
5 end
6 require 'rubygems'
7 require 'activerecord'

8 class Item < ActiveRecord::Base
9 has_one :page
10 belongs_to :feed
11 validates_presence_of :feed_id
12 end

13 require 'rubygems'
14 require 'activerecord'

15 class Page < ActiveRecord::Base
16 belongs_to :item
17 validates_presence_of :item_id
18 end
thufir@ARRAKIS:~/projects/rss2mysql$


Although I think I'll change "Url" back to "Feed" and perhaps "Item"
to "Article".


-Thufir

[Rick DeNatale]

On Sun, Nov 22, 2009 at 3:05 PM, Thufir <hawat....@gmail.com> wrote:
>
>
> On Nov 21, 5:01 pm, Frederick Cheung <frederick.che...@gmail.com>
> wrote:
> [...]
>> > uninitialized constant CreateUrls::Url
>> > /home/thufir/projects/rss2mysql/rakefile:9
>>
>> Do you have a Url model ?
> [...]
>
> Certainly.  The rake migration, or migration to version, worked fine
> providing no data was written to the db from the migration.

Because when you had those lines commented out, there was no reference
to a Url Ruby class.

 In any
> event, the models:
>
> thufir@ARRAKIS:~/projects/rss2mysql$ nl url.rb item.rb page.rb
>     1  require 'rubygems'
>     2  require 'activerecord'
>
>     3  class Url <  ActiveRecord::Base
>     4   has_many :items
>     5  end
>     6  require 'rubygems'
>     7  require 'activerecord'
>
>     8  class Item <  ActiveRecord::Base

Well the message is saying that there is no constant named either Url
or CreateUrls::Url defined. This is a standard Ruby message. You
probably know this but, just in case you don't, when you reference a
unprefixed constant name Ruby first looks for it in the root
namespace, and if it doesn't find it it looks in the namespace of the
current Class/Module if there is one.

So I suspect that either:

1. The automatic loading of ActiveSupport didn't match up Url with
the name of the file containing the class definition for the Url
model, or
2. The file isn't there at the time you ran the migration because
of a different version control checkout.

I suspect that it's the first assuming that there were no repository
operations in the mean time.

I'm also suspicious of your nl command parameters.

First my version of nl only takes one file name not multiples. Are
there really 3 files being concatenated together before being
numbered? I can't tell. I guess your's does work that way.

Second you seem to be executing this command from the root of the
project directory, and that's not somewhere where activesupport will
find the class files, model class files need to be in under the
apps/models directory structure.

And on my item 2 above, it's generally bad ju-ju to depend on the
external definition of model classes within migrations. Instead you
should create the inside:

class CreateUrls < ActiveRecord::Migration
class Url < ActiveRecord::Base
end

def self.up
create_table :urls do |t|
t.column :url, :string
end

Url.create(:url => 'http://www.slashdot.org/index.rss')

Url.create(:url => 'http://groups.google.ca/group/ruby-talk-
google/feed/rss_v2_0_msgs.xml')

end

This isolates the migration from the application code as it changes.
The model will be scoped within the migration class.
The only thing you need to include in the model class for the migration are:
The fact that it's a subclass of ActiveRecord::Base
Any association declarations needed by the migration
and
any special methods needed by the migration, copied from the real
class (possibly modified) as it needs to be at the state of the
database represented by THIS migration.

But using seeds rather than loading data in a migration is usually
wiser. Data manipulation in a migration should be reserved to do
things like migrating existing data to and from a new schema when the
db can't do it for you automatically.


HTH.

--
Rick DeNatale

Blog: http://talklikeaduck.denhaven2.com/
Twitter: http://twitter.com/RickDeNatale
WWR: http://www.workingwithrails.com/person/9021-rick-denatale
LinkedIn: http://www.linkedin.com/in/rickdenatale

[Thufir]

On Nov 22, 12:54 pm, Rick DeNatale <rick.denat...@gmail.com> wrote:
[...]
> > Certainly.  The rake migration, or migration to version, worked fine
> > providing no data was written to the db from the migration.
>
> Because when you had those lines commented out, there was no reference
> to a Url Ruby class.

Exactly, just process of elimination that those lines were
problematic.

[...]

> Well the message is saying that there is no constant named either Url
> or CreateUrls::Url defined.  This is a standard Ruby message.  You
> probably know this but, just in case you don't, when you reference a
> unprefixed constant name Ruby first looks for it in the root
> namespace, and if it doesn't find it it looks in the namespace of the
> current Class/Module if there is one.
>
> So I suspect that either:
>
>    1. The automatic loading of ActiveSupport didn't match up Url with
> the name of the file containing the class definition for the Url
> model, or

See, I was thinking there was a name space problem, but "Url" was
probably a poor choice.

>    2. The file isn't there at the time you ran the migration because
> of a different version control checkout.

I haven't checked this into subversion/etc yet, but I should.

> I suspect that it's the first assuming that there were no repository
> operations in the mean time.
>
> I'm also suspicious of your nl command parameters.
>
> First my version of nl only takes one file name not multiples.  Are
> there really 3 files being concatenated together before being
> numbered?  I can't tell.  I guess your's does work that way.

hmm, "nl *" will do the whole directory for me, bog standard ubuntu,
FWIW. Yes, it's a bit vague, so I won't do that in the future.

> Second you seem to be executing this command from the root of the
> project directory, and that's not somewhere where activesupport will
> find the class files, model class files need to be in under the
> apps/models directory structure.

Ah, that might very well be the problem -- it's not a rails app per
se, just AR at this stage. I'll probably add rails to it, but may add
a ruby FOX (?) GUI as well or instead.

> And on my item 2 above, it's generally bad ju-ju to depend on the
> external definition of model classes within migrations.  Instead you
> should create the inside:

Ok, this is the crux of the issue and the solution for my particular
road block :)

>   class CreateUrls < ActiveRecord::Migration
>         class Url < ActiveRecord::Base
>         end

What I did was to add: require 'url'
and that fixed the problem. Although, it's black magic, to me, as to
how ruby found the correct Url class, but "works for now." I don't
like the way the model is defined in multiple places as shown above.
However, please make your case on the bad ju-ju! It's more
conventional to define the model in the migration?

>         def self.up
>           create_table :urls do |t|
>             t.column :url, :string
>           end
>          Url.create(:url => 'http://www.slashdot.org/index.rss')
>          Url.create(:url => 'http://groups.google.ca/group/ruby-talk-
> google/feed/rss_v2_0_msgs.xml')
>        end
>
> This isolates the migration from the application code as it changes.

But the model is "defined" twice, sorta.

> The model will be scoped within the migration class.

yes.

> The only thing you need to include in the model class for the migration are:
>     The fact that it's a subclass of ActiveRecord::Base
>      Any association declarations needed by the migration

would you expand? I'm not sure what an association declaration is.

>   and
>      any special methods needed by the migration, copied from the real
> class (possibly modified) as it needs to be at the state of the
> database represented by THIS migration.

copied? That sounds like a maintenance nightmare...?

In "real apps" are you really mucking with the db that much with
migrations? It just *sounds* fragile, as described above.

>
> But using seeds rather than loading data in a migration is usually
> wiser.  

Right, for now I think I'll move on, but will come back to seeds.

> Data manipulation in a migration should be reserved to do
> things like migrating existing data to and from a new schema when the
> db can't do it for you automatically.
>
> HTH.
>
> --
> Rick DeNatale

Yes, very helpful. From my perspective, it seems like the black-box
magic which ruby and rails perform is sometimes inconsistent. Why
does *only* that one migration, which populates the db, require an
explicit class definition or import while the other migrations don't?

Actually, it's working now (I think) so I'm just curious on a few
points.



thanks to all,

Thufir

[Frederick Cheung]

On Nov 24, 5:17 am, Thufir <hawat.thu...@gmail.com> wrote:

> Ah, that might very well be the problem -- it's not a rails app per
> se, just AR at this stage.  I'll probably add rails to it, but may add
> a ruby FOX (?) GUI as well or instead.

Does that mean that ActiveSupport::Dependencies.load_paths is not
configured ? (if you're not loading rails itself and you're not doing
this explicity then the answer is no). What the setting does is tell
ActiveSupport where to look for files when it needs to find a not yet
loaded class.

>
> > And on my item 2 above, it's generally bad ju-ju to depend on the
> > external definition of model classes within migrations.  Instead you
> > should create the inside:
>
> Ok, this is the crux of the issue and the solution for my particular
> road block :)
>
> >   class CreateUrls < ActiveRecord::Migration
> >         class Url < ActiveRecord::Base
> >         end
>
> What I did was to add:   require 'url'
> and that fixed the problem.  Although, it's black magic, to me, as to
> how ruby found the correct Url class, but "works for now."  I don't
> like the way the model is defined in multiple places as shown above.
> However, please make your case on the bad ju-ju!  It's more
> conventional to define the model in the migration?
>

Yes. The problem with using the external definition of the model is
that when you deploy your code, the code all gets deployed in one go,
but your migrations get run one by one. For example, in migration 3
you might decide to add a new column to Url and add a validation on
that column. Now if you deployed your code and tried to run migration
1 it would fail, because even though the database does not yet have
that column Rails would still try to run a validation against it.

Fred