How to generate Rails models from database schemas

[kenatsun]

I'm evaluating Rails (vs Django) for a new project.  A requirement of this project is that the Rails app(s) be able to work with a database (PostgreSQL, in case that matters) whose schemas (table definitions) have been created and, over time, will be modified independently of the Rails app(s).

Though I'm a newbie to Rails, it already seems to be that Rails' standard mode of operation is:  (1) Models are created and modified within Rails, and then (2) Rails Migration creates and modifies the database table definitions.

What I need is the opposite sequence:  (1) The database table definitions are created and modified through tools external to Rails, and then (2) the Rails models are created and modified to fit the database table definitions.

So my questions are:

1.  First and most important, is there a Gem (or some other tool in the Ruby-Rails world) that can generate a model from a database table definition?
2.  If not, presumably one could use Rails on a pre-existing database by manually editing the Rails models.  Right?
3.  If that's right, what are the requirements for the table designs to enable them to work as viable Rails models?  It seems that one requirement is that each table have a sequence-assigned primary key named "id".  Right?  Any others?

~ Thanks in advance
~ Ken

[Elizabeth McGurty]

You don't always have to work with Rails functionality to utilize a database, not sure always true for PostgreSQL.  The critical aspect is how you define your database connectivity in config/database.yml. 

Perhaps helpful, check out brilliant Ryan Bates' RailsCast:  http://railscasts.com/episodes/342-migrating-to-postgresql

Liz 

[kenatsun]

I know how to connect a Rails app to a PG database.  My question is about how to align the Rails contents (model definitions) with the database contents (table definitions), so that the app can read from and write to the tables. 

It think perhaps you're answering "yes" to my question 2 - that is, I can keep the app and database contents aligned without using "Rails functionality" by editing them manually.  Right?

That still leaves question 1 and 3 unanswered.

[Hassan Schroeder]

On Fri, May 22, 2015 at 12:44 PM, kenatsun <kena...@gmail.com> wrote:
> I'm evaluating Rails (vs Django) for a new project. A requirement of this
> project is that the Rails app(s) be able to work with a database
> (PostgreSQL, in case that matters) whose schemas (table definitions) have
> been created and, over time, will be modified independently of the Rails
> app(s).
>
> Though I'm a newbie to Rails, it already seems to be that Rails' standard
> mode of operation is: (1) Models are created and modified within Rails, and
> then (2) Rails Migration creates and modifies the database table
> definitions.
>
> What I need is the opposite sequence: (1) The database table definitions
> are created and modified through tools external to Rails, and then (2) the
> Rails models are created and modified to fit the database table definitions.
>
> So my questions are:
>
> 1. First and most important, is there a Gem (or some other tool in the
> Ruby-Rails world) that can generate a model from a database table
> definition?

If you have a table named "things", all you need is a file named e.g.
thing.rb that contains

class Thing < ActiveRecord::Base
end

That will give you access to all the fields of that table.

> 2. If not, presumably one could use Rails on a pre-existing database by
> manually editing the Rails models. Right?

See above.

> 3. If that's right, what are the requirements for the table designs to
> enable them to work as viable Rails models? It seems that one requirement
> is that each table have a sequence-assigned primary key named "id". Right?

Not exactly. You do need a primary key, and it's *easiest* with an
integer, and *easiest* if the name is "id", but not necessary; I've got
an app using string ids based on a legacy db schema (from what
was previously a Java app).

> Any others?

Depends on what you're trying to do. I'd suggest just trying it :-)

The "schemas independently modified over time" part presages
some future pain, I would think; make sure you have *really good*
test coverage.

HTH, and good luck.
--
Hassan Schroeder ------------------------ hassan.s...@gmail.com
http://about.me/hassanschroeder
twitter: @hassan
Consulting Availability : Silicon Valley or remote

[Scott Ribe]

On May 22, 2015, at 2:32 PM, kenatsun <kena...@gmail.com> wrote:
>
> My question is about how to align the Rails contents (model definitions) with the database contents (table definitions), so that the app can read from and write to the tables.

Largely, you just ignore migrations and mostly ignore Rails models.

Inherit from ActiveRecord::Base, list the attributes that can be bulk set, and you’re done. Rails connects to the db and reads the schema. Now of course you will wind up adding validations and calculations and relations and so forth, and you may have to specify the table name if it doesn’t follow RoR conventions. But really, the answer to your question is not just “yes”, it’s “yes and it’s so easy that you’re completely missing it by looking for something harder” ;-)

--
Scott Ribe
scott...@elevated-dev.com
http://www.elevated-dev.com/
https://www.linkedin.com/in/scottribe/
(303) 722-0567 voice

[Colin Law]

On 22 May 2015 at 20:44, kenatsun <kena...@gmail.com> wrote:
> I'm evaluating Rails (vs Django) for a new project. A requirement of this
> project is that the Rails app(s) be able to work with a database
> (PostgreSQL, in case that matters) whose schemas (table definitions) have
> been created and, over time, will be modified independently of the Rails
> app(s).

Google for
rails legacy database
and you will get lots of useful hits

Colin

[kenatsun]

Thanks a million to Scott, Hassan, & Colin, for this rich feast of possibilities!  It's gonna take me some time to digest.  Then I'll be back with the solutions I'm finding and/or with more questions.

~ Ken

[kenatsun]

Yaay!  I'm happy to report success with my first attempt to make a Rails app that works from an existing database. 

This first experiment used (a) the simplest "instructions" I could find (basically, Hassan's); (b) a streamlined version of the "Getting Started Guide" process for setting up a Rails app (but - this is crucial - omitting the step of running the migrator); and (c) a pre-existing ("legacy") database that conforms to the Rails naming conventions.

For the benefit of other Rails newbies (or oldbies) that need to build apps on existing DBs, a play-by-play of what I did follows.

1. Created new Rails app, x:

[sunward@web324 rails_eval]$ rails new x

2. Populated the "legacy" SQLite database with table toys, and added one row to that table:

[sunward@web324 x]$ rails dbconsole
SQLite version 3.6.20
sqlite> CREATE TABLE toys ("id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, "name" TEXT);
sqlite> INSERT INTO toys (name) VALUES ("Tinker");
sqlite> SELECT * FROM toys;
1|Tinker|58

3. Added the toys resource and root to routes.rb:     

Rails.application.routes.draw do
    resources :toys
    root 'toys#index'
end

4. Used rake to see the resulting routes:

[sunward@web324 x]$ rake routes
  Prefix Verb   URI Pattern              Controller#Action
    toys GET    /toys(.:format)          toys#index
         POST   /toys(.:format)          toys#create
 new_toy GET    /toys/new(.:format)      toys#new
edit_toy GET    /toys/:id/edit(.:format) toys#edit
     toy GET    /toys/:id(.:format)      toys#show
         PATCH  /toys/:id(.:format)      toys#update
         PUT    /toys/:id(.:format)      toys#update
         DELETE /toys/:id(.:format)      toys#destroy

5. Generated toys controller:

[sunward@web324 x]$ rails generate controller toys
      create  app/controllers/toys_controller.rb
      invoke  erb
      create    app/views/toys
      invoke  test_unit
      create    test/controllers/toys_controller_test.rb
      invoke  helper
      create    app/helpers/toys_helper.rb
      invoke    test_unit
      invoke  assets
      invoke    coffee
      create      app/assets/javascripts/toys.coffee
      invoke    scss
      create      app/assets/stylesheets/toys.scss

6. Added the necessary CRUD methods to the toys controller:

class ToysController < ApplicationController
    def index
        @toys = Toy.all
    end
   
    def show
        @toy = Toy.find(params[:id])
    end
     
    def new
        @toy = Toy.new
    end

    def create
        @toy = Toy.new(toy_params)
        if @toy.save
            redirect_to @toy
        else
            render 'new'
        end
    end
   
    def edit
        @toy = Toy.find(params[:id])
    end

    def update
        @toy = Toy.find(params[:id])
        if @toy.update(toy_params)
            redirect_to @toy
        else
            render 'edit'
        end
    end



    def destroy
        @toy = Toy.find(params[:id])
        @toy.destroy

        redirect_to toys_path
    end    
   
    private
        def toy_params
            params.require(:toy).permit(:name)
        end    
end

7. Made index.html.erb, show.html.erb, new.html.erb, edit.html.erb, _form.html.erb, index.json.jbuilder, show.json.jbuilder, following the pattern in http://guides.rubyonrails.org/getting_started.html.

8. Created the Toy model file, toy.rb:

class Toy < ActiveRecord::Base
end

9. In my browser, tried http://x.sunward.webfactional.com/toys.  Feel free to try it yourself.  (However, this is a temporary site that may be gone by the time you read this.)  It works!!!  That is, all the CRUDs execute as they should.

10. An odd note is that there still is no schema.rb file in x/db/.  If this is what Rails uses as its "database catalog", is the lack of it going to cause trouble down the road?

Thanks again to Scott, Hassan, & Colin for your key clues.

My next question is whether it is possible to sue the amazing Rails generators - in particular, scaffold - to more quickly create an app that works with an existing database - so it would not be necessary to go thru all those manual steps shown above.  I tried it once, and it tried to run the migrator and croaked when it found that the DB table was already there.  But that may have been pilot error on my part.  Comments and advice would be welcome.

~ Ken

[Nugi Nugraha]

I think you just need to create Toy model and then use scaffold controller
http://stackoverflow.com/questions/4333393/using-rails-generate-scaffold-when-model-already-exists

On Mon, Jun 1, 2015 at 7:06 AM, kenatsun <kena...@gmail.com> wrote:

My next question is whether it is possible to sue the amazing Rails generators - in particular, scaffold - to more quickly create an app that works with an existing database - so it would not be necessary to go thru all those manual steps shown above.  I tried it once, and it tried to run the migrator and croaked when it found that the DB table was already there.  But that may have been pilot error on my part.  Comments and advice would be welcome.

~ Ken

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-ta...@googlegroups.com.
To post to this group, send email to rubyonra...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/4a506cc5-5bf7-4f46-9628-b4dc6e7555b2%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Mou Dareka no, tame janakutte
Jibun no Tame ni Warette Iru

( Aqua Timez - Alones )

[Hassan Schroeder]

On Sun, May 31, 2015 at 5:06 PM, kenatsun <kena...@gmail.com> wrote:

> 10. An odd note is that there still is no schema.rb file in x/db/. If this
> is what Rails uses as its "database catalog", is the lack of it going to
> cause trouble down the road?

Possibly, but you can use `rake db:schema:dump` to generate it
from your existing DB.

> My next question is whether it is possible to sue the amazing Rails
> generators - in particular, scaffold - to more quickly create an app that
> works with an existing database - so it would not be necessary to go thru
> all those manual steps shown above. I tried it once, and it tried to run
> the migrator

I've never seen a scaffold command automatically run migrations;
can you be more specific about what led up to that?

You should be able to scaffold a model and then just delete the
migration file.

[kenatsun]

Thank you, byakugie ~

Your link led me to the gem "schema-to-scaffold", which enabled the following scenario:

The aim is to get all the components generated automatically, but based on the database schema.  I did it my using the gem schema_to_scaffold, which simply generates a script (one command in this case) that runs the scaffold generator in a way that drives off of the database schema for the new table (users).

1. Created a new database table, users:

sqlite> CREATE TABLE "users" ("id" INTEGER PRIMARY KEY

AUTOINCREMENT NOT NULL, "name" text, "email" textuser);

sqlite> INSERT INTO users (name) VALUES (‘Jo’);

sqlite> SELECT * FROM users;

1|Jo|

2. Installed schema_to_scaffold gem:

sqlite> gem install schema_to_scaffold

3. Ran schema_to_scaffold for users:

[sunward@web324 x]$ scaffold -p db

Looking for schema.rb in db

0. db/schema.rb

Select a path to the target schema:

Loaded tables:

0. toys

1. users

Options are:

4 for table 4; (4..6) for table 4 to 6; [4,6] for tables 4 and 6; * for all Tables

Select a table: 1

 

Script for scaffold:

rails generate scaffold User name:text email:text --no-migration

4. Ran the script produced by schema_to_scaffold:

[sunward@web324 x]$ rails generate scaffold User name:text email:text --no-migration

      invoke  active_record

      create    app/models/user.rb

      invoke    test_unit

      create      test/models/user_test.rb

      create      test/fixtures/users.yml

      invoke  resource_route

       route    resources :users

      invoke  scaffold_controller

      create    app/controllers/users_controller.rb

      invoke    erb

      create      app/views/users

      create      app/views/users/index.html.erb

      create      app/views/users/edit.html.erb

      create      app/views/users/show.html.erb

      create      app/views/users/new.html.erb

      create      app/views/users/_form.html.erb

      invoke    test_unit

      create      test/controllers/users_controller_test.rb

      invoke    helper

      create      app/helpers/users_helper.rb

      invoke      test_unit

      invoke    jbuilder

      create      app/views/users/index.json.jbuilder

      create      app/views/users/show.json.jbuilder

      invoke  assets

      invoke    coffee

      create      app/assets/javascripts/users.coffee

      invoke    scss

      create      app/assets/stylesheets/users.scss

      invoke  scss

      create    app/assets/stylesheets/scaffolds.scss

5. It works!  See http://x.sunward.webfactional.com/users/.

Amazing!

Next step will be to make a change in the database schema and see if/how I can get Rails to automagically update its model, controller, and views to fit. 

Any how-to hints would be welcome.

~ Ken

[kenatsun]

Thanks, Hassan ~ The rake command does the job of building (and updating) schema.rb. ~ Ken

[kenatsun]

Next step: I have succeeded in adding a column to a database table, then using the generator tools to update the Rails objects that implement that model.  The process, which I share for what it's worth to others, is:

1.      Add attribute toys.color (text):

sqlite> ALTER TABLE toys ADD COLUMN color TEXT;

sqlite> INSERT INTO toys (name, color) VALUES ('Tinker', 'Green');

sqlite> select * from toys;

1|Tinker|Green

2.      Ran rake db:schema:dump to update schema.rb:

ActiveRecord::Schema.define(version: 0) do

  create_table "toys", force: :cascade do |t|

    t.text "name"

    t.text "color"

  end

3.      Ran schema_to_scaffold for toys:

 

[sunward@web324 x]$ scaffold -p db

...

Loaded tables:

0. toys

1. users

...

Select a table: 0

Script for scaffold:

rails generate scaffold Toy name:text color:text --no-migration

4.      Ran scaffold using that script:

[sunward@web324 x]$ rails generate scaffold Toy name:text color:text --no-migration

      invoke  active_record

    conflict    app/models/toy.rb

  Overwrite /home/sunward/webapps/rails_eval/x/app/models/toy.rb? (enter "h" for help) [Ynaqdh] Y

       force    app/models/toy.rb

      invoke    test_unit

      create      test/models/toy_test.rb

      create      test/fixtures/toys.yml

      invoke  resource_route

       route    resources :toys

      invoke  scaffold_controller

    conflict    app/controllers/toys_controller.rb

  Overwrite /home/sunward/webapps/rails_eval/x/app/controllers/toys_controller.rb? (enter "h" for help) [Ynaqdh] Y

       force    app/controllers/toys_controller.rb

      invoke    erb

       exist      app/views/toys

    conflict      app/views/toys/index.html.erb

    Overwrite /home/sunward/webapps/rails_eval/x/app/views/toys/index.html.erb? (enter "h" for help) [Ynaqdh] Y

       force      app/views/toys/index.html.erb

    conflict      app/views/toys/edit.html.erb

    Overwrite /home/sunward/webapps/rails_eval/x/app/views/toys/edit.html.erb? (enter "h" for help) [Ynaqdh] Y

       force      app/views/toys/edit.html.erb

    conflict      app/views/toys/show.html.erb

    Overwrite /home/sunward/webapps/rails_eval/x/app/views/toys/show.html.erb? (enter "h" for help) [Ynaqdh] Y

       force      app/views/toys/show.html.erb

    conflict      app/views/toys/new.html.erb

    Overwrite /home/sunward/webapps/rails_eval/x/app/views/toys/new.html.erb? (enter "h" for help) [Ynaqdh] Y

       force      app/views/toys/new.html.erb

    conflict      app/views/toys/_form.html.erb

    Overwrite /home/sunward/webapps/rails_eval/x/app/views/toys/_form.html.erb? (enter "h" for help) [Ynaqdh] Y

       force      app/views/toys/_form.html.erb

      invoke    test_unit

    conflict      test/controllers/toys_controller_test.rb

    Overwrite /home/sunward/webapps/rails_eval/x/test/controllers/toys_controller_test.rb? (enter "h" for help) [Ynaqdh] Y

       force      test/controllers/toys_controller_test.rb

      invoke    helper

   identical      app/helpers/toys_helper.rb

      invoke      test_unit

      invoke    jbuilder

    conflict      app/views/toys/index.json.jbuilder

    Overwrite /home/sunward/webapps/rails_eval/x/app/views/toys/index.json.jbuilder? (enter "h" for help) [Ynaqdh] Y

       force      app/views/toys/index.json.jbuilder

    conflict      app/views/toys/show.json.jbuilder

    Overwrite /home/sunward/webapps/rails_eval/x/app/views/toys/show.json.jbuilder? (enter "h" for help) [Ynaqdh] Y

       force      app/views/toys/show.json.jbuilder

      invoke  assets

      invoke    coffee

   identical      app/assets/javascripts/toys.coffee

      invoke    scss

    conflict      app/assets/stylesheets/toys.scss

    Overwrite /home/sunward/webapps/rails_eval/x/app/assets/stylesheets/toys.scss? (enter "h" for help) [Ynaqdh] Y

       force      app/assets/stylesheets/toys.scss

      invoke  scss

   identical   

5.      It works! See http://x.sunward.webfactional.com/toys/.

However:  This process has only a limited ability to update the Rails objects to handle the database change while preserving manually entered changes to them.  Specifically:  For each Rails file where the new (generated) version differs from the existing (possibly manually modified) version, you get to choose between keeping the old version (so it doesn't match the database changes) or replacing it with the new version (so any manual updates to the old version are lost).  In other words, I haven't found a way to generate a new file that is a combination of the old and the new - that is, it is identical to the old file in all respects except that it implements the database schema changes. 

So...

Next question:  Are there any Rails tools that can generate a "combined" file in the sense just defined?  Is there any way to generate a "combined" file in the sense just defined? - either with other Rails tools or with options to the tools used in the above scenarios? 

I know that meaningfully merging two versions of an object quickly gets us into the "hard problem" zone.  But I notice that the prompts that scaffold puts out do offer to show you a "diff" of the old vs new versions of each file.  It shouldn't be too hard to give the user the opportunity to choose between the old and the new versions of each line that differs, and that would enable the user to produce the desired "combination" in most cases.  Short of that, an option to tag the new file with a slightly different file name from the old file would enable the user to use an external "diff editor" to compare and manually merge the two versions fairly efficiently.

So, I'm wondering, does the Rails World contain any tools that can do this?

~ Thanks yet again
~ Ken

[Scott Ribe]

On Jun 4, 2015, at 2:21 PM, kenatsun <kena...@gmail.com> wrote:
>
> Next step: I have succeeded in adding a column to a database table, then using the generator tools to update the Rails objects that implement that model. The process, which I share for what it's worth to others, is:
>
> 1. Add attribute toys.color (text):

> 2. Ran rake db:schema:dump to update schema.rb:

> 3. Ran schema_to_scaffold for toys:

> 4. Ran scaffold using that script:

> However: This process has only a limited ability to update the Rails objects...

I do wonder if you really understand: steps 2 through 4 were unnecessary. You can manage your database schema using Rails & migrations, or you can manage it completely outside of Rails. Either option is pretty easy, while a mishmash combination of the two is not so easy.

[Colin Law]

On 4 June 2015 at 21:21, kenatsun <kena...@gmail.com> wrote:
> ...

> However: This process has only a limited ability to update the Rails
> objects to handle the database change while preserving manually entered
> changes to them. Specifically: For each Rails file where the new
> (generated) version differs from the existing (possibly manually modified)
> version, you get to choose between keeping the old version (so it doesn't
> match the database changes) or replacing it with the new version (so any
> manual updates to the old version are lost). In other words, I haven't
> found a way to generate a new file that is a combination of the old and the
> new - that is, it is identical to the old file in all respects except that
> it implements the database schema changes.
>
> So...
>
> Next question: Are there any Rails tools that can generate a "combined"
> file in the sense just defined? Is there any way to generate a "combined"
> file in the sense just defined? - either with other Rails tools or with
> options to the tools used in the above scenarios?

The reason no work has been done on this is that the scaffold is
intended merely as a quick and dirty method of getting some basic
functionality up and running. By the time you have a real application
going it is most unlikely that much of the original scaffolding code
will remain.

Colin

[kenatsun]

You're misreading my step 1: "Add attribute toys.color (text)".  That was added to the database schema, not to the Rails model definition.  So steps 2-4 (or some equivalent process in Rails) absolutely were necessary.  Otherwise the Rails app would have gone on oblivious to the change in the DB schema.

[kenatsun]

Yeah, I figured something like that was the case.  Probably that's the case with most/all Rails generators (like most of the generators in the world) - they're good for a one-time kick-start, but not much good after that.

Still, a generator that could write on something other than a blank slate would be a very valuable addition to the toolkit.  And doable.

[Scott Ribe]

On Jun 4, 2015, at 5:26 PM, kenatsun <kena...@gmail.com> wrote:
>
> You're misreading my step 1: "Add attribute toys.color (text)". That was added to the database schema, not to the Rails model definition. So steps 2-4 (or some equivalent process in Rails) absolutely were necessary. Otherwise the Rails app would have gone on oblivious to the change in the DB schema.

No, I did not misread it. As I suspected, you still do not understand what we have been trying to tell you: Rails WILL READ THE SCHEMA FROM THE DATABASE. Your steps 2 through 4 were unnecessary, and, as I said before:

[kenatsun]

Well, it would be nice to do in one step what I thought required four!  So let me try...

1.    Add another attribute toys.size to the schema (this looks different from the previous step 1 only because I have replaced my SQLite DB with a PostgreSQL one, named sunward_x:

[sunward@web324 x]$ rails dbconsole

...
psql (9.1.15)
SSL connection (cipher: DHE-RSA-AES256-SHA, bits: 256)
sunward_x=> \d toys
                            Table "public.toys"
 Column |       Type        |                   Modifiers
--------+-------------------+-----------------------------------------------
 id     | bigint            | not null default nextval('toy_seq'::regclass)
 name   | character varying |
 color  | character varying |
 price  | money             |
sunward_x=> ALTER TABLE toys ADD size INTEGER;
ALTER TABLE
sunward_x=> \d toys
                            Table "public.toys"
 Column |       Type        |                   Modifiers
--------+-------------------+-----------------------------------------------
 id     | bigint            | not null default nextval('toy_seq'::regclass)
 name   | character varying |
 color  | character varying |
 price  | money             |
 size   | integer           |

If step 1 is all that's needed, I should be able to bring up any toys page an see the new size column, right?  But I don't:

 
As far as I can tell, at this point the app does indeed seem to be oblivious to the change in the DB schema.  If so, certainly something more needs to be done to update the app.

Perhaps I can go right ahead to my old step 4, which is to run scaffold, without updating schema.rb (old step 2) or running schema_to_scaffold (old step 3).  Perhaps, as you say, "Rails WILL READ THE SCHEMA FROM THE DATABASE" and gen everything up if I just do this.

[sunward@web324 x]$ rails generate scaffold Toy  --no-migration

Well, no.  What it has done is to gen up a Toys model (and supporting views) with no columns at all:



Maybe step 2 is necessary to do before step 4, still skipping step 3?

[sunward@web324 x]$ rake db:schema:dump
[sunward@web324 x]$ rails generate scaffold Toy  --no-migration

That does, of course, add the new column to schema.rb.  But the scaffold generator still puts out model objects that are devoid of columns; the toys web page looks just like it did before.

Of course, I can omit step 3, because all schema_to_scaffold does is "READ THE SCHEMA FROM THE DATABASE" and put out a suitable command for invoking scaffold.  But without it, I have to read the database schema myself and manually type the list of columns into the scaffold command.  Not sure what's easier about that.

So, if there really is a simpler process for doing this, I guess I need you to spell it out for me.

Hmm...  Maybe this statement from you is a clue to where we have miscommunicated: "You can manage your database schema using Rails & migrations, or you can manage it completely outside of Rails. Either option is pretty easy, while a mishmash combination of the two is not so easy."  This whole thread is not about "managing my database schema" - as stated in the original post, I intend to create and make all changes to my DB schema entirely outside of Rails.  Given that intention, this post is about managing the Rails objects that work with the database schema, and using Rails generators and other tools to automate that process as much as is possible.  If you have any ideas for how to do that faster/better/easier than the procedures that I have cobbled together so far thru trial-and-error and the hints provided in this thread, I would be grateful to hear them. 

~ Tx, Ken

[Scott Ribe]

On Jun 4, 2015, at 9:19 PM, kenatsun <kena...@gmail.com> wrote:
>
> Well, it would be nice to do in one step what I thought required four!

1. Exactly as you did, add the column to the table.

Now it's there. Use it, just like any other column. Add it to any view you wish. Do anything you wish with it in a controller.

Why on earth would you keep re-generating scaffolds over and over??? A scaffold is a starting point; it is not your app code. People have told you (over and over) that you do not maintain Rails apps by generating & re-generating scaffolds.

[Colin Law]

On 5 June 2015 at 04:19, kenatsun <kena...@gmail.com> wrote:
>
> Well, it would be nice to do in one step what I thought required four! So let me try...
>
> 1. Add another attribute toys.size to the schema (this looks different from the previous step 1 only because I have replaced my SQLite DB with a PostgreSQL one, named sunward_x:

> ...

> If step 1 is all that's needed, I should be able to bring up any toys page an see the new size column, right? But I don't:
>

> As far as I can tell, at this point the app does indeed seem to be oblivious to the change in the DB schema. If so, certainly something more needs to be done to update the app.

The *only* thing you have to do now is to add the new field to the
view. Everything else is handled automatically. All you are
achieving by running the generator is adding that field to the view.
Rails does know about the new field, it is just that the view has not
asked for it to be displayed.

Colin

[kenatsun]

Scott & Colin ~

Perhaps this conversation has reached the end of its useful life.  But in case there still is anything left to be learned, let me respond to a couple of your comments.

"Why on earth would you keep re-generating scaffolds over and over???" [Scott]

Because generators, when they work, are easier and more reliable than manually updating code.  Rails obviously has a powerful set of generators for jump-starting development.  That's one big reason why I'm spending all this time evaluating it (vs other platforms like Django). This whole thread is trying to understand the powers and limits of the Rails generators, specifically in the somewhat-Rails-nonstandard scenario where the DB schema exists independently of the Rails app, so the requirement is to tailor (and re-tailor) the Rails objects to fit the DB schema, with as much automagic support as possible.  Of course, the job of generators rapidly becomes harder when you ask them to update objects that have been modified manually since they were originally generated.  But some generators can go farther into update-land than others, and I'm trying to understand where Rails is on this dimension.

"The *only* thing you have to do now is to add the new field to the
view.  Everything else is handled automatically.  All you are
achieving by running the generator is adding that field to the view."

Not so, from my experiments.  Running "rake db:schema:dump" added the field to schema.rb, and the output from scaffold indicates that it edited a total of 8 different files for me, namely:

    conflict      test/fixtures/toys.yml
       force      test/fixtures/toys.yml
    conflict    app/controllers/toys_controller.rb
       force    app/controllers/toys_controller.rb
    conflict      app/views/toys/index.html.erb
       force      app/views/toys/index.html.erb
    conflict      app/views/toys/show.html.erb
       force      app/views/toys/show.html.erb
    conflict      app/views/toys/_form.html.erb
       force      app/views/toys/_form.html.erb
    conflict      test/controllers/toys_controller_test.rb
       force      test/controllers/toys_controller_test.rb
    conflict      app/views/toys/index.json.jbuilder
       force      app/views/toys/index.json.jbuilder
    conflict      app/views/toys/show.json.jbuilder
       force      app/views/toys/show.json.jbuilder

Not only did the generator update all these files for me, but it actually knew which files had to be updated so that my app could handle the new database column - another huge advantage for a newbie like me, who would otherwise have to spend hours figuring that out.

Were not all of these 9 edits necessary to weave the new database column into my app?

So, what's the best approach:  Run 3 simple console commands, or manually locate and edit 9 files (and debug the edits)?

I guess I see a lot more value in the Rails generators than you guys do.  And I'm still looking to see if they have still further powers (especially in update-land) that I haven't discovered yet.

[Scott Ribe]

On Jun 5, 2015, at 2:25 PM, kenatsun <kena...@gmail.com> wrote:
>
> Were not all of these 9 edits necessary to weave the new database column into my app?

Actually, no, not at all.

> So, what's the best approach: Run 3 simple console commands, or manually locate and edit 9 files (and debug the edits)?

The generators are not going to put them into your views located where you need them and styled the way your app needs them. The generators are not going to be able to write the actual tests for them. The generators are not going to have a clue how they fit with your application's logic and what might need to be done with them in your models or controllers.

[Colin Law]

On 5 June 2015 at 21:25, kenatsun <kena...@gmail.com> wrote:
> ...

> Were not all of these 9 edits necessary to weave the new database column
> into my app?

Diff the files and see what the changes were.

Colin

[kenatsun]

All of the changes were to files that specifically mentioned the model's attributes (aka the table's columns), and all the changes consisted of adding a mention of the new column.  For example, the auto-edit of show.json.jbuilder when I added the size attribute changed

json.extract! @toy, :id, :name, :color, :price, :created_at, :updated_at

to

json.extract! @toy, :id, :name, :color, :price, :size, :created_at, :updated_at

[kenatsun]

[Colin Law]

Did the generator add the validations for that attribute?

Did it add the code in the toy.rb that implements the business logic
involving that attribute?

Did it add the attribute to the views that do not just consist of the
simplistic tables and so on generated by the controller?

Did it add the required tests to make sure that all the uses of the
attribute are correct?

So is it worth the hassle of messing about with the generator just for
the few minor changes that it did make?

Colin

[kenatsun]

Comments inserted below.

Did the generator add the validations for that attribute?

Did it add the code in the toy.rb that implements the business logic
involving that attribute?

Did it add the attribute to the views that do not just consist of the
simplistic tables and so on generated by the controller?

Did it add the required tests to make sure that all the uses of the
attribute are correct?

For the Rails generators and their options that I have discovered so far, the answer to all these questions is, obviously, No.

Could there be generators that do some of these things?  Yes.

Are there generators (or other tools) in the Rails World that do some of these things?  That's what I'm trying to find out in this thread.

So is it worth the hassle of messing about with the generator just for
the few minor changes that it did make?

As you develop an app starting with from a generated set of objects, the answer at first is clearly Yes.  As you manually modify the objects produced by prior runs of the generator, at some point the answer crosses over into No.  How soon you reach that point depends on the amount and kind of manual modifications you have made, vs the power of your generators to synthesize (or to help you synthesize) their new outputs with the manual refinements.

The synthetic abilities of the Rails generator that I've tried so far (basically scaffold and whatever it invokes) seem to be limited to:

1. Detecting whether an object that it is about to generate already exists.
2. If so, detecting whether the existing and new versions of the object are identical.
3. If the versions are not identical, showing you a diff between them.
4. Giving you a choice between keeping the old version or replacing it entirely with the new version.

I haven't yet found any help from Rails tools with synthesizing the old (manually modified) and the new (generated) versions of a particular object.  I'll mention just two possibilities:

1. Even a tool that compares two files textually and helps you merge them by picking line-by-line between the two versions would be a significant help. 
   
2. Better yet would be a "smart generator" - perhaps better called a synthesizer rather than a generator - that knows the structure of Rails objects and uses that knowledge to make (or at least suggest) syntheses of old and new object versions that preserve the functionality of both.  For example, to address one of Colin's questions, suppose a "validates :name, presence: true" validation had been added to the Toy model before the new "size" column was added to the database.  A synthesizer would recognize a validation declaration, and it would know that a validation declaration on one attribute is independent of the existence of another attribute, so it would leave the validation in place and add the new attribute to all the Rails objects where it is relevant.  Or, more cautiously, the synthesizer would suggest this synthesis to the developer and let the developer decide.  A suite of synthesizers that worked like this would extend the value of running generators/synthesizers waaay deep into the Rails development process.  And such synthesizers are especially doable in Rails because they can exploit the fact - which even us newbies can see and appreciate - that Rails is a highly structured, and well-structured, system.


Sorry to get into sermonizing.  All I really want from this thread is to learn if any tools or facilities that provide additional generating/synthesizing functionality currently exist in the Rails World.

[Walter Lee Davis]

I don't know of any within the true meaning of "generator" that do all that you have outlined here. But there are systems that ape the Django admin view (RailsAdmin etc) that introspect your database and build an interface to modify it on the fly. This is similar to how the Scaffold system in Rails worked in versions less than 1 (and maybe a few minors after 1). The problem with these systems that I have encountered is that they have a well-formed opinion of how any type of field should be represented, and in what order, and they build you an 80% solution in no time at all. If you're happy with what they give you, then you're completely set. But if you want something even a little bit off the tracks they've laid, then you have a whole world of work to fake them into giving it to you.

Rails took a different path after that first foray into scaffolding-by-introspection, and decided that they would make it really cheap and easy to get a static file, and then let you modify it later as you go. I believe the thinking was that there was no way a professional would not exert themselves to make a truly custom application if that was what they were being paid to do.

One thing you might try is after you've made significant changes, try making a new stunt-double app with the generator, with the new version of your schema. Then copy the relevant files over to your actual working app. You'll be able to cherry-pick which parts of the new code make sense, so you don't have the generator overwrite something you spent time customizing.

Walter

>
> ~ Thanks yet again
> ~ Ken
>

> --
> You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-ta...@googlegroups.com.
> To post to this group, send email to rubyonra...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/abfb70b4-bd2c-4f80-896d-d9d04de1a69d%40googlegroups.com.

[kenatsun]

Thanks Walter ~

Comments interspersed below.

On Sunday, June 7, 2015 at 11:38:12 AM UTC-4, Walter Lee Davis wrote:

 

I don't know of any within the true meaning of "generator" that do all that you have outlined here. But there are systems that ape the Django admin view (RailsAdmin etc) that introspect your database and build an interface to modify it on the fly. This is similar to how the Scaffold system in Rails worked in versions less than 1 (and maybe a few minors after 1). The problem with these systems that I have encountered is that they have a well-formed opinion of how any type of field should be represented, and in what order, and they build you an 80% solution in no time at all. If you're happy with what they give you, then you're completely set. But if you want something even a little bit off the tracks they've laid, then you have a whole world of work to fake them into giving it to you.

I have some experience with Django, and it fits with what you're saying.  I'm pretty sure that Rails is ahead of Django in its ability to to the kind of generation/synchronization I'm looking for.  For example, as far as I know, Django's schema-to-model generator (inspectdb) generates only models (not controllers or views); and it can only "introspect" the whole DB and generate models for all of its tables (as opposed to Rails where you can run the generators table-by-table).  BTW if any Django experts are listening in and I have sold Django short, please steer me straight.

One thing you might try is after you've made significant changes, try making a new stunt-double app with the generator, with the new version of your schema. Then copy the relevant files over to your actual working app. You'll be able to cherry-pick which parts of the new code make sense, so you don't have the generator overwrite something you spent time customizing.

Good idea.  Could use a "file compare and merge" tool to do the synthesis.

Another thing that's emerging from this discussion is the interaction between a platform's generator abilities and the optimal design of the apps that you build with it.  There's a lot to be gained if the objects it generates are "good enough" for your purposes, even if they aren't exactly what you would create if starting from a blank slate, because you can the time and effort saved to writing custom code where it really matters or simply to getting done earlier.  (For example, I hear that the generated Django "admin interface" was inspired by the fact that Django's original purpose was to build online newspapers.  The data-entry people were basically the paper's reporters, who could do fine with plain-vanilla entry forms; the designer/developer creativity could be devoted to the read-only pages, to attract and serve the readers/customers.)  So, it's valuable to understand what kinds of automagic your platform can and can't do, and proceed accordingly.

~ Ken