Advice and opinions?

2 views
Skip to first unread message

Ron Phillips

unread,
Aug 8, 2007, 11:39:16 AM8/8/07
to colum...@googlegroups.com
I am working on a Rails app for traffic crashes. They're fairly simple, except that the data is exchanged with ODOT in "Tractape" format.
 
Tractape was developed in the 1970's, when bytes were like $1 each or something. At any rate, it consists of a dozen or so text fields, and maybe 30 or 40 "flag" fields where the user has a booklet to look up what the flag means.
 
For instance, in the SECONDARY_ROAD_CONDITION field, an "01" means "Dry", an "02" means "Wet", an "03" means "Snow", etc. I don't want to make my users look up all those flags; what's a computer for, anyway?
 
I suppose the sledgehammer approach is to make a little table for each flag, then make a model of each little table for feeding "collection_select"s, then updating both the flag and its interpretation in the "crashes" table. Or, should I make a big "flags" table, with "flag_name", "flag_value", and "flag_meaning" fields? Or, some other approach altogether?
 
Any opinions?
 
Ron
 
 
 
 

Adam McCrea

unread,
Aug 8, 2007, 11:46:09 AM8/8/07
to colum...@googlegroups.com
If the flags don't change, you could keep them as hashes within the code, right?  Then you could override your attribute writer methods in your models to replace the flag with the actual value, or maybe hook into the before_save callback.  Not sure if it's the best design, but I'm just throwing out ideas off the top of my head.

- Adam

Ron Phillips

unread,
Aug 8, 2007, 11:54:07 AM8/8/07
to colum...@googlegroups.com
Well, theoretically they don't change often.
 
I like the simplicity of your suggestion. Maybe I'll combine your idea with a db for persistence -- keep a serialized version of the flags in a data table. That way they're not buried in the code, but they're also not making a bunch of joined tables every time the db looks up a crash.
 
Ron

>>> "Adam McCrea" <adam....@gmail.com> 08/08/2007 11:46 AM >>>

Aaron Bedra

unread,
Aug 8, 2007, 1:12:20 PM8/8/07
to colum...@googlegroups.com
If the flags change, you can consider using the active record method_missing approach. whenever it finds a flag that doesn't exist you can remap your flags and well... do anything you want with them. This might be more trouble than it's worth considering the rate of change of the flags, however, it's always fun when you can dig into ruby's metaprogramming capabilities :)

Adam McCrea

unread,
Aug 8, 2007, 1:50:22 PM8/8/07
to colum...@googlegroups.com
I don't like my advice anymore. :-)  Translating the flags before they get persisted in the database means you lose the flag in its raw form forever.  If (when) the flags change, you'll have to question all of the data you've collected.  Instead of overriding the attribute writers, I would actually override the readers, so that the raw flag stored in the DB is hidden by the model.

If the flags don't change very often, and if the programmer (you) is the one responsible for keeping them updated, I'd still maintain the flag mappings in the code rather than in the DB.  I'd put them in their own module somewhere so they don't get buried, but the only reason I see to keep them in the DB is so that a user can maintain them, or if they change so often that you create an administrative interface to reduce the time it takes for you to update them.

Good luck!

- Adam

Matt Williams

unread,
Aug 8, 2007, 2:26:16 PM8/8/07
to colum...@googlegroups.com
Something I have done to insulate people from needing to know anything about the code or the values (and to be able to add new functionality fairly easily) is to add tables for, say, in your case, RoadCondition, then I created code (I'll get to it) which would create class level attributes for me, so that I could say RoadCondition.dry and it would be an instance of the "Dry" record (which was read only once, upon class load) to try to cut down on the reads.  The reason I chose this route was to allow the definition of new variables, as well as not knowing (in my case) ahead of time what the value would be.  YMMV, of course, but it did make some aspects of my code easier.....

In order to use it, you'd do a:

class RoadCondition << ActiveRecord::Base
  class_attr_lookup
end

The class_attr_lookup code is:

class Module
  def class_attr_reader(*symbols)
    symbols.each do |symbol|
      self.class.send(:define_method, symbol) do
        self.class_eval "@@#{symbol}"
      end
    end
  end

  def class_attr_writer(*symbols)
    symbols.each do |symbol|
      self.class.send(:define_method, "#{symbol}=") do |value|
        self.class_eval "@@#{symbol} = value"
      end
    end
  end

  def class_attr_accessor(*symbols)
    class_attr_reader(*symbols)
    class_attr_writer(*symbols)
  end

  def class_attr_lookup
    self.find(:all).each do |rec|
      r = rec.name.downcase.underscore
      class_attr_reader(r)
      self.class_eval "@@#{r} = rec"
    end
  end

end


On 8/8/07, Ron Phillips < RPhi...@engineer.co.summit.oh.us> wrote:



--
I can say to myself and the world, "Look at all I am doing, am I not being busy? Am I not contributing? Am I not having an impact on all those around me and with whom I come into contact? See, my life has meaning."
To which the Tao responds, "You are doing, yes, but you are not being. Slow down, go with the flow, work with life, not against it. By being, you do. By doing, you cease to be."

Ron Phillips

unread,
Aug 8, 2007, 2:37:45 PM8/8/07
to colum...@googlegroups.com
This has turned into a most interesting discussion for me. I have at least 4 Ruby/Rails books out on the desk, with an average of 2-3 bookmarks in each. I didn't realize that there would so many approaches that were better than the "sledgehammer." How many languages would allow so many intriguing techniques? Well, back to my "homework."
 
Thanks, everyone for the responses so far.
 
Ron

>>> "Matt Williams" <otte...@gmail.com> 08/08/2007 2:26 PM >>>

Ron Phillips

unread,
Aug 8, 2007, 2:38:33 PM8/8/07
to colum...@googlegroups.com
Thanks, Aaron
 
 Hey, I'm in it for the fun! Seriously, that would make it more robust, and, especially at first, help catch the flags I miss, too. I am trying a spike test now.
Ron
 
>>> Aaron Bedra <abe...@wcnet.org> 08/08/2007 1:12 PM >>>

Joe OBrien

unread,
Aug 8, 2007, 2:43:00 PM8/8/07
to colum...@googlegroups.com
make sure that if you don't know what to do with it, you throw it back up the chain

def method_missing(method_name, *args)
#inspect method_name and try to work with it
# return what you are trying to return ... OR
super.method_name(*args)
end

Syntax might not be right (just typing out what's in my head at the moment) but the moral of the story is still there. Don't swallow Rails method_missing things. It's a fun hook to play with, but tread very lightly in the Rails world with it. They have so many nuances that can be easily missed.


-joe

Aaron Bedra

unread,
Aug 8, 2007, 2:44:53 PM8/8/07
to colum...@googlegroups.com
I am jumping on the IRC channel if anyone else wants to join and discuss this
On Aug 8, 2007, at 2:38 PM, Ron Phillips wrote:

jherber

unread,
Aug 9, 2007, 8:15:50 AM8/9/07
to Columbus Ruby Brigade
ron, if i understand your problem correctly, when presenting to the
user, you want to convert from a flag value to a human readable value
and visa-versa when taking input from a user and storing the
solution. so, i see "Flag" class with two simple functions that
encode and decode, backed by an easy to edit yaml table (in case the
interpretation of a flag needs to be modified). thus, when you render
the model you call "decode" and when you accept input, you call
"encode".

now, if you have families of flags, you should probably have a class
for each type all wrapped in one module (e.g.
"UserFlags::SecondaryRoadConditions.decode(x)" ) . imo, yes meta-
programming is cool, but it should not be considered unless it is
keeps the intention of the code obvious and is significantly cheaper
(in terms of time to implement and support).

best,

jim herber

Ron Phillips

unread,
Aug 9, 2007, 1:02:24 PM8/9/07
to Columbus Ruby Brigade
Thanks, everyone. I kind of ended up using some of most of your suggestions, sort of.
 
I hoped to be able to call Crash.facade_occurrence, for example, and get back the encoded value like "On Roadway" instead of "1", which is returned for Crash.occurrence. I think it will be handy to be able to call Crash.facade_occurrence = "On Shoulder" instead of Crash.occurrence = '2', so I tried to implement that, as well.
 
Here's Flagger, whose only job is to encode flag & key pairs, or decode flag & value pairs:
##########
class Flagger
    FLAGS = {"occurrence"=>{
  '1' => 'On Roadway',
  '2'=>'On Shoulder',
  '3'=>'In Median',
  '4'=>'On Roadside',
  '5'=>'On Gore',
  '6' =>'Outside Trafficway',
  '7' =>'Unknown'}
  }
 
  def encode(flag, key)
    FLAGS[flag][key]
  end
  def decode(flag, value)
    FLAGS[flag].index(value)
  end
  def flaglist
    FLAGS
  end
end
##################
That was simple.
 
Here's the Crash class with the method_missing overwritten.
###############
class Crash<ActiveRecord::Base
  FACADE_RE = /(facade_)(\w+)(=?)/
  FLAGGER = Flagger.new
  def method_missing(m, *args, &block)
    facade = FACADE_RE.match(m.to_s)
    if facade
      message = facade[2]
      suffix = facade[3]
      #first, handle facade requests where a facade exists
      if FLAGGER.flaglist.has_key? message
        args[0]=FLAGGER.decode(message, args[0]) unless args[0].nil?       
        suffix.nil? ? result=super(message) : result=super(message+suffix, args)
        #return the (possibly new) facade value
        FLAGGER.encode(message,result)
      else
        #if there's no facade available, just return the value
        super(message)
      end
    else
      #it's not a facade request, just pass it along
      super(m, *args, &block)
    end
  end
end
##############
Not simple, for me, but not that bad, either. It seems to work just fine.
 
c = Crash.find(:first)
puts c.docno
#~ 2005400453
puts c.facade_docno
#~ 2005400453
puts c.occurrence
#~ 1
puts c.facade_occurrence
#~ On Roadway
puts c.facade_occurrence = 'On Gore'
#~ On Gore
puts c.occurrence
#~ 5
puts c.occurrence = 1
#~ 1
 
Thanks, again, everyone. Let me know if you see anything out of line. Now to see what the controller and view look like!
 
Ron

>>> jherber <jimh...@gmail.com> 08/09/2007 8:15 AM >>>

Ken Barker

unread,
Aug 9, 2007, 1:11:31 PM8/9/07
to colum...@googlegroups.com
Anyone else hoping that 'On Gore' means something different than expected?   ;-)
--
Ken Barker
EdgeCase
http://theEdgeCase.com

Ron Phillips

unread,
Aug 9, 2007, 1:25:01 PM8/9/07
to colum...@googlegroups.com
Ok, I checked with my traffic buddies. "Gore" has several meanings, as you point out. It's 'bloody' awkward, no 'bull', but in traffic talk, the "gore" of a road is like a person's "taint," except it's a road -- tain't the expressway, tain't the exit. That triangle of grass is the 'gore' of the exit. Who knew?
 
Ron

>>> "Ken Barker" <ken.b...@gmail.com> 08/09/2007 1:11 PM >>>

Anyone else hoping that 'On Gore' means something different than expected?   ;-)

Adam McCrea

unread,
Aug 9, 2007, 1:37:37 PM8/9/07
to colum...@googlegroups.com
Pretty slick!  My only concern is that your usage of the words "encode" and "decode" is reversed from what I would expect.  "On Roadway", for instance, is a *decoded* value to me.  It's such a small thing, but it actually threw me a bit while I was digesting the code.  Am I alone on that?


On 8/9/07, Ron Phillips <RPhi...@engineer.co.summit.oh.us> wrote:

Ron Phillips

unread,
Aug 10, 2007, 7:34:00 AM8/10/07
to colum...@googlegroups.com
I think I agree with you. From the standpoint of the database, perhaps, "On Roadway" might be considered a code for "1", but from the human standpoint, "1" is a code for "On Roadway".
 
I suppose the field names reflect this, because most of the "flag" fields are named something like "hit_skip_flag", but it isn't consistent. I  probably should change decode/encode, though; (hu)man is the measure of all things, right?
 
Ron

>>> "Adam McCrea" <adam....@gmail.com> 08/09/2007 1:37 PM >>>

Lari Kirby

unread,
Aug 10, 2007, 8:02:26 AM8/10/07
to colum...@googlegroups.com
A triangular bit of extra stuff - same meaning in sewing: a skirt with gores.



Lari Kirby
--
Footsteps in the sands of time are not made by sitting down.
---
The greatest glory does not exist in never falling, but in rising every time we fall.
---
Mathematicians are lazy: they prove something once and then take it as given from then on. - Hal Hanes, Math Professor (paraphrased)
---
"Why program by hand in five days what you can spend five years of your life automating?" - Terence Parr, creator of ANTLR

----- Original message -----
From: "Ron Phillips" <RPhi...@engineer.co.summit.oh.us>
To: colum...@googlegroups.com
Date: Thu, 09 Aug 2007 13:25:01 -0400
Subject: [CRB] Advice and opinions?

Ok, I checked with my traffic buddies. "Gore" has several meanings, as you point out. It's 'bloody' awkward, no 'bull', but in traffic talk, the "gore" of a road is like a person's "taint," except it's a road -- tain't the expressway, tain't the exit. That triangle of grass is the 'gore' of the exit. Who knew?

Ron

>>> "Ken Barker" <ken.b...@gmail.com> 08/09/2007 1:11 PM >>>
Anyone else hoping that 'On Gore' means something different than expected? ;-)

Mike Sandman

unread,
Aug 11, 2007, 4:07:12 PM8/11/07
to colum...@googlegroups.com
howdy all,

nice discussion. first let me offer a link to some reference material.

http://www.dps.state.oh.us/CrashRequests/Common/CrashTTDoc.pdf

i personally find the two most fun things i do is acquiring new tools
and choosing the perfect tool for the job. it is the mark of a
craftsman. of course, you can always use a general purpose tool, if
that's all you have. in my wood shop, when i am cutting veneer i reach
for my veneer saw, even though the table saw or band saw would also
work, just not as well.

as i read all of the posts, i am reminded of the old saying, "when all
you have is a hammer, everything looks like a nail." so let me offer a
(likely unpopular...lol) opinion suggesting you put the ruby
sledge-hammer back in the tools box and take this opportunity to learn a
new specialty tool. and don't even think about reaching for the
swiss-army chainsaw a.k.a perl :) as an old timer in coding terms (25+
years) i have quite a large tools box--not bragging, i'm just old ;)
and, i encourage you to enlarge yours.

anyway, back to your problem. your are parsing data from lines of texts.
i doubt you will be getting little snippets of text hundreds of times
per day. you will likely want to bulk process the in-coming text data
in a single, simple, quick process. this begs to be processed as a
stream, of course. there are many tools to do this and many approaches
to building your parser. however, i think the key factor here is that
you will likely write this script, and then use it for long periods of
time. time enough to "forget" exactly how it works--perhaps you might
not even be the next person to modify it.

so, you need a tool that adds very little cruft around the job of
parsing and "decoding" your field data. in fact, the perfect tool will
help you to see the rules and definitions of of parsing (your grammar)
at-a-glance. this is where the general-purpose languages fall down.
that said, i will not tell you which tool you should pick, that's part
of the fun. but i will leave you with this small example to amplify my
point.

on page 6 of the PDF above, column 3 gives you the field lengths. using
that information, the following awk script will parse a tractape file in
to a tab separated values file. note i only used the first 10 fields
for example brevity i.e. laziness ;)

BEGIN { OFS = "\t"
FIELDWIDTHS = "11 10 5 2 2 1 1 2 5 1" }
{ print $1, $2, $3, $4, $5, $6, $7, $8, $9, $10 }

i doubt even Ruby can be more succinct then that! almost anyone with
minimal programming experience can quickly grasp that this 3 line script
will (on line 1) use tabs to separate values and (on line 2) split
fields at fixed intervals of 11 10 5 etc. obviously, the print
statement (on line 3) could be turned into a loop rather than explicitly
printing each field. though, i kind of like fact that you get a quick
field count from the print statement. moreover, if you wanted to wrap
some extra text around certain fields, it would be trivial even for a
first year programming student.

my point? that, using the right tool for the job makes the job
wonderfully simple and fun. okay, okay, this is a Ruby group, so why
not use Ruby...and i am all for it. just take this little script and
pop it into a system exec statement e.g.

system("cat myfile | awk 'BEGIN {...}'")

and viola!

my two cents...

-sandman

Ron Phillips

unread,
Aug 13, 2007, 7:23:47 AM8/13/07
to colum...@googlegroups.com
Ummm, well, that wasn't really the intent of my question, although I do have a use for your answer!
 
My original post concerned 'rendering' or 'masking' or 'encoding' a bunch of flag fields with the human-readable values, AFTER I'd pulled the data out of the Tractape format and into a db. Your suggestion, though, is a nice, slim answer to how to pull the data out of Tractape and work with it.
 
 I still think I want to pull it into a SQL database table (or, actually, three) eventually. The engineers here are always coming up with new ways to slice and dice the data -- perfect for a db.
 
However, the data does come in by little dribs and drabs, as you said. The awk script has the advantages you mention, and it might be just the ticket for validating the data as it comes in. I could red-flag things like: a head-on collision between two south-bound vehicles, or a daylight crash at 2 am, or any of the other myriad ways that I've seen that the contents of the fields are suspicious at best.
 
Thanks for the suggestion -- it's a nice, clean approach to sucking the data out of a text file, and when the time comes, I'll surely be giving it another look.
 
Ron

>>> Mike Sandman <mi...@thesandmans.com> 08/11/2007 4:07 PM >>>

Ron Phillips

unread,
Aug 16, 2007, 8:04:51 AM8/16/07
to colum...@googlegroups.com
Just to finish off the thread: I ended up using the method_missing to "facade" fields with their "human" readable equivalents, as described previously. The model also includes a constant for each of the flags, which is a hash of the code=>text pairs.
 
The final, or latest, geo_crash.rb:
////////////////////////////
class GeoCrash < ActiveRecord::Base
  File.open('lib/flags.yml') do |file|
    FLAGLIST = YAML::load(file)
  end
  FACADE_RE = /(facade_)(\w+)(=?)/
 
  #I want any message that starts with facade_ to return the f_encoded value
  #or to set the f_decoded key. So, facade_occurrence should return 'On Road'
  #and facade_occurrence='On Gore' should set occurrence = '5'.

  def method_missing(m, *args, &block)
    facade = FACADE_RE.match(m.to_s)
    if facade
      message = facade[2]
      suffix = facade[3]
      #first, handle facade requests where a facade exists
      if FLAGLIST.has_key? message
        args[0]=f_encode(message, args[0]) unless args[0].nil?       
        suffix.nil? ? result=super(message) : result=super(message+suffix, args)
        #return the (possibly new) facade value
        f_decode(message,result)

      else
        #if there's no facade available, just return the value
        super(message)
      end
    else
      #it's not a facade request, just pass it along
      super(m, *args, &block)
    end
  end
  private
  def f_decode(flag, key)
    FLAGLIST[flag][key]
  end
  def f_encode(flag, value)
    FLAGLIST[flag].index(value)
  end
 
end
/////////////////////////////////////////
 
I added to my helpers:
/////////////////////////////////////////
  def facade(obj,message)
     "#{obj.send(message)} - #{obj.send('facade_'+message)}"
  end
//////////////////////////////////////////
So in my rthml pages,
 
<%= facade @geo_crash, 'collision_type' %>
 
shows both the coded and decoded value of the field, like "4 - Head On"
 
 
Also the helper:
////////////////////////////////////////
  def cs_current(object,name, method)
    result = "<select name='#{name}[#{method}]'>\n"
    collection = object.class::FLAGLIST[method]
    collection.keys.sort.each do |key|
      if object.send(method) == key
        result << "<option value='#{key}' selected='selected'>#{collection[key]}</option>\n"
      else
        result << "<option value='#{key}'>#{collection[key]}</option>\n"
      end
    end
    result << "</select>\n"
    return result
  end
//////////////////////////////////////
so in _form.rhtml,
 
<%= cs_current(@geo_crash,'geo_crash','crash_severity') %>
 
results in:
 
<select name='geo_crash[road_contour]'>
<option value='1'>Straight Level</option>
<option value='2' selected='selected'>Straight Grade</option>
<option value='3'>Curve Level</option>
<option value='4'>Curve Grade</option>
<option value='5'>Unknown</option>
</select>
 
which is pretty easy.
Thanks again, everyone.
Ron
Reply all
Reply to author
Forward
0 new messages