Nested? Import Bug
Andy Hutchinson
Happened to be coding a project using a series of nested imports.
Generally, they work just fine, but I found what I think is a legal
piece of code that seems to be failing for some reason. I've boiled
down the bug into a small file that demonstrates the problem:
require 'rubygems'
require 'bud'
module StuffProtocol
state do
interface input, :pipe_in
end
end
module BED
include StuffProtocol
bloom do
stdio <~ pipe_in.inspected
end
end
module RD
include StuffProtocol
import BED => :bed
bloom do
bed.pipe_in <= pipe_in
end
end
module B
include RD
state do
interface input, :long_pipe_in
end
bloom do
pipe_in <= long_pipe_in
end
end
module BUser
import B => :b
bloom do
b.long_pipe_in <= [["Travelin"]]
end
end
class TestNestedImport
include Bud
include BUser
end
tni = TestNestedImport.new
tni.tick
And here is the correspodning error message:
/usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud.rb:855:in
`stratum_fixpoint': exception during Bud evaluation. (Bud::Error)
Exception: #<NameError: undefined local variable or method `bed' for
#<TestNestedImport:0x7f6f6e2570d0>>.
Rule: bed.b__pipe_in <= (b__pipe_in)
from /usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud.rb:186:in
`each_with_index'
from /usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud.rb:833:in `each'
from /usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud.rb:833:in
`each_with_index'
from /usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud.rb:833:in
`stratum_fixpoint'
from /usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud.rb:721:in
`tick_internal'
from /usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud.rb:186:in
`each_with_index'
from /usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud.rb:721:in `each'
from /usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud.rb:721:in
`each_with_index'
from /usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud.rb:721:in
`tick_internal'
from /usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud.rb:337:in `start'
from /usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud.rb:622:in
`schedule_and_wait'
from /usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/
eventmachine.rb:312:in `call'
from /usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/
eventmachine.rb:312:in `schedule'
from /usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/
eventmachine.rb:996:in `call'
from /usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/
eventmachine.rb:996:in `run_deferred_callbacks'
from /usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/
eventmachine.rb:996:in `each'
from /usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/
eventmachine.rb:996:in `run_deferred_callbacks'
from /usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/
eventmachine.rb:256:in `run_machine'
from /usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/
eventmachine.rb:256:in `run'
from /usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud.rb:600:in
`start_reactor'
from /usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud.rb:599:in
`initialize'
from /usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud.rb:599:in `new'
from /usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud.rb:599:in
`start_reactor'
from /usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud.rb:334:in `start'
from /usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud.rb:699:in `tick'
from test_nested_import.rb:56
Seems to be some issue with rewriting tables during the import
process?
Let me know if anyone can reproduce this. I just pulled the git repo
and built bud, btw.
--Andy
Neil Conway
Thanks for the report -- that does look like a bug, and repros for me.
I'll take a look at fixing it.
Neil
Neil Conway
I checked in a fix for this issue. Can you please fetch the latest Bud
sources from GitHub and let me know if the problem is resolved?
Thanks,
Neil
On Tue, Oct 25, 2011 at 1:36 AM, Andy Hutchinson
<andy.is.i...@gmail.com> wrote:
Andy Hutchinson
Awesome! Works like a charm as far as I can tell. I'll let you know
if it crops back up in the project.
Thanks!
--Andy
Andy Hutchinson
Suppose I have this interface:
module Interface
state do
interface input, :pipe_in
end
end
And that I have several different implementations:
module A
include Interface
end
module B
include Interface
end
Now, I want to write third module which imports the interface twice:
module C
import Interface => :one
import Interface => :two
bloom do
one.pipe_in <= [["travelin"]]
end
end
And then I want "flavor" C with different implementations:
module AC
include C
import A => :one
import A => :two
end
module BC
include C
import B => :one
import B => :two
end
module ABC
include C
import A => :one
import B => :two
end
This particular "way" of achieving this was working before the patch
and now it doesn't. I'm open to new ways of doing this, but I would
still like to do something "like" this. Right now it gives an error
because of a duplicate symbol like:
Bud::CompileError: duplicate import symbol one in C
/usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud/rewrite.rb:276:in
`build_import_table'
/usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud/rewrite.rb:275:in `merge!'
/usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud/rewrite.rb:275:in
`build_import_table'
/usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud/rewrite.rb:267:in `each'
/usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud/rewrite.rb:267:in
`build_import_table'
/usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud/rewrite.rb:256:in `initialize'
/usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud.rb:207:in `new'
/usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud.rb:207:in
`rewrite_local_methods'
/usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud.rb:151:in `initialize'
/usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud.rb:150:in `each'
/usr/lib/ruby/gems/1.8/gems/bud-0.0.8/lib/bud.rb:150:in `initialize'
test/tc_ballotbox.rb:197:in `new'
test/tc_ballotbox.rb:197:in `test_timed_max_turnout'
The other method i tried was to write code in C that is depended on
"one" and "two" but not try to import them.
module C
include C
# import Interface => :one
# import Interface => :two
bloom do
one.pipe_in <= [["travelin"]]
end
end
I get a different error now when referring to one.pipe_in in C. Bud
says "pipe_in" does not exist:
Bud::CompileError: Table does not exist: 'pipe_in' in rule block......
What is the right way to do this?
-- Andy
On Thu, Oct 27, 2011 at 12:37 AM, Andy Hutchinson
Neil Conway
On Thu, Oct 27, 2011 at 1:07 AM, Andy Hutchinson
<andy.is.i...@gmail.com> wrote:
> Now, I want to write third module which imports the interface twice:
>
> module C
> import Interface => :one
> import Interface => :two
>
> bloom do
> one.pipe_in <= [["travelin"]]
> end
>
> end
>
> And then I want "flavor" C with different implementations:
>
> module AC
> include C
> import A => :one
> import A => :two
> end
I think the right long-term way to do this is to use module parameters
(i.e., parametric types, generics in Java, or C++ templates). module C
would take two type parameters:
module C<T1, T2>
import T1 => :one
import T2 => :two
end
module AC
import C<A, A> => :c
end
and so on. That would let you do what you'd like to do, right?
Unfortunately, module parameters are not yet implemented. (The current
implementation of the module system is pretty kludgy and adding module
parameters would make it even more complicated -- the module system
will probably need a rewrite before it becomes feasible to add
parameters.)
In the short term, there should be an easy fix: your program is being
rejected by an error check I added, and we can simply remove the
check. Previously, we rejected attempts to import modules A and B
using the same bind name (A => :one, B => :one) but only if both
imports occurred inside the same module; if the imports were spread
across multiple modules that were glued together via include, we
neglected to check for this situation. I'm impressed you were able to
find and exploit this behavior :)
Since the original behavior was inconsistent, we could instead allow
two imports with the same bind name in all circumstances, and
basically merge the namespaces of the two modules. That is:
import X => :foo
import Y => :foo
would allow "foo.whatever" to refer to anything in either X or Y.
Importantly, rules defined in X would be able to interact with state
defined in Y (and vice versa). That violates module encapsulation and
is probably not a great design pattern to encourage -- but it would
probably let you write the program you want to write.
Can you elaborate a bit on the complete program you're trying to
write? i.e., what modules C, AC, BC, and ABC are in the previous
example.
Neil
Andy Hutchinson
pattern we run into is to use a certain kind delivery protocol for
network traffic, and then mix and match them in different modules.
module DeliveryProtocol
# abstract interface to delivery messages and report success of delivery
end
module ReliableDelivery
include DeliveryProtocol
# provides "at least" once delivery before reporting success
end
module BestEffortDelivery
include DeliveryProtocol
# provides "at most" once delivery before reporting success
end
If we don't using two delivery protocols, we can just flavor C as follows:
module C
include DeliveryProtocol
end
module RDC
include C
include ReliableDelivery
end
module BEDC
include C
include ReliableDelivery
end
I see this done fairly often in code in the sandbox. But suppose D is
some module that requires two channels of communication to the outside
word. Perhaps they are both designated for different kinds of traffic
which would otherwise be inconvenient to mix together on one channel.
We can't include both modules, because they will clobber each other's
interface. Using the override syntax,
module D
import DeliveryProtocol => :one
import DeliveryProtocol => :two
# logic for communicating to outside world with two delivery protocols
# don't care about their implementation for correctness
end
Then we flavor the module with different delivery protocols.
module RDD
include D
import ReliableDelivery => :one
import ReliableDelivery => :two
end
module BEDD
include D
import BestEffortDelivery => :one
import BestEffortDelivery => :two
end
module AsymmetricDeliveryD
include D
import BestEffortDelivery => :one
import ReliableDelivery => :two
end
Doing this without the override method would require duplicating all
the code in module D that depends on the delivery protocol into BEDD
and RDD. Might be a ton of duplicated code that we now have to
maintain separately.
I think this is a valuable design pattern, despite the break in
encapsulation. That break already occurs when you mixin modules with
each other with "include" statements. They can clobber commonly named
tables and bloom blocks without you noticing until the bug happens.
By extension, having the same thing potentially happen in an import
namespace is no worse in my mind. One mixes because modules can have
dependencies on one another in the same logical namespace:
module BallotBox
# interface for recording votes and displaying tallies of those votes
end
module StaticMembership
# interface for creating static, unchangeable group
end
module RegisteredBallotBox
include BallotBox
include StaticMembership
# Prevents votes from non-registered voters
# Dependencies:
# Need to check static membership before accepting vote in BallotBox
# Need to register voter in StaticMembership before voting
end
It is possible that something in StaticMembership clobbers BallotBox,
but you see this done quite often despite the risk. How is this
different from:
module RegisteredElection
include StaticMembership
import BallotBox => :box
import StaticMembership => :box
# Code provides interface for running an election
# -- Candidates are registered in top level StaticMembership module
# It also provides registered voting semantics in box namespace
# -- glues together ballotbox and staticmembership
end
Admittedly, RegisteredElection isn't probably the best design choice.
Yet, the concept of a having a separate namespace seems more general
than just importing.
In a high level sense, imports provide two separate functions in my
mind. (1) Create a separate namespace in which we can define tables
and bloom blocks, much like "classes" in Object-Oriented languages
provide a separate namespace in which you can define class and
instance methods/variables. (2) Conveniently copy over an existing
module into that namespace. Currently, you can't use (1) directly
without placing everything in (2). I feel like creating separate
namespaces could be useful in organizing code on its own without
needing to use the notion of importing code to get it.
Perhaps a compromise would be for the compiler to warn you if it
thinks you're about to clobber code? Although, sometimes this is
intended as a way of overriding included, named bloom-blocks with new
special logic. I agree that you lose encapsulation if you do two
imports into the same namespace, but it you could trivially keep
encapsulation by not importing into the same namespace. Ruby modules
by themselves weren't designed to be safely encapsulated, anyway.
What do you think?
--Andy
Neil Conway
<andy.is.i...@gmail.com> wrote:
> module BestEffortDelivery
> include DeliveryProtocol
> # provides "at most" once delivery before reporting success
> end
Nitpick: channels can actually deliver a message more than once (at
least as currently implemented, since we just send them via UDP). We
should probably clarify this behavior in the docs.
> Perhaps a compromise would be for the compiler to warn you if it
> thinks you're about to clobber code? Although, sometimes this is
> intended as a way of overriding included, named bloom-blocks with new
> special logic. I agree that you lose encapsulation if you do two
> imports into the same namespace, but it you could trivially keep
> encapsulation by not importing into the same namespace. Ruby modules
> by themselves weren't designed to be safely encapsulated, anyway.
I can buy that. In the long term, I think we should add module
parameters and then get rid of the include behavior because it breaks
encapsulation and makes the interaction between modules hard to
understand. In the short term I'm inclined to do as you suggest and
remove the error check.
Neil
Neil Conway
> I can buy that. In the long term, I think we should add module
> parameters and then get rid of the include behavior because it breaks
> encapsulation and makes the interaction between modules hard to
> understand. In the short term I'm inclined to do as you suggest and
> remove the error check.
FYI, this should be fixed; sorry for the delay. Please let me know if
you encounter any further problems.
Neil