Worst bug ever

[David Miguel Antunes]

Hi,

This summer:

Worst Bug Ever Part II - The Scala Version

object listeners {
    val fileEdited = collection.mutable.Set[() => Unit]()
}

A candy for the first person to find it.

Cheers,

David

[Vlad Patryshev]

Well, how do you actually compare two functions for equality? If you have a val f:(()=>Unit) = () => {println("i'm f")}, this f maintains its identity; but if you fileEdited.add(() => {println("i'm f")}), it's a different story.

Thanks,
-Vlad

--
You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Stefan Hoeck]

Also, the order in which listeners are called is not the same as the order in which they were added - it's quite arbitrary.

[Nils Kilden-Pedersen]

The bug here, is trying to put functions (side effecting to boot) into a Set. What could possibly go wrong?

--

[Bardur Arantsson]

On 05/15/2013 09:08 PM, Nils Kilden-Pedersen wrote:
> The bug here, is trying to put functions (side effecting to boot) into a
> Set. What could possibly go wrong?
>

Why are functions members of the Eq typeclass? ;)

/ducks

[Bardur Arantsson]

(Or, "Ord", even)

[Roland Kuhn]

No, the most glaring bug is to put mutable data into an “object”. Problems with Set vs. Function or even the concurrency evergreen “mutable val vs. immutable var” follow at a large distance …

Dr. Roland Kuhn
Akka Tech Lead
Typesafe – Empowering professional developers to build amazing apps.
twitter: @rolandkuhn

See you at Scala Days 2013 in NYC!
June 10th - June 12th
www.scaladays.org

[Rex Kerr]

I'm not sure this counts as the "worst bug ever".  Most of the time this works the way you want.

You only get into trouble when you use partial application to generate your function that you're trying to later remove (or will spuriously add multiple times):

  def badIdea() { println("fish") }

  listeners.fileEdited += badIdea

  listeners.fileEdited += badIdea   // We now have two, oops

  listeners.fileEdited -= badIdea  // Still have two....

If you don't add it more times than you need it, and you never subtract it, or if you do add multiple times and/or subtract it's from a function you've already captured:

  val f = () => println("bird")

everything will be peachy.

Well, unless you expect some particular order of callbacks.  But why would you do that?

  --Rex

--

[Rex Kerr]

I just assumed it was an inner object.

scala> class Q { object mut { val x = collection.mutable.Set("salmon","cod") } }
defined class Q

scala> val q = new Q
q: Q = Q@4d1a802

scala> q.mut.x += "bass"
res56: q.mut.x.type = Set(cod, bass, salmon)

scala> q.mut.x += "herring"
res57: q.mut.x.type = Set(cod, herring, bass, salmon)

scala> q.mut.x -= "salmon"
res58: q.mut.x.type = Set(cod, herring, bass)

scala> (new Q).mut.x += "perch"
res59: Q#mut.x.type = Set(perch, cod, salmon)

All looks good to me.

It's not like it was a runnable sample.  As posted, there's no issue at all--you create an object with an empty data structure which is never even initialized :).

If it's global, sure, we're back to the classic global-variables-are-bad thing.

  --Rex

[Chris Bissell]

If the implementations of the functions stored in the Set are closures, and the holder is a singleton, they can create a lovely memory leak as they hold onto the references to their outer context...

[Roland Kuhn]

[Texan voice, chewing on straw] “shared mutable memory-leaking undefined-order-side-effecting sonofabitch”

Jokes aside: who got the candy? Or did you manage to sneak past all these and were tripped up by something else?

Dr. Roland Kuhn
Akka Tech Lead
Typesafe – Empowering professional developers to build amazing apps.
twitter: @rolandkuhn

[yves]

Well,I hardly see any bug here.

The code compiles and may even do what you want it to. Of course,it all depends on what you actually want it to do.

I do not see much use for this piece of code.

For once, it's a Set, not a Map : you cannot retrieve a function at will, only check it's presence in the Set. That itself doesn't sound very usefull.
Second, it's a Set, not a LinkedHashSet, a List, or whatever collection where order is well defined : so iterating on it seems pointless, unless the order in which you call said functions is irrelevant (it could be, if each does an independant task.)
Third, the code looks unsound because using Sets usually also means having well defined equality and hashcode methods. Here, the code relies on the basic scheme (Object equality/hashcode) ; if your collection of storable functions is well known, this is not a problem,but it's unlikely not the case. It seems to me that in order to prevent any bad use of such a Set (for exemple toprevent you from accidentaly put an unwanted function in your Set), your functions should also extend something like Comparable. This would prevent easy mistakes such as putting a compiler generated function in your Set (closures or the like).

Little more can be said unless the environment where this is supposed to be used is disclosed.

At least,we might want to know what kind of problems you met.

Yves

[David Miguel Antunes]

Well, yes it is possible that there is no bug.

I was having very strange random behavior in the application, and it seemed that the listeners weren't always being called.

I though it may have to be with the functions changing their hash on some situations, in which case it would be similar to the bug of having objects in sets or hashmaps and changing their hashCode value:

http://stackoverflow.com/questions/5110376/hashset-contains-problem-with-custom-objects

When I changed to a ListBuffer, all seemed to be ok.

Probably I was inserting new listeners into the set, while iterating with foreach. I don't know what is the expected behavior in this situation (I think in Java it crashes), but I suppose the iteration would go wrong silently.

With a ListBuffer, even if I appended new listeners, the iteration of the existing ones probably continues as expected, this is probably why the ListBuffer seemed to fix it.

(Right now I create a copy and iterate on the copy)

P.S.:

No bug, but still: candies for everyone :)

Best,

David

[Som Snytt]

> Well, yes it is possible that there is no bug.

I agree that that truly is the worst bug of all.

[yves]

The function will not change their hash, so that is not the issue.

Inserting a new function while iterating is inherently unsafe ; the iteration order is undefined,so you don't know if the inserted function will or will not be called when the iteration goes on. Likely some will and some will not ; and the behaviour may change from run to run.
You would have to create a custom function with well defined hashcodes/equality to ensure that things are at least predictable.

By contrast, the ListBuffer is likely more predictable.However, the scala spec doesn't specify (or I have failed to see it) how iterators (and foreach) behave when inserting new elements while iterating.

Yves

[Oliver Ruebenacker]

     Hello,

On Wed, May 15, 2013 at 2:54 PM, Stefan Hoeck <efasc...@gmail.com> wrote:

Also, the order in which listeners are called is not the same as the order in which they were added - it's quite arbitrary.

  I think that should not be a problem, because you should not rely on the order in which listeners are called.

     Take care
     Oliver

--
Head of Systems Biology Task Force at PanGenX (http://www.pangenx.com)
Any sufficiently advanced technology is indistinguishable from magic.

["Ionuț G. Stan"]

On 5/17/13 2:09 AM, David Miguel Antunes wrote:
> Well, yes it is possible that there is no bug.
>
> I was having very strange random behavior in the application, and it
> seemed that the listeners weren't always being called.
>
> I though it may have to be with the functions changing their hash on
> some situations, in which case it would be similar to the bug of having
> objects in sets or hashmaps and changing their hashCode value:
> http://stackoverflow.com/questions/5110376/hashset-contains-problem-with-custom-objects
>
> When I changed to a ListBuffer, all seemed to be ok.
>
> Probably I was inserting new listeners into the set, while iterating
> with foreach. I don't know what is the expected behavior in this
> situation (I think in Java it crashes), but I suppose the iteration
> would go wrong silently.

Depends what you're using in Java. With a synchronizedSet it will throw
a ConcurrentModificationException if you mutate the collection in the
midst of the iteration. Even in a single-threaded program. With other
types of collections, you'll probably get to iterate over the newly
added elements as well.

In general it's a pretty bad idea to mutate the collection you're
iterating over. Some would go as far as saying that's a pretty bad idea
to mutate :)

Not sure what's different in the implementations of mutable.Set and
ListBuffer in Scala. The hierarchy is too large to dig in right now.

> With a ListBuffer, even if I appended new listeners, the iteration of
> the existing ones probably continues as expected, this is probably why
> the ListBuffer seemed to fix it.
> (Right now I create a copy and iterate on the copy)
>
> P.S.:
> No bug, but still: candies for everyone :)

> Inline images 1
>
> Best,
> David
>
>
> On 16 May 2013 21:05, yves <yves....@gmail.com

> <mailto:scala-user%2Bunsu...@googlegroups.com>.

> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "scala-user" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to scala-user+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

--
Ionuț G. Stan | http://igstan.ro

[Oliver Ruebenacker]

     Hello,

On Fri, May 17, 2013 at 10:42 AM, "Ionuț G. Stan" <ionut....@gmail.com> wrote:

On 5/17/13 2:09 AM, David Miguel Antunes wrote:

Probably I was inserting new listeners into the set, while iterating
with foreach. I don't know what is the expected behavior in this
situation (I think in Java it crashes), but I suppose the iteration
would go wrong silently.

Depends what you're using in Java. With a synchronizedSet it will throw a ConcurrentModificationException if you mutate the collection in the midst of the iteration. Even in a single-threaded program. With other types of collections, you'll probably get to iterate over the newly added elements as well.

  I used to have ConcurrentModificationExceptions in Java with regular HashSets.

     Take care
     Oliver

["Ionuț G. Stan"]

On 5/17/13 5:47 PM, Oliver Ruebenacker wrote:
>
> Hello,
>
> On Fri, May 17, 2013 at 10:42 AM, "Ionuț G. Stan"
> <ionut....@gmail.com <mailto:ionut....@gmail.com>> wrote:
>
> On 5/17/13 2:09 AM, David Miguel Antunes wrote:
>
>
> Probably I was inserting new listeners into the set, while iterating
> with foreach. I don't know what is the expected behavior in this
> situation (I think in Java it crashes), but I suppose the iteration
> would go wrong silently.
>
>
> Depends what you're using in Java. With a synchronizedSet it will
> throw a ConcurrentModificationException if you mutate the collection
> in the midst of the iteration. Even in a single-threaded program.
> With other types of collections, you'll probably get to iterate over
> the newly added elements as well.
>
>
> I used to have ConcurrentModificationExceptions in Java with regular
> HashSets.

Ah, yes you're right.

[Alexandru Nedelcu]

On Fri, May 17, 2013 at 5:42 PM, "Ionuț G. Stan" <ionut....@gmail.com> wrote:

In general it's a pretty bad idea to mutate the collection you're iterating over. Some would go as far as saying that's a pretty bad idea to mutate :)

In cases where I need to mutate, I find it far easier to use an immutable data-structure in combination with an AtomicReference.

--
Alexandru Nedelcu
http://bionicspirit.com

[Stefan Hoeck]

Sure, it might be bad style to rely on the order in which listeners are called. But then the whole concept of registering and updating listeners is an imperative concept and in imperative programming, order of evaluation typically matters. After all, a listener might update other state which results in other listeners being called some of which might depend on some other state already being updated and oops! all hell breaks loose.

The bottom line: Scrap listeners, use reactive programming. :-)

Cheers, Stefan

[Raoul Duke]

> The bottom line: Scrap listeners, use reactive programming. :-)

1) how do you specify such ordered inter-relations in reactive programming?
2) is there a reactive programming library that actually has worked
out all the update glitch bugs?
(seriously, not trying to be huffy or snarky.)

[Oliver Ruebenacker]

     Hello,

  I thought using listeners is a form of reactive programming. ;)

  Seriously, what is the solution? Being so restrictive that circular data flow is impossible? Or something clever to sort it out?

     Take care
     Oliver

--

You received this message because you are subscribed to the Google Groups "scala-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scala-user+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Bardur Arantsson]

On 05/17/2013 11:17 PM, Oliver Ruebenacker wrote:
> Hello,
>
> I thought using listeners is a form of reactive programming. ;)

Nah.

>
> Seriously, what is the solution? Being so restrictive that circular data
> flow is impossible? Or something clever to sort it out?
>

You weren't talking about circular data flow. (Which would be...
interesting unless you have a function with a fixpoint.)

You were talking non-determinism, as far as I can tell.

Reactive programming makes data flow explicit and declarative, i.e. the
opposite of non-determinisim.

[Haoyi Li]

I don't think it's possible to work out "all" the glitch bugs; it's simply a tradeoff on how much glitching you can tolerate v.s. how much rigidity you can tolerate. If you're happy to statically fix the dataflow graph once and for all, you can topo-sort the whole thing and have no glitches, but programming that way is no fun. The more flexibility in modifying/etc. your dataflow graph dynamically, the more glitching you're gonna get. If you want parallelism, then that's more glitching, no way around it =D.

Not sure what you mean by ordered inter-relations.

source: I wrote Scala.Rx

[Stefan Hoeck]

I should have been more specific: Functional reactive programming (though I am well aware that it can have its own shortcomings; and of course it can be implemented using listeners in the background).

Concerning circular data flows, this is how I do it in dire:

A reactive dependency graph is an isolated entity, hidden from clients. There is an Actor responsible for updating (parts of) the reactive graph whenever another part changes (fires an event). This might lead to new (circular) requests to update the graph which are sent back to the master Actor. So there is a delay before circular calls are processed and they are guaranteed to being processed only when the rest of the graph has been updated. So far this seems to work out well, but then dire is a work in progress so it still has to proof its worth.

Cheers, Stefan

[Raoul Duke]

> So far this seems to work out well, but then dire is a work in
> progress so it still has to proof its worth.

how does that approach compare to other FRPs (in other languages)?
seems to me like people often re-tread this issue.

[Stefan Hoeck]

Well, as I said, FRP can have its own drawbacks (space and time leaks) especially if you allow event switching (= dynamic changes in the reactive graph; these typically come from implementing (or trying to implement) a 'Signal Monad' instead of just an Applicative Functor). If you are interested in this topic (especially if your focus is on referential transparency) you can check out dire which provides a referentially transparent FRP implementation based on scalaz. As I said, it's experimental stuff but I already use it in my own projects and enjoy it so far. There's also a link to a blog where I describe some of the design decisions behind dire in random order.

Cheers, Stefan