Rubymotion crashing everywhere in production app

[Greg Fodor]

Hi all, I have an app I've shipped to the App Store, and have been cleaning up and diagnosing crashing issues for the last month or two via crash dumps to Crittercism. Almost all of the crashing issues I have seem to point to one thing: Rubymotion's ARC-like reference counting seems to have a lot of issues. My crash reports all seem to indicate my code terminating due to a access error, either on self or on local or instance variables in my code. It has lead me to think I will not build my future app in Rubymotion, despite the speed up in developer time it has given me. 

I have run into the following scenarios dozens of times after aggregating crash reports:

- If you have a variable that is closed over by a block passed to another method further down, and you reference that variable in the block, some % of the time it ends up being corrupted and causes a crash. This happens with local vars.

- Conversely, if you close over method arguments, they appear to be retained and never released properly, resulting in memory leaks.

- I get lots of crashes when calling "super" from within a viewWillAppear/viewWillDisappear/viewDidAppear/viewDidDisappear method. I also get crashes when calling "super" in other places.

- If I declare an attr_accessor :foo, if I try to access @foo before it is initially set (for example, if @foo), it can crash instead of @foo being set to nil.

- If I save a &block to an instance variable, often times the app crashes when I try to .call the block in another thread or after a callback.

Almost all of these problems seem to be exacerbated when I am dealing with asynchronous code. (UI, network, or otherwise.) Essentially, this feels very much like a situation where memory is being released and dumb luck causes things to work the majority of the time, but in the case of asynchronous callbacks the liklihood of memory being overwritten goes up. So, in the end I see a small % of crashes in my production users, but can't reproduce any of it myself.

Now, I've already submitted a few bug reports for the closure-related memory leaks, and I don't believe that they have been fixed. Before I go through the tedious process of submitting reports for all these other bugs, the vast majority I am unable to reproduce but only have evidence of via crash dumps, I wanted to know if I am missing something. Right now, only a small % of my users experience these crashes, but they are littered *all over* my app, and I really don't think I could be doing anything wrong so systemically, especially because they come in the same 5 or 6 flavors above.

Are there known patterns that can cause extra releases in Rubymotion code? Why would "super" ever cause a crash? Am I one of the only people who has a app in production that is reporting these crashes? I would expect to see a lot of complaints about Rubymotion crashes, but most of the posts in this forum are about exceptions and crashes that they reproduce themselves, not retain/release errors they have seen coming in from crash reports. I worry that the cross section of Rubymotion developers, who have apps in production, who have a non-trivial number of users, who are measuring crash reports are so small that serious memory-related issues may be getting missed in Rubymotion.

Thanks for any insights!

-Greg

[Ben Sheldon]

> Am I one of the only people who has a app in production that is reporting these crashes?

It's not just you. I'm experiencing these memory-related types of crashes (like SIGSEGV and SIGBUS) with about 10-20% of users in production. What's most frustrating is that they're nearly impossible to track down with desymbolicated stacktraces (we're using Crittercism) because there usually nothing in the thread that corresponds to our code.

We're now searching for a crash reporting tool that would allow us to set breadcrumbs or some method for trying to isolate the issues down to particular controllers or classes. If anyone has any suggestions, it would be much appreciated.

Ben

-Greg

--
 
---
You received this message because you are subscribed to the Google Groups "RubyMotion - Ruby for iOS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rubymotion+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Greg Fodor]

Crittercism supports breadcrumbs but it's a paid feature. Here are some tips that have seemed to help:

- When you create a block, you need to be very conscious of the closure you create:

  - Never access local variables in a closure, they can crash.

  - If you close over method arguments, ensure you set them to nil otherwise they will leak.

- Always initialize instance variables in loadView or initialize, even if you declared an attr_accessor. Trying to access instance variables before setting them can cause a crash.

- No tips for the "super" issue. I've found "super" crashes happen more frequently in viewDidDisappear and viewWillAppear than viewWillDisappear and viewDidAppear, likely because the controller is more likely to be retained multiple times at those points, I guess?

- If you store a Proc into a instance variable for use later, manually retain and release it after you .call it, and set it to nil afterwards.

That said there are still many places where crashing is happening that I can't narrow down to these cases. As you mention, the crappy stack traces I get don't help.

[Greg Fodor]

Also, I should note that I have played around my app with Zombies turned on, and obviously this hasn't helped find any crash issues. So odds are the problem goes beyond simple retain/release counts and something more internal to Rubymotion, which is above my pay grade.

[Daniel Dickison]

The issue with unretained closed-over local variables is almost certainly due to http://hipbyte.myjetbrains.com/youtrack/issue/RM-3

The recent comment by Laurent suggests they are actively working on a fix now.  I'm keeping my fingers crossed, because this one is by far the biggest productivity killer with RubyMotion for me.

There are workarounds but they are quite tedious.  I've been using NSOperationQueue a lot, with either custom NSOperation subclasses (explicitly create ivars for values you would have closed over in a block implementation), or using NSInvocationOperation (passing arguments which are retained by the operation).

[Greg Fodor]

Ah excellent. This sounds like it may capture the instance variable and super crash cases as well, if self can get GCed. Here's hoping they fix this soon and my users can stop e-mailing me that the app keeps crashing :(

[aceofspades]

Wow, I thought it was bad to have to deal with this constantly in development. It must be embarrassing and overwhelming to deal with this in a production app. 

[Greg Fodor]

Yeah I am pretty surprised that this bug has stayed open for so long. It seems like a "drop everything and fix this" issue because anyone releasing a production app is going to have a ton of crashes since they are not "visible" during development due to probabilities. Why spend time adding OS X support when basic closures don't even work properly on iOS?

[Greg Fodor]

Also, this has so far only managed to get me a single one star review, but I am very nervous to increase publicity or exposure of the app until this is fixed. Also, I am throwing money away since now some % of users I drive to the app abandon it due to the crashes (it's hard to measure how many.)

[Matt Green]

These posts make me really sad to read.

I raised the question about RM-3 on the recent Motion Meetup and Laurent/Watson both responded (Laurent on camera, Watson in IRC). Watson mentioned that RM-3 is the toughest bug to fix, and Laurent discussed how he tried a few approaches but was never happy with them. Both devs are smart and strong coders, so I take them at their word.

Nevertheless, this is a serious bug. And I say that because I've spent a little bit of time thinking about how I would solve it if I knew what to do, and never arrived at a satisfactory solution (short of bundling a GC). For a long while, I believed blocks could simply be something handled specifically by the compiler, namely the contents of a block could be statically analyzed to determine if the block references variables outside of its scope. For all of those variables, I reasoned, the compiler could simply retain each of them upon block creation, and then release each of them upon block destruction. This would tie the lifetime of the variables to that of the block (not the 'complete' lifetime in some cases, of course).

One problem: instance_eval. The contents of the block may or may not be used in a way you can expect ahead of time. It might be possible to throw more static analysis at this and arrive at the correct conclusion, but this is decidedly non-trivial. (And now I want to try to detect it, but I have a bunch of projects already. If you're interested deep-diving this stuff, let me know.) Another problem is the fact that there's a high likelihood of self being retained due to how blocks are often used, which can lead to retain cycles. Retain cycles are like the opposite of this problem, and decidedly less deadly, but still undesirable. There may be some merit to this idea still, where the compiler guesses the context of the block ahead of time, but only makes those retains/releases if the block actually ends up being evaluated in that context.

However, I'm not a compiler-writer, just someone who'd like to be. I'm not speaking for RM, and I don't want to show them up. They probably know way more about the problem than I do.

That's why my current line of thinking is that a proper fix for RM-3 isn't a 100% solution. I'd rather see some libraries that make it harder to screw this sort of thing up. Or, at the very least, a list of anti-patterns that people can check their code against. Because this isn't quite the same as conventional Ruby in this respect. (And that's OK.) I'm not sure what a library solution would look like, though I've done work in the area (Elevate's start/finish callbacks suffer from the same problems as everyone else's). It might mean ditching the Ruby way in favor of something a little safer.

Matt

[iwar...@gmail.com]

This class of bug has bitten me several times, and I am not sure what the proper way around it is (yes instance variables, I know, but that doesn't always help).

Joe Noon has a novel approach here: https://github.com/joenoon/rm-extensions#context that is worth you checking out as a way to get around this problem temporarily.

I am with you, Matt: I would forego 100% ruby compliance for this case if it meant we had a solution (and it was documented!!!) to this bug. 

ian

[Rob Heittman]

I agree in every particular with what Matt said, and unless a solid and fully automatic solution is on the horizon really soon, I think the most important thing to do is to develop a solid analysis of exactly what code patterns are susceptible and a list of known good workarounds ... as opposed to where we are now, with some suspects and suggestions.

Without access to RM source and without more knowledge of Obj-C than I have, I couldn't help *fix* RM-3, but I sure could make tools to hunt known crash-producing antipatterns and make libraries or write different code that doesn't employ them. I'll be sure to check out Joe Noon's lib (thanks Ian).

At this point I have several decent-size RM projects in development, as I'm sure many RM customers do, and frankly they just cannot ship with known crash heisenbugs. I can certainly accept schedule slips to code things in a less than optimal way. I can accept runtime performance penalties, if there's a reference counting fix that works but is slow (maybe it could live behind a compiler switch). But I definitely need to do something, as simply marching on to ship known buggy apps is not a choice.

One other thought -- about Greg's OP -- I have not seen reference counting heisenbugs with "super" in development or production myself. I did have one app that I consulted on that was crashing often in those view[Will/Did][Appear/Disappear] super calls, but that turned out to be reproducible behavior and not reference counting related. I doubt the particular sin in that case (badly managing resources and drawing the same view lots of times) would apply to anyone else's properly written code. Is anyone else seeing the problem with super?

- Rob

[Matt Green]

In certain cases, you can workaround RM-3:

* sometimes, ivars do what you want. Not always.

* if you *know* the block will be run once, then you can exploit this fact by playing games with retain/release (https://github.com/mattgreen/elevate/blob/2300d2189c9b4d12663a553cb56aebdfa1326103/lib/elevate/operation.rb#L85). If there's any chance the block won't run, or will be run more than once, this won't work.

* you can generalize the last point to: if you know it will be run N times, you can issue N pairs of retain/release in a similar manner.

But, really, these are all infuriating things to learn the hard way.

[Rob Heittman]

It's ok if using an ivar is just a pain or unsuitable, but if there are some cases where they can go away unexpectedly too, those cases would be good to understand (or fix ahead of a full solution).

If data collection in the wild could be part of the solution, I manage a collection platform that I could volunteer to receive and digest the data ... but I have no idea what would be useful app side instrumentation to create. Some kind of RM-aware auto-breadcrumb reporter that an affected user could trivially drop into their next update? Just spitballing.

[aceofspades]

We cannot simply say there is a simple workaround for the issue based on a reduced example. The manifestations of this appear to go far beyond a simple test case, and without a fix (non-performant if necessary) we are unable to see its extent or whether each failure is in fact the same issue or another. It is not reasonable to wait months for each fix, only then to uncover the next issue.

I see a clear lack of a sense of urgency over this and many other bugs. Perhaps the overly positive attitude about the community, or the quantity of simpler apps that are able to ship anyway have enabled it. I don't expect solutions are easy, but when new platform releases come out, that clearly indicates where priorities are. Quite misplaced. Serious bugs are not unexpected in a young product, but a laid-back approach to dealing with them is very costly to and an insult to developers. Engineering time is expensive, and time to market is critical. These are emergencies.

[Marcus]

This particular issue is VERY frustrating, and I've managed to work around it using instance variable most of the time. My apps still get some random crashes which I believe are related to this problem. Having blocks not function perfectly seems like a very alarming issue that I would hope would be at the top of the list for the RM team.

[Rúben Fonseca]

Because of this and similar memory related bugs, I didn't renew my RubyMotion license this year. I realized pretty quickly that this would affect my production applications very badly, causing unexpected performance issues and obscure crashes.

Wrote to the mailing list a couple of times, created bug report with very simple reproducible examples, but releasing for OSX seems to have higher priority. At one time I even got @lrz himself writing that RubyMotion is not ready for anything professional or serious, only for basic apps. w00t?

I really hope that this problems are sorted out. I know @lrz has the ability to solve it, but probably not in the timeframe we all want.

I know that one day RubyMotion will be free of these bugs, and I'll haply buy a license again :)

Have fun,

Ruben

--

 
---
You received this message because you are subscribed to the Google Groups "RubyMotion - Ruby for iOS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rubymotion+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Will work for bandwidth

[Matt Green]

That's why I think we:

1. Make a fuss about this until things get fixed

2. Abandon the dogmatic adherence to the 'Ruby way', because it's clearly not working for us.

Upshot: if this is biting you, then ditch block-based clients until this is done if you are shipping. You're responsible to your customers, not to DSLs. Implementing a delegate is not the worst thing in the world. I would like to see more docs (especially on BW) to deal with this well; people shouldn't have to come around asking how to cancel a request once it's in-flight.

[Rob Heittman]

I totally understand the level of piling-on here. It is a really big deal, and I hope anyone from HipByte reading this thread gets the level of indignation in the community. I agree releasing for new platforms with fundamental bugs outstanding was a terrible move.

But I would still like to focus on what we can do to help the platform and each other. HipByte is a small team and probably is getting pulled in many directions by various priorities. RM-3 is one thing, which HipByte is clearly aware of and working on, and hopefully understands the urgency of. As I said before, as an outsider, I certainly can't fix RM-3 myself, but if there's anything I can do to help test, workaround, document, or monitor it, I would be happy to do so. That's all I got to give.

Beyond RM-3, I am *much more* concerned by general sense on the thread by people I respect (Ruben, I still use your Keychain Ti module...) that there are just a ton of memory related bugs both documented and undocumented out there. It's scary and saddening, but especially scary because there's not yet enough detail for me to chase down those assertions and see whether they actually apply to my situation. I think the community would be really well served if the posts in this thread were loaded with as much citations as condemnation (links to bug reports, etc). Even if that has the effect of driving some of us away from RubyMotion in the short term, I'd like to make that a data-driven decision rather than a fear-driven decision.

The countervailing possibility is that really there is just RM-3, and that because blocks are everywhere in the Ruby way of coding, any nontrivial RubyMotion apps can be crashy ... even if you never used a block or closed over a local var in any other way yourself, you probably used a gem that did. I think this is where Matt and I are largely at. I don't have my own evidence of memory bugs unrelated to this. If they are out there, I just want to learn about them so I can make intelligent decisions about the problems as a complete set.

[Nick Quaranto]

If a fix is going to take a long time, how about a warning from RubyMotion itself that at least detects when this is going to be a problem? I'd love to see some more code samples of when this happens in action, too.

[Greg Fodor]

I'm probably missing something, but it seems like a potential solution would be to introduce some new (non-Ruby) syntax that allows the programmer to declare which locals need to be bound to the closure a la ObjC. Legacy code could "just work" and perhaps a warning could be turned on if the compiler notices you referencing local vars not declared on the block so you can patch up your code. (Again I'm probably missing something.)

With regards to the tricky edge cases (instance_eval, etc) causing static analysis to prevent proper memory retain/releasing, I'd just say that I'd be disappointed if it's just these edge cases and the quest for a universal fix that has made it so my shipped production code doing the 99% case (local vars) is causing user crashes and a stream of support requests.  If I had known Rubymotion was not production ready and didn't get closures right, I would have stuck with Apple's toolchain which, for all its flaws, I can at least rely upon to be consistently reliable as they improve upon the sugar. I assumed since I was paying for it that it was past the "beta" phase, and if it wasn't, that it would become solid very quickly.

So, it seems to me introducing new non-Ruby syntax is not a step backwards from where we are now (you shouldn't, as a professional developer, ship Rubymotion apps) unless you assume there is a solution that could let us have our cake (no non-Ruby semantics) and eat it too (no memory issues) within reach. I was singing Rubymotion's praises during the development process, but now that I've shipped and I look like a two-bit developer who ships buggy apps to my users my tune has changed.

[Doug Puchalski]

I'm not in favor of non-ruby syntax. My money says what is wrong here is this issue, and others, are not seen as important and are denied attention.

Laurent mentioned he doesn't want to publish a roadmap. I think we deserve one.

You received this message because you are subscribed to a topic in the Google Groups "RubyMotion - Ruby for iOS" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rubymotion/x6-9c__IHH0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to rubymotion+...@googlegroups.com.

[Doug Puchalski]

The only thing we should be making a fuss about is that we have to make a fuss about things.

--
 
---
You received this message because you are subscribed to a topic in the Google Groups "RubyMotion - Ruby for iOS" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rubymotion/x6-9c__IHH0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to rubymotion+...@googlegroups.com.

[Hwee-Boon Yar]

I'm not sure if it's been captured anywhere, and I've mentioned it several times, but RM-3 seems to only talk about not not retaining objects. But there is another related problem — values aren't captured correctly in blocks even if you retain them separately.

Anyway, a good workaround now is to create a separate class wrapping each block operation with its own ivars, as suggested by many people in other threads. In certain cases, this is absolutely necessary as there is no delegate alternative, e.g. #beginBackgroundTaskWithExpirationHandler: and #endBackgroundTask:.

—

Hwee-Boon

http://hboon.com

[Greg Fodor]

Also, since this thread has gotten some attention and Rob was asking for additional specifics, I thought I'd highlight the bug report I *did* submit, a long time ago, which now exists as an addendum to another report, which describes a severe memory leak issue, yet unfixed. Basically, it appears that if you create a closure and reference method arguments from within the closure, those arguments are retained and not released. 

Current issue:

http://hipbyte.myjetbrains.com/youtrack/issue/RM-32

Legacy trackback: 

https://hipbyte.freshdesk.com/helpdesk/tickets/461

As you can imagine, this is particularly damaging in delegate callbacks, since those callbacks often are passed the controller as the first argument, which will be pinned along with it's object graph.

[Jamon Holmgren]

I'm not sure if it's the type of apps I've been making with RubyMotion or my style of coding, but I've rarely run into the bugs you guys are reporting (and when I have, ivars have fixed them). When making ProMotion (up until 0.4) I did have some serious memory leakage, likely (at least partially) due to some of these issues. (My BigDay! Reminder app is as leaky as Congress but the client pulled the plug on continuing development for now so I haven't bothered patching that app. Most of that was likely our fault due to unfamiliarity with RubyMotion nine months ago.)

It's been my experience that most of my reported bugs have been fixed in the next RM release. It's my opinion that the community is "overly positive" as a result of a (generally) really good product. I'm not sure I've used a piece of software this early in development that hasn't had fairly major issues to work through, and overall RubyMotion is surprisingly stable in the apps I've created.

This is the first major bump in the road for RubyMotion and it will be interesting to see how the HipByte team responds.

Jamon Holmgren (@jamonholmgren)

[Laurent Sansonetti]

Hi guys,

I just woke up and read about this. First I have to apologize, I'm not actively following the Google group since a long time due to time constrains.

As far as I know (but correct me if I'm wrong), the issue discussed on this thread is the infamous http://hipbyte.myjetbrains.com/youtrack/issue/RM-3, which basically makes dynamic variables (block variables) subject to premature release. 

That bug existed since the first version of RubyMotion, and I personally tried a few times to fix it, but wasn't able to come with a good solution that would also perserve performance. 

Fortunately, there is a workaround. Instead of using dynamic/block variables, one can use instance variables instead. Instance variables will always create object references. The workaround has been discussed a lot of times and I know that pretty much everyone with a popular RubyMotion app on the store is carefully using it. 

Since the workaround was simple I decided that we should focus on fixing other bugs and adding more features to the platform. Actually, we are right now developing two new major features for the toolchain.

But I realize that the problem might still affect people. I agree that keeping RM-3 alive is a disappointment. So, starting from this very day, we will stop working on other things and 100% focus on fixing RM-3.

You can expect RubyMotion 2.4 to ship with a fix for that problem.

If you have any question or concern please feel free to direct email me: l...@hipbyte.com

Cheers.

Laurent

[Sebastian Bachmann]

Sounds good!

Thank you Laurent.

Regards,

Sebastian

--
 
---

You received this message because you are subscribed to the Google Groups "RubyMotion - Ruby for iOS" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rubymotion+...@googlegroups.com.

[Rúben Fonseca]

Thank you so much Laurent!

Ruben

[Rob Heittman]

Thanks, Laurent, for attacking this, and thanks, Greg for the additional specifics -- I will look for those patterns in my projects as well. Laurent, it would be great if you get to check RM-32 and kin in passing while attacking RM-3. If we're really lucky :-)  But I have to say that for my user stories, leaking -- retaining too much -- is an order of magnitude less concern than crashing -- releasing too much.

[Orion Engleton]

Thank You Laurent

Best

Orion

[Greg Fodor]

This is excellent news, thanks Laurent, looking forward to 2.4 :)

[Steve Ross]

I'm concerned about and tracking this issue as well, however... I feel it's only right to point out how much everyone *hates* Xcode. Buggy, crashing, etc. Ok, that's not the compiler, but the implication that the Apple toolchain is rock-solid is not absolutely true. Add to that the amazing amount of mind-numbing tweaking you have to do in you Obj-C to get proper retain/release semantics and I'm not sold that the Obj-C toolchain is immune.

It's true that random in-the-wild crashes are unfortunate. Really unfortunate. However, I remain optimistic that HipByte will find and fix this.

To HipByte: What can the community do to make the importance of this kind of showstopping issue more visible to you?

Steve

[Colin T.A. Gray]

To HipByte: What can the community do to make the importance of this kind of showstopping issue more visible to you?

Steve

On this note, I want to tell everyone that I feel like I dropped the ball on this bug.  Part of my role as Community Manager at HipByte includes bringing issues from the community to Laurent and Watson, so that they can focus on coding AND keep informed of the woes and joys of the community.

And I did that, I pointed out this thread yesterday (shortly before Josh published his article), but there was general mayhem until Laurent read all the messages (and blog post (and hacker news (and tweets))) this morning.

I'm going to talk with Laurent about how my job can be made more explicit to the group.  Everyone should know that they can come to me and dump heaps of criticism or complaints, and I'll do everything I can do (a) help find a solution and (b) bring those concerns to Laurent and Watson in a constructive way.

What Jamon pointed out also applies: RubyMotion is young, and it's an incredible tool for being so young, but people need to remember the tradeoffs.  For my part, I am much happier being a part of a great community and having to deal with some quirks and hacks, than to be in the Xcode or Xamarin or Titanium world.  The local-variable thing is annoying, but I've dealt with it in a few ways, if not ivars than by using background Worker classes and notifications or using custom NSOperation subclasses and assigning the locals to that class, and if there's a bunch of them that need to run at the same time, maintain a queue.  I've ended up with code that is much better organized, much thinner controllers, and more easily tested.

There is obviously a need to bring these show-stopping bugs into light, but I hope in the future the conversation can be about what bugs and features are being vs NEED to be prioritized.  Less teeth gnashing and running of the mouth.

I think Josh's article is very level-headed and open, and I appreciated reading it a lot more than I did reading through this thread.

@colinta

colinta.com

github.com/colinta

[Kris Rasmussen]

Thank you for fixing this. I'm still concerned that this is just one of many ARC bugs outstanding. I've submitted a number of support issues recently with clear code repros that seem to demonstrate problems with ARC in rubymotion that may or may not be related to this specific bug. For example, I find that often times blocks cause objects to be retained indefinitely, even though I'm unable to find any retain cycles in the code. Given that I've not received a reply to any of these requests I'm going to try opening them in your issue tracker. 

I've spent a lot of time building out a large rubymotion app but I'm also very disappointed with the state of the ARC in rubymotion and feel a little stilly for investing in it at this point. I was just about to rewrite the entire app (months of work) in objective-c. I'd like to hold off, but I need to believe that you guys are actually going to be able to fix all of the ARC bugs around blocks, and not just this one. Like many others out there, I really believe it's critical that you ship a stable ARC implementation before adding any additional features to rubymotion.

So thank you again for going after this bug and listening to the community, but please don't stop with this bug if there are more ARC bugs out there.

[GantMan]

my 2 cents on this bug.

Yeah it sucks, Yeah it's hard to identify.

After burning half a day on figuring out how it bit me once, I solved the issue and was fine with all other work.  Seems this bug has recently grown in popularity, but it's importance has over-inflated.

[Marcus]

Kris you bring up a very good point. I have just noticed major memory problems with my complex app as well. My memory usage is absurd and I have traced to back to this problem of blocks retaining objects indefinitely. This may seem minor, but in my case very large objects are being retained indefinitely and it is eating up a TON of memory. If the solution to RM-3 fixes this issue, then great. If not, this should be addressed ASAP as well, in my opinion. When one of the major selling points of RM is not having to deal with memory management, and it turns out that memory management is buggy and leaky, then I believe this should be a top priority. If it isn't, RM shouldn't be advertised as a system where a developer should have no concern for memory, and it should be clearly documented how to avoid or mitigate the memory problems.

A simple example of a block indefinitely retaining object is below, if anyone is interested in trying it out on their own.

class Tiger

  def dealloc

    puts "DEALLOC"

    super

  end

  def self.blocker(&block)

    block.call

  end

  def self.set_tiger(new_tiger)

    @tiger = new_tiger

  end

  def self.set_tiger_with_block(new_tiger)

    blocker do

      @tiger = new_tiger

    end

  end

end

# Will print "DEALLOC" 10 times which is expected:
10.times { Tiger.set_tiger(Tiger.new) } # => Prints "DEALLOC" 10 times

# Does not print "DEALLOC". All tigers are retained:
# (Although maybe that's better than freeing a bunch of tigers into the streets... :) )
10.times { Tiger.set_tiger_with_block(Tiger.new) }

[Greg Fodor]

Marcus, this looks like it could be RM-32. The "new_tiger" method argument is closed over in "set_tiger_with_block" which causes it to hit RM-32. (Method arguments leak when closed over.) My guess is if you set "new_tiger = nil" at the end of that block you will see things work properly.

In my case, I noticed that many modal view controllers or table views (and their requisite object graphs, which can be huge) were stuck in memory even after being dismissed/popped, and it turned out to be due to delegate callbacks using blocks which receive these controllers as their first parameter. The workaround seems to be to nil out the method arguments at the end of your block, though of course this idiom is so common that inevitably a few slip by and it's hard to track down. I also can't say with 100% certainty this has always done the trick.

[aceofspades]

I wonder if some sort of development mode could be helpful. I'm quite fine with a performance hit if it speeds development by detecting conditions so the causes of the all-to-frequent memory issues can be narrowed down. 

[Marcus]

Greg, thanks for the heads up. I'll try to hunt down all the cases where this is done, however unrealistic that might be. The more I look into this problem the worse it gets. Blocks can't close over method arguments, yet I do that nearly everywhere in my app, since blocks are common place in ruby. Yikes...

[Greg Fodor]

Note I'm pretty sure that this only leaks if you *reference* the method arguments inside your block. So this might save you some refactoring :)

[Joe Noon]

I'm very happy to hear about the focus on memory management. This is a great step to stabilize the foundation of rubymotion. 

​Unfortunately I leave today for a 10 day trip, but I do have a library that I've been using for some time now published on github: http://github.com/joenoon/rm-extensions  - it is my trial and error effort to deal with some of these issues while an upstream fix is in the works. It likely isn't perfect, but it's been a huge help to me. I'd really appreciate community feedback and contribution to find flaws and improve it.

 

On Thu, Jun 27, 2013 at 12:50 PM, aceofspades <dou...@gmail.com> wrote:

I wonder if some sort of development mode could be helpful. I'm quite fine with a performance hit if it speeds development by detecting conditions so the causes of the all-to-frequent memory issues can be narrowed down. 

--
 
---
You received this message because you are subscribed to a topic in the Google Groups "RubyMotion - Ruby for iOS" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rubymotion/x6-9c__IHH0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to rubymotion+...@googlegroups.com.

[Matt Green]

Joe:

Meant to say a few weeks ago: great work. I'm going to give it a try and see if it fixes RM-3 issues.

[Matthew Nguyen]

Since there are a lot of "known issues" and "workarounds", why not gathering them and put them into the official documentation?

I'm sure there would be less people complaining if everything was properly documented.

There are a lot of crucial information hidden in the threads of this google group that definitely should be in the official documentation. And I mean not in the future next new version of the documentation available in several months. I mean NOW.

[Colin T.A. Gray]

There are a lot of crucial information hidden in the threads of this google group that definitely should be in the official documentation. And I mean not in the future next new version of the documentation available in several months. I mean NOW.

Given that 100% of efforts are going towards the memory issues, this is not likely to become a top priority.

However, if you or anyone wants to gather these nuggets and send them to me - I know there are many - I will volunteer to insert them into the documentation.

I'm definitely going to write up a guide on dealing with the local variables.  It's not clear how long it will take to fix the bugs.

@colinta

colinta.com

github.com/colinta

[Muescha]

@colinta:

> For my part, I am much happier being a part of a great community and having to deal with some quirks
> and hacks, than to be in the Xcode or Xamarin or Titanium world.  The local-variable thing is annoying,
> but I've dealt with it in a few ways, if not ivars than by using background Worker classes and notifications
> or using custom NSOperation subclasses and assigning the locals to that class, and if there's a bunch of
> them that need to run at the same time, maintain a queue.  I've ended up with code that is much better
> organized, much thinner controllers, and more easily tested.

Can you make a step by step guide in a blog post for your best patterns (old code vs new code) - it would be nice to see how you solved this.

[Hwee-Boon Yar]

I wrote a post about how I workaround the bug in my app at http://hboon.com/rubymotion-retain-bug-rm3-workaround/ a few days ago. Feel free to ask questions or email corrections.

—

Hwee-Boon

http://hboon.com

[Colin T.A. Gray]

Yeah this really nails it!  With these types of workers running in the background, your code is very well encapsulated, I think.

@colinta

colinta.com

github.com/colinta

[Eric S.]

It looks like RM-3 was just fixed today.  YouTrack says it is fixed and scheduled to be included in the next release.  I believe (according to a Laurent twitter response) that the next update will be out next week.  Maybe we'll see it then.

[Eric S.]

It appears RM-3 was fixed recently.

http://hipbyte.myjetbrains.com/youtrack/issue/RM-3

It says the fix will be included in the next release.

[Colin T.A. Gray]

What are you going by?  It looks "unassigned" (not true, Laurent is working on it), in the "Analyze" state, and "Unscheduled".

@colinta

colinta.com

github.com/colinta

[Colin T.A. Gray]

Btw, the "Fixed in build" setting has a default value "Next build", but that is not actually a reliable metric of whether it's going to be in the next build or not.

@colinta

colinta.com

github.com/colinta

[iwar...@gmail.com]

What version of RubyMotion are you running? I am on 2.2, and couldn't reproduce your first crashing example.

I replaced my app_delegate.rb with your code, ran rake and got this:

"Start"

"running..."

"End, going to run completion block"

"In block"

"In block with task_id 1"

"End task with 1"

"After running completion block"

On Friday, June 28, 2013 12:24:28 PM UTC-5, Hwee-Boon Yar wrote:

[Mark Villacampa]

Looks like it is fixed now:

http://hipbyte.myjetbrains.com/youtrack/issue/RM-3#comment=74-416

[Hwee-Boon Yar]

Not published yet though. But great news!

—

Hwee-Boon

[Hwee-Boon Yar]

I'm on 2.3, but I'm pretty sure it crashed on 2.2 and (likely all prior versions).

Did you copy the code verbatim? Try add a print at the end of #run_task_with_local_vars and verify that is the *first* thing that is printed when the app runs.

—

Hwee-Boon

http://hboon.com

[Ian Warshak]

@Hwee-Boon - I added a print line to the end of #run_task_with_local_vars and it showed up first. I also upgraded to 2.3, and it still worked. Could you post your repo on github? 

--

 
---
You received this message because you are subscribed to a topic in the Google Groups "RubyMotion - Ruby for iOS" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/rubymotion/x6-9c__IHH0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to rubymotion+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

ian

[Ian Warshak]

@Hwee-Boon - nevermind. I was running in the simulator, not device. 

When I do rake debug=1, I get the expected crash. Sorry...

--

ian

[Hwee-Boon Yar]

It's up at https://github.com/hboon/rmblocklocalvarcrash. Crashes for both `rake` and `rake retina=4`. I'm on Xcode 5 DP 2 in case that's different. But I'm very sure it crashed even on prior public releases.

Interestingly, I have always been able to reproduce this in the simulator, and without debug=1.

—

Hwee-Boon

[Laurent Sansonetti]

Hi guys,

As promised, we fixed RM-3 (and other memory-related crashes) in 2.4. The update is available and we recommend that you upgrade.

With this update you should see less crashes related to dynamic variables. In the past, blocks would not retain variables created outside their scope. Now, these variables are allocated as heap-memory inside the Proc object, and the calling scope points to their storage accordingly. 

Also, we rewrote the way blocks are represented in the runtime. Before, blocks were separate data structures cached by the VM object and never freed. Now, blocks and Proc objects are basically the same, we use only one data structure and it's properly released when no longer used. So, you should see less memory usage.

This is a big update with a lot of changes in the compiler and runtime. We tested it intensively by adding a lot of new tests/scenarios in our suite and also by running the test suite of 30+ existing popular RubyMotion libraries (such as BubbleWrap, motion-support, etc.). We also sent this build to some of our users that have large applications.

We are confident the update does not bring regressions and should work fine. However, if you find anything wrong, please contact us via a support ticket. You can then downgrade safely to 2.3 by using the `motion update --force-version=2.3' command.

Now, the team will go back to work and resume hacking on exciting new features :-)

Cheers.

Laurent

On Thursday, June 27, 2013 11:46:19 AM UTC+2, Laurent Sansonetti wrote:

Hi guys,

I just woke up and read about this. First I have to apologize, I'm not actively following the Google group since a long time due to time constrains.

As far as I know (but correct me if I'm wrong), the issue discussed on this thread is the infamous http://hipbyte.myjetbrains.com/youtrack/issue/RM-3, which basically makes dynamic variables (block variables) subject to premature release. 

That bug existed since the first version of RubyMotion, and I personally tried a few times to fix it, but wasn't able to come with a good solution that would also perserve performance. 

Fortunately, there is a workaround. Instead of using dynamic/block variables, one can use instance variables instead. Instance variables will always create object references. The workaround has been discussed a lot of times and I know that pretty much everyone with a popular RubyMotion app on the store is carefully using it. 

Since the workaround was simple I decided that we should focus on fixing other bugs and adding more features to the platform. Actually, we are right now developing two new major features for the toolchain.

But I realize that the problem might still affect people. I agree that keeping RM-3 alive is a disappointment. So, starting from this very day, we will stop working on other things and 100% focus on fixing RM-3.

You can expect RubyMotion 2.4 to ship with a fix for that problem.

If you have any question or concern please feel free to direct email me: l...@hipbyte.com

Cheers.

Laurent

[Jonathan Penn]

GIVE THIS MAN A HUG!

--
 
---
You received this message because you are subscribed to the Google Groups "RubyMotion - Ruby for iOS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rubymotion+...@googlegroups.com.

[Daniel Dickison]

This is really great news.  I'm pulling out some of the workarounds I put in place to do some testing.  Thanks!

One question: is `self' retained during the lifetime of a block (assuming the block accesses methods or ivars of the instance)?  If so, any suggestions for avoiding retain cycles, and if not, is this still a valid solution?

retain

Dispatch::Queue.main.after(10) do

  puts self

  autorelease

end

[Laurent Sansonetti]

Hi Daniel,

The self object is now properly retained by the block. In case the block is a Proc object, just make sure not to keep any reference to it to avoid a cycle.

Cheers.

Laurent

[Eric S.]

I'm curious about this as well, but I think there may be a better solution in this case.  Something like a DYI zeroing weak reference.  Should be easy since you have self retained.

Probably something like:

attr_accessor :my_queue

@my_queue = Dispatch::Queue.main.after(10) do

    puts self

    self.my_queue = nil

end

[Eric S.]

DIY (not DYI) ;-)

[Greg Fodor]

Excellent news!

[Greg Fodor]

Also, re: RM-32:

- Is RM-32 confirmed to be a legit issue?

- If so, was it addressed by this update?

- If not, is it on the roadmap?

Thanks again for your hard work fixing this!

-Greg