> One of the world's great gc experts, Hans Boehm, once observed to me > that reference counting has reasonable performance as long as there > are > not multiple threads, in which case ref counting has serious problems.
I have heard, as a bit of semi-common surprising wisdom, that reference counting (plus a memory allocator) is slower than a good GC because of the overhead of adjusting the reference counts and of managing freeing individual blocks of memory. Not true?
sha-256-hl6w2x74ixy6pi5n.yurl.net looks like a yurl
Now if yurls were really implemented, then when the browser attempted to access httpy://sha-256-hl6w2x74ixy6pi5n.somedomain, what would happen is that it would get the network address, and upon contacting the domain, receive a public key and a rule endorsing that public key, whose sha-256 has was hl6w2x74ixy6pi5n, and from this information, together with the information in its request, construct a shared secret, used to encrypt subsequent communications.
This would have the considerable advantage that since no intermediate trusted authorities are involved, the user would not see complicated mystery error messages and would not be trained to click through those mystery error messages, nor would the authorities be able to mim websites by suborning one of innumerable certificate authorities that no one has ever heard of.
That this looks like yurl implies that something is implemented, though I suspect considerably less than a full yurl implementation. What is actually implemented and working today? _______________________________________________ cap-talk mailing list cap-t...@mail.eros-os.org http://www.eros-os.org/mailman/listinfo/cap-talk
On 2011-03-05 3:43 AM, Raoul Duke wrote: > word. this has bugged me about a lot of gc languages for a > long time. i asked about it wrt alice ml a while back and > they were not interested :-). the C# idiom sucks less than > the java stuff, but it still sucks vs. RAII in C++ > (shudder, i said i liked something about C++?!). clearly > i'm not a vm/gc author, but as a programmer interested in > things being more correct than not, i've wanted a gc (i > have to go look at python now) that checks for zero > ref-counts and frees those then and there.
It seems to me that the only reliable way to have RAII working is to destroy a variable when it goes out of scope, and if another variable has a reference to that variable, generate an error at that moment in the code.
This also corrects the propensity of programs written in GC languages to develop memory leaks, due an ever growing web of variables referencing each other.
Of course this would not play nice with closures, nor 101 other features of garbage collected languages. The RAII idiom is a situation where you really do need to have reliable and direct control over memory allocation, so any attempt to support RAII in a garbage collected language is going to be problematic. _______________________________________________ cap-talk mailing list cap-t...@mail.eros-os.org http://www.eros-os.org/mailman/listinfo/cap-talk
On Fri, Mar 4, 2011 at 3:17 PM, Kevin Reid <kpr...@switchb.org> wrote: > On Mar 4, 2011, at 13:41, Marc Stiegler wrote:
>> One of the world's great gc experts, Hans Boehm, once observed >> to me that reference counting has reasonable performance as long >> as there are not multiple threads, in which case ref counting has >> serious problems.
> I have heard, as a bit of semi-common surprising wisdom, that > reference counting (plus a memory allocator) is slower than a > good GC because of the overhead of adjusting the reference > counts and of managing freeing individual blocks of memory. > Not true?
Both of your statements are true. They aren't contradictory. _______________________________________________ cap-talk mailing list cap-t...@mail.eros-os.org http://www.eros-os.org/mailman/listinfo/cap-talk
On Fri, Mar 4, 2011 at 3:17 PM, Kevin Reid <kpr...@switchb.org> wrote: > On Mar 4, 2011, at 13:41, Marc Stiegler wrote:
> > One of the world's great gc experts, Hans Boehm, once observed to me > > that reference counting has reasonable performance as long as there > > are > > not multiple threads, in which case ref counting has serious problems.
> I have heard, as a bit of semi-common surprising wisdom, that > reference counting (plus a memory allocator) is slower than a good GC > because of the overhead of adjusting the reference counts and of > managing freeing individual blocks of memory. Not true?
For single threaded applications, it proves that most pointer copies are to temporaries, and a suitable optimizer can eliminate the up and down counting on those. For multithreaded applications this optimization is invalid, because races between the threads can cause the counts to go to zero if this optimization is undertaken.
But reference counting doesn't handle cycles. Some other mechanism - which usually turns out to be GC - is needed for that.
On Fri, Mar 4, 2011 at 9:43 AM, Raoul Duke <rao...@gmail.com> wrote: > word. this has bugged me about a lot of gc languages for a long time.
You should avoid depending upon GC-based finalizers. Even assuming that order of destruction isn't a problem for you, the highest performance real-time or concurrent GCs allow a certain epsilon of 'floating' garbage that will never be collected.
> i've wanted a gc that checks for zero ref-counts and frees those > then and there [...] at least do it for those things that are tagged > as being non-simple-memory-gcable resources e.g. via the C# > IDisposable interface or whatever.
That sounds feasible, though it is usually preferable if you know precisely where something is freed. Some sort of linear programming feature, i.e. where you statically prove/enforce that there is only one thread holding the reference in one place, would be sufficient to give you C++ idioms. _______________________________________________ cap-talk mailing list cap-t...@mail.eros-os.org http://www.eros-os.org/mailman/listinfo/cap-talk
On Fri, Mar 4, 2011 at 5:29 PM, Jonathan S. Shapiro <s...@eros-os.org>wrote:
> But reference counting doesn't handle cycles. Some other mechanism - > which usually turns out to be GC - is needed for that.
Also, the performance of optimized reference counting is highly dependent on the application. This is equally true of GC.
In general, GC isn't slow. The problem is that it alters where the cost appears. This can be a practical impediment for two classes of applications:
- Applications with tight pause time constraints - Applications that retain large heaps.
On modern processors, the first can be addressed by concurrent GC (look at Bjarne Steensgaard's work). The second can *often* (but not always) be addressed by region-aware collection and compilation.
On Fri, Mar 4, 2011 at 5:36 PM, David Barbour <dmbarb...@gmail.com> wrote: > On Fri, Mar 4, 2011 at 9:43 AM, Raoul Duke <rao...@gmail.com> wrote: > > word. this has bugged me about a lot of gc languages for a long time.
> You should avoid depending upon GC-based finalizers.
Strongly agree!
> Even assuming > that order of destruction isn't a problem for you, the highest > performance real-time or concurrent GCs allow a certain epsilon of > 'floating' garbage that will never be collected.
Can you source that statement? Such a collector would, by its nature, be imprecise, and there are just a hoard of issues with an imprecise collector of any sort. I'm not aware of any true-parallel concurrent and incremental collector, for example, that allows garbage to survive for more than two generations.
If you know of a counter-example, I *really* want to look at it.
> > i've wanted a gc that checks for zero ref-counts and frees those > > then and there [...] at least do it for those things that are tagged > > as being non-simple-memory-gcable resources e.g. via the C# > > IDisposable interface or whatever.
> That sounds feasible...
Feasible, but you don't want it, because you don't want to take the latency consequences of the chain of deletes that can result from this.
GC is one of those things where hybrid approaches are often unhappy things. What I suspect Raul is after here is actually a region management sort of an issue, but the research on region-aware GC is very much a live issue.
On Mar 4, 2011 5:33 PM, "Jonathan S. Shapiro" <s...@eros-os.org> wrote:
>> I have heard, as a bit of semi-common surprising wisdom, that >> reference counting (plus a memory allocator) is slower than a good GC >> because of the overhead of adjusting the reference counts and of >> managing freeing individual blocks of memory. Not true?
> For single threaded applications, it proves that most pointer copies are
to temporaries, and a suitable optimizer can eliminate the up and down counting on those. For multithreaded applications this optimization is invalid, because races between the threads can cause the counts to go to zero if this optimization is undertaken.
Also, in a multi-core world, concurrent gc is one way to get at least a bit more parallel execution out of an app "for free:" Looking at the app (mutator) threads, the barrier overhead imposed by concurrent gc (e.g. write barrier prior to each salient heap update) can reasonably be expected to be much less costly than the refcount accounting overhead imposed by a refcount-based heap. And with concurrent gc, as the name implies, much/most of the heap management work can happen without any involvement from the app threads, possibly on a separate CPU core and hence not even competing with the app threads for CPU time.
On Fri, Mar 4, 2011 at 5:46 PM, Jonathan S. Shapiro <s...@eros-os.org> wrote:
>> Even assuming that order of destruction isn't a problem for you, >> the highest performance real-time or concurrent GCs allow a >> certain epsilon of 'floating' garbage that will never be collected.
> Can you source that statement? Such a collector would, by its nature, be > imprecise, and there are just a hoard of issues with an imprecise collector > of any sort.
The statement that the highest performance collectors will have 'floating garbage' is based on my own research, though I should have been more precise: I'm talking about performance in large-scale, concurrent, real-time garbage-collection where most of the 'image' is on disk. It is infeasible to run a sweep across all of data in such systems. You preferentially favor collecting 'regions' that are mostly in-cache or in-memory. But, because you collect only part of a system at a time, you will have floating garbage. Good design can keep floating garbage down to an epsilon (e.g. you guarantee a maximum of 10% floating garbage).
I am curious what 'hoard of issues' you anticipate. My understanding is that the epsilon bound takes care of most issues. E.g. you can still support real-time and bounded-space programming. The only major problems with it I know of are: (a) you cannot depend on gc-triggered finalizers, and (b) you might be concerned about passwords and such hanging around too long or making it into long-term memory.
In the gc/allocation system I was designing, (a) was handled by other design, and (b) was handled by some guarantees that small and short-lived things are properly collected before ever making it to disk.
> I'm not aware of any true-parallel concurrent and incremental collector, for > example, that allows garbage to survive for more than two generations.
It is true that whole-memory GCs will be precise. Perhaps the region-based variations never came up in your research?
I.e. I don't even use whole-system 'generations'. I have a couple generations per a processor, but a processor doesn't even bother doing a larger collection if its small collections (each processor has a rotating nursery and 1st gen pages it 'owns') are getting the job done. (This fits the epsilon requirement since no new floating garbage is being introduced, but it also means that no 'old' garbage is being destroyed.)
> Feasible, but you don't want it, because you don't want to take the latency > consequences of the chain of deletes that can result from this.
Yeah, unpredictable spikes in latency are a problem for real-time programming. You need to keep your latency spikes predictable.
On Thu, Mar 3, 2011 at 4:48 PM, David Wagner <d...@cs.berkeley.edu> wrote: > My recollection is that Waterken is structured as follows: there is > a main loop, written in Java, which calls Joe-E code for each turn. > I believe there is a top-level exception handler in the main loop which > catches any exceptions or errors thrown by the Joe-E code and handles them > appropriately. I am hoping it handles VirtualMachineErrors by aborting > the turn and rolling back its state, or something else equally fatal, > rather than exposing the inconsistent state of the Joe-E code to others. > However, I haven't verified that this is indeed the case and I might be > totally confused. Hopefully Tyler can correct any mistakes in the above.
> I don't know what the tradeoffs are with making Errors fatal. Perhaps the > main loop could add an exception handler that catches all Errors, so that > if an Error occurs, the turn is aborted and the state is rolled back to > that before the turn was attempted? I'm speculating wildly here...
That's exactly what the Waterken server does... the Error rollback, not the speculating wildly. ;)
As a consequence of this implementation, the source vat that originated the call that resulted in an Error is no longer able to successfully send any further messages to the destination vat. Other vats can still successfully message with the destination vat. This is useful in that a browser talking to a Waterken server vat might send a request that causes an Error, but this won't prevent other browsers from successfully exchanging messages with the server (modulo the DOS issues). One case that is unfortunate is when the application itself is buggy and so causes an Error in one of its own local asynchronous calls. This prevents the local event queue from making further progress. External clients can still message with the vat, but having the local event queue permanently stalled might prevent the application from making progress.
> sha-256-hl6w2x74ixy6pi5n.yurl.net looks like a yurl
> Now if yurls were really implemented, then when the browser attempted to > access > httpy://sha-256-hl6w2x74ixy6pi5n.somedomain, what would happen is that > it would get the network address, and upon contacting the domain, > receive a public key and a rule endorsing that public key, whose sha-256 > has was hl6w2x74ixy6pi5n, and from this information, together with the > information in its request, construct a shared secret, > used to encrypt subsequent communications.
> This would have the considerable advantage that since no intermediate > trusted authorities are involved, the user would not see complicated > mystery error messages and would not be trained to click through those > mystery error messages, nor would the authorities be able to mim > websites by suborning one of innumerable certificate authorities that no > one has ever heard of.
> That this looks like yurl implies that something is implemented, though > I suspect considerably less than a full yurl implementation. What is > actually implemented and working today?
The Waterken server implements full YURL semantics and so for distributed apps composed of Waterken servers talking to each other you have the security properties described above. Much of the work MarcS and Alan have been doing involves Waterken servers talking to each other, so it works out for them.
The longstanding gap in the system is compatibility with the Web browser. A Web browser will initially pop an annoying dialog when navigating to a YURL. Even after clicking through the dialog to accept the certificate, the browser still allows the site to be impersonated by a certificate from a recognized CA. None of the browsers make it easy to fix this problem, so it'll require someone sitting down for a few weeks to crank out the needed code. Somehow, I've always had other things higher up on the TODO list.
On Sat, March 5, 2011 02:29, Jonathan S. Shapiro wrote: > On Fri, Mar 4, 2011 at 3:17 PM, Kevin Reid <kpr...@switchb.org> wrote:
>> On Mar 4, 2011, at 13:41, Marc Stiegler wrote:
>> > One of the world's great gc experts, Hans Boehm, once observed to me >> > that reference counting has reasonable performance as long as there >> > are >> > not multiple threads, in which case ref counting has serious problems.
>> I have heard, as a bit of semi-common surprising wisdom, that >> reference counting (plus a memory allocator) is slower than a good GC >> because of the overhead of adjusting the reference counts and of >> managing freeing individual blocks of memory. Not true?
> For single threaded applications, it proves that most pointer copies are > to > temporaries, and a suitable optimizer can eliminate the up and down > counting > on those. For multithreaded applications this optimization is invalid, > because races between the threads can cause the counts to go to zero if > this > optimization is undertaken.
> But reference counting doesn't handle cycles. Some other mechanism - which > usually turns out to be GC - is needed for that.
If I understand correctly that is also how cPython works. Reference counting as base paradigm and gc for the very rare cases where cycles hapen. I feel this construct, that still allows for C++ style destructor based RAII should be the way that object capability languages work. IMO the principle of least authority favours destructor based RAII over 'finaly' constructs (see my example earlier) AND favours message passing concurency over shared state concurency. The combination of these two seems like a good fit for using a RAII/destructor compatible cPython style way of reference counted collection.
On Fri, Mar 4, 2011 at 11:10 PM, Rob Meijer <capib...@xs4all.nl> wrote: > I feel this construct, that still allows for C++ style destructor based > RAII should be the way that object capability languages work.
I feel we should more broadly reconsider the whole notion of resource acquisition and resource management from a security perspective. I've said before that, rather than associating memory with 'regions', we should instead be associating them with a 'purse'. I have a lot of Nick Szabo's research in this area (cf. http://szabo.best.vwh.net/scarce.html). But that research is still immature. Even without jumping to a full market-based solution, though, I feel we could avoid explicit 'acquire/release' patterns in ocap systems. (They're fairly problematic for distributed programming and orthogonal persistence in any case.) _______________________________________________ cap-talk mailing list cap-t...@mail.eros-os.org http://www.eros-os.org/mailman/listinfo/cap-talk
> I distinguish that between __del__ and finaly for the reason that finaly > often braiks the principle of least knowledge while __del__ does not, and > for the reason that finaly spreads responsibility for a resource all the > way up the tree from where it is being initially acquired, thus 'finally' > in fact brakes encapsulation principles. Let me give a simple example.
> Lets say Alice acquires a new Bob and a new Carol. Now Bob acquires a Dave > and Carol acquires an Eve. Now Dave acquires a resource lets say a lock.
> Alice invokes a method of Carol who invokes a method of Eve and this last > method throws an exception.
> With __del__, when the exception travels up, the exception will let Bob go > out of scope, bob gets destroyed, its member Dave with it. Dave's > destructo gets called and the lock will get released promptly.
> Now look at the finaly solution. Dave will need a 'release' method so that > Bob can ask it to release its lock resource (should Dave know there is a > lock resource?). Bob will also need a 'release' method so that Alice can > ask Bob to ask Dave to release the lock. Now with that in place, Alice can > invoke Bob.release() in its finaly for invocation of Carol.somemethod(). > This basicaly means two things to me. Alice and Bob become responsible for > resource management of a resource held by Dave. This brakes the principle > of least knowledge and makes the whole system brittle by spreading > responsibility and breaking encapsulation.
> I hope the above example clarifies my argument.
I see.
And yet, tying resource management to object lifetime (in the refcounting or gc sense) is potentially confusing* and composes awkwardly. How does Alice know that she is using this resource and must not cache Bob? Is she expected to do that "DisTasteful = Nothing" that you frequently see in VB code before calling other methods that may require a resource? Idiomatically?
If Dave cannot acquire the lock as a with statement because it is expected to be held across several calls, the appropriate interface would be to return a closable that the caller invokes those methods within context of.
For those situations where this sort of pattern is not possible, I would like to have an interface to consistently access either dynamically scoped (in the 'with' sense) or transaction scoped objects that assist in lifecycle management of such resources. While this approach still has some composition related overhead (you must explicitly create a new lifecycle context when Alice is to be passed a closure by Bob in the above example, with which he tags resources associated with the return value, etc), it means that the programmer is not constrained by an implicit stack discipline (inflexible B&D) or a ref-counting algorithm (fragile and difficult to reason about). It puts lifecycle management in the hands of the programmers.
On the other hand, that idea seems to share some symmetry with explicit region annotation, applied to abstract resources rather than just memory. Hard to say if it would feel just similarly complicated.
* I believe it was Tim Peters who said that __del__ is responsible for burning more brain cells than every other python language feature combined (including descriptors and metaclasses). From my (possibly very biased!) circle; weakref callbacks seem to be the generally preferred interface over __del__. Because they provide fewer and simpler guarantees, you are forced to understand what you are doing. Much like the difference between threads and processes or asynchronous message sends. Where asynchronous sends force you to structure your application, weakref callbacks force you to think about what conditions need to be preserved when your finaliser is called.
> But reference counting doesn't handle cycles. Some other mechanism - > which usually turns out to be GC - is needed for that.
Referencing counting is GC, but does need a cycle collection algorithm (like Bacon's Recycler) to be "complete". Perhaps by "GC", you meant it turns out to be "tracing", which I agree, cycle collection is a form of local tracing.
On 2011-03-04 9:41 PM, Dan Bornstein wrote:
> Also, in a multi-core world, concurrent gc is one way to get at least a > bit more parallel execution out of an app "for free:" Looking at the app > (mutator) threads, the barrier overhead imposed by concurrent gc (e.g. > write barrier prior to each salient heap update) can reasonably be > expected to be much less costly than the refcount accounting overhead > imposed by a refcount-based heap.
Perhaps of naive ref-counting implementations. See Bacon and Petrank's on-the-fly ref-counting GC. Increment/decrement elision techniques significantly improve performance, and deferred ref-counting schemes scale fairly well in concurrent scenarios.
> And with concurrent gc, as the name > implies, much/most of the heap management work can happen without any > involvement from the app threads, possibly on a separate CPU core and > hence not even competing with the app threads for CPU time.
Indeed, see "on-the-fly" GC. There are reference counting and mark-sweep variants available. The pause times are very low (soft real-time), but the throughput isn't quite as good as stop-the-world GCs.
I have read some messages on this thread and skimmed others. I've also occasionally looked at explanations of RAII. I must be missing something, as I don't get it. It seems we're talking about two different use cases, both of which are better handled by other more conventional and safer means:
1) Releasing local memory, i.e., RAII as a convenience for expressing manual memory deallocation.
2) Releasing an external resource, or more generally, reliably bracketing arbitrary code with an acquire action and a release action.
For #1 by itself, I think others here have already explained why GC is generally better. Given a well engineered GC, the one remaining case justifying manual deallocation is modeled well by something like KeyKOS space banks, where Alice has allowed Bob to reside in memory she own and now wishes to evict Bob. If other have references to Bob, those references become broken, preserving memory safety. Such safe manual deallocation trades one availability threat (resource exhaustion) for another (broken references), which is often a fine thing.
If we consider "local" to mean "only within a preemptively deallocatable unit, then we've mapped such inter-unit deallocation to #2.
#2 is where I think I must be missing something. Let's take the familiar case of opening a file to get a stream, doing stuff with the stream, and then reliably closing the stream when we're done with it, whether by normal or exceptional exit. I understand how to code this in RAII. But why isn't the following just as good in a language without RAII:
The equivalent of defining a given RAII abstraction:
On Mar 5, 2011 9:18 AM, "Sandro Magi" <naask...@higherlogics.com> wrote:
> On 2011-03-04 9:41 PM, Dan Bornstein wrote: > > Also, in a multi-core world, concurrent gc is one way to get at least a > > bit more parallel execution out of an app "for free:" Looking at the app > > (mutator) threads, the barrier overhead imposed by concurrent gc (e.g. > > write barrier prior to each salient heap update) can reasonably be > > expected to be much less costly than the refcount accounting overhead > > imposed by a refcount-based heap.
> Perhaps of naive ref-counting implementations. See Bacon and Petrank's > on-the-fly ref-counting GC. Increment/decrement elision techniques > significantly improve performance, and deferred ref-counting schemes > scale fairly well in concurrent scenarios.
It also doesn't seem to guarantee collection being particularly prompter, since there's always the risk of having to wait for the cycle collector before seeing memory/resources get reclaimed. I thought the point of advocating for refcounting in the context of RAII was specifically about the guarantees, so it's not clear to me what the real advantage of "on-the-fly" is, in this case.
My high-level take is that the paper shows there is still plenty of room for gc innovation, not that one technique is objectively superior.
Indeed, though the two orders of magnitude lower pause times are a great selling point. This GC would provide hard real-time bounds for programs executing in CPS form, though it's the sliding views and not the ref-counting that matters here. Petrank's mark-sweep based on sliding views has essentially the same advantages.
> It also doesn't seem to guarantee collection being particularly > prompter, since there's always the risk of having to wait for the cycle > collector before seeing memory/resources get reclaimed. I thought the > point of advocating for refcounting in the context of RAII was > specifically about the guarantees, so it's not clear to me what the real > advantage of "on-the-fly" is, in this case.
Yes, prompt reclamation was not the goal. This sort of GC has three advantages that I can see:
1. More efficient in tighter heaps (discussed in the paper). 2. Friendlier to virtual memory, unlike tracing. Major collections of large heaps in tracing GC may involve lots of paging. You can imagine how a ref-counting scheme with local cycle collection is less likely to page in objects that are not being operated on. 3. No "stop the world", ie. mutators and collectors are all running concurrently, resulting in very low latency (~2ms IIRC).
On Sat, Mar 5, 2011 at 9:52 AM, Mark S. Miller <erig...@google.com> wrote:
> 2) Releasing an external resource, or more generally, reliably bracketing > arbitrary code with an acquire action and a release action.
Your "more generally" is the error in reasoning, here. Bracketing code in this manner requires that you know, statically, the scope in which a resource might be used. More generally, you must manage resources where the scope for holding the resource is not statically known. For example, the 'with(resource-controller, action)' idiom would not work so well if some interactions with the resource wait on a promise (when block). Data-streams are relatively common cases where the lifetime of a resource is not known statically.
It is with these broader cases in mind that GC-triggered finalization code (e.g. destructors or an IDisposable action) become more flexible than try/finally and with blocks. Basically, since you lack static knowledge for the scope of a resource, you are relying upon a GC service to provide conservatively estimated feedback about maximum scope of a resource. _______________________________________________ cap-talk mailing list cap-t...@mail.eros-os.org http://www.eros-os.org/mailman/listinfo/cap-talk
On Sat, March 5, 2011 18:52, Mark S. Miller wrote: > #2 is where I think I must be missing something. Let's take the familiar > case of opening a file to get a stream, doing stuff with the stream, and > then reliably closing the stream when we're done with it, whether by > normal > or exceptional exit. I understand how to code this in RAII. But why isn't > the following just as good in a language without RAII:
> The equivalent of defining a given RAII abstraction:
> where "stream.available()" is an example of "arbitrary code that uses the > stream", as one might have coded in a C++ RAII block.
> I did see a comment in this thread that try/finally isn't as good as RAII, > but I don't understand why. What am I missing?
What I think you may be missing is the combination with 'encapsulation' that I tried to give an example of. Lets say your file handle resource is a deeper encapsulated resource. With RAII the knowledge that there is a file resource that needs closing stops with the file class. Once an object with a deeply encapsulated resource goes out of scope 'close()' will get invoked on the file handle. With 'finaly', the close() method will need to be copied by the encapsulating object and any object encapsulating the encapsulating object, etc.
On 2011-03-06 3:52 AM, Mark S. Miller wrote: > The equivalent of defining a given RAII abstraction: > > ? def withInputStream(file, func) { > > def stream := file.inputStream() > > try { > > return func(stream) > > } finally { > > stream.close() > > } > > } > > The equivalent of using it: > > ? withInputStream(<file:index.html>, fn stream { > stream.available() }) # value: 1510 > > where "stream.available()" is an example of "arbitrary code > that uses the stream", as one might have coded in a C++ > RAII block. > > I did see a comment in this thread that try/finally isn't > as good as RAII, but I don't understand why. What am I > missing?
In your example there is only one resource, and not much code, so it is easy to make sure that for the one open, there is always the one close.
Suppose, however, there are thousands of files each containing thousands of lines of code and hundreds of resources written by hundreds of people over a period of a decade or two.
Suppose your code, your little procedure that runs for a few milliseconds in a gigantic program written by hundreds of people that runs for weeks at a time, grabs a mutex. Then something unexpected and bad happens, possibly in a routine written by someone else years ago, which routine does all sorts of strange stuff unknown to you. Whatever your code was doing is now aborted, so obviously the mutex has to be released. An exception is thrown, possibly in someone else's code, and handled at much higher level in the code, also someone else's code - a high level in the code that cannot possibly know whether the mutex was grabbed or not, and should not even know that the mutex exists. _______________________________________________ cap-talk mailing list cap-t...@mail.eros-os.org http://www.eros-os.org/mailman/listinfo/cap-talk
On Sat, Mar 5, 2011 at 12:11 PM, Sandro Magi <naask...@higherlogics.com> wrote: > Yes, prompt reclamation was not the goal. ["On-the-fly" refcounting] GC has three > advantages that I can see: [...] > 2. Friendlier to virtual memory, unlike tracing.
This isn't clear to me. A generational copying tracing gc (I think that's a reasonably accurate, if glib, description of the dominant gc paradigm) has pretty friendly virtual memory behavior.
> Major collections of > large heaps in tracing GC may involve lots of paging.
My impression is that the "on-the-fly" cycle collection work would involve similar amounts of memory access.
> 3. No "stop the world", ie. mutators and collectors are all running > concurrently, resulting in very low latency (~2ms IIRC).
State-of-the-art concurrent tracing collectors are nearly pauseless as well. For example, my understanding is that the much-lauded Azul gc only pauses one thread at a time, and only to do a root set scan. My impression is that such pauses are in the low-to-fractional msec range on Azul's hardware.
At the other end of the sophistication scale, as of the Gingerbread release (late 2010), Dalvik (the application-running virtual machine that the Android project uses) includes a fairly straightforward non-generational concurrent tracing gc. On currently produced Android hardware — which is quite wimpy compared to modern desktop / server hardware — we see typical "stop the world" pause times in the 2–4msec range (with two such pauses per full heap trace[*]), and the high end of that is usually because the process got preempted during the pause.
-dan
[*] The first pause is to queue up the root set. The second pause is to scan the "card table" and rescan any memory that experienced pointer updates during the concurrent scan.
>> While I believe that Joe-E indeed fixes a lot of loop-holes in Java, there >> seems to be one big design flaw in Java (and any JVM dependant language, >> and some other modern languages), and that is the fact that the often >> praised concept of garbage collection breaks resource management. When a >> language has support for (real) destructor, this enables a resource >> management idiom called RAII (Resource Acquisition Is Initialization).
> Why is this an issue in most cases the GC is trusted code and can release > memory when needed if you want to do something like RAII in .NET you just > use a using block ...
I agree. A "with"/"using" construct in practice has all the advantages of RAII, but without placing any constraints on memory management techniques. Only a small minority of object types need timely disposal, so it is a mistake to place constraints on the memory management by conflating disposal with memory reclamation for all objects.
I also find a "with" construct clearer when reading code -- the use of this construct makes it explicit that disposal at the end of the "with" scope is a correctness requirement, whereas it is not a correctness requirement for objects that are initialized without using "with".
> (other resources like file handles, locks etc. are normally released in > a finalizer before the current object is collected) .
If a file handle or lock hasn't been closed or released explicitly and is still open/acquired in its finalizer, that's an application bug, and should be at least logged as such. There's a case for treating it as a fatal error, so that it can't be ignored and is more likely to be caught during testing.
> This isn't clear to me. A generational copying tracing gc (I think > that's a reasonably accurate, if glib, description of the dominant gc > paradigm) has pretty friendly virtual memory behavior.
> State-of-the-art concurrent tracing collectors are nearly pauseless as > well. For example, my understanding is that the much-lauded Azul gc > only pauses one thread at a time, and only to do a root set scan. My > impression is that such pauses are in the low-to-fractional msec range > on Azul's hardware.
Sure, emphasis being "on Azul's hardware". The on-the-fly collectors have this exact behaviour without requiring special hardware, and still exhibit an order of magnitude lower latency than even the JVM's new, and much lauded, tunable "garbage first" algorithm.
Also, the 2ms latency of the on-the-fly was on a 550Mhz PIII, so scale accordingly to current CPU speeds. We have at least 5.5 times more transistors, so we're looking at < 0.5ms on current hardware.
> At the other end of the sophistication scale, as of the Gingerbread > release (late 2010), Dalvik (the application-running virtual machine > that the Android project uses) includes a fairly straightforward > non-generational concurrent tracing gc. On currently produced Android > hardware — which is quite wimpy compared to modern desktop / server > hardware — we see typical "stop the world" pause times in the 2–4msec > range (with two such pauses per full heap trace[*]), and the high end > of that is usually because the process got preempted during the pause.
It's not clear how to compare these times due to the differences in architecture; how does a 1GHz Hummingbird CPU compare to the 550MHz PIII Xeon used in the paper?
Would be nice to have these benchmarks compare cycles or instructions instead of execution times. The former could at least give us some idea how this scales with increasing processor speed and across architectures.
I suspect the low pause times in the Dalvik GC are due to the different heaps on Android devices: small applications with relatively small heaps. But, I haven't read up on Dalvik's GC, so perhaps the difference isn't as significant as I believe.
In any case, for soft real-time purposes, as long as you can guarantee less than ~3ms, I think it's generally good enough for audio work, which is a pretty demanding domain.
