In article <359AEA95.8D99B...@ccnet.com>,
Gilbert Chang  <gch...@ccnet.com> wrote:

Yes, you are correct.

No, OMG has no concrete plans to address this in the future.  Some
people in the OMG community think it would be a good idea, but nobody
is running with it at the moment.  [Somewhere, I saw a message last
year from someone who wanted to form a mailing list on this topic, but
I've seen nothing since.]

I am not aware of any CORBA implementations that address garbage
collection at all.  Given that there is no CORBA spec for GC on the
horizon, there is no real basis for interoperability ...

The most common approach seems to be to ignore the problem, and arrange
that clients and servers are restarted periodically to flush out any
transient garbage objects.

Other approaches include

  1) define the IDL so that a client can clean up after itself,

  2) have the server delete objects that haven't been used recently, or

  3) have the server check on the liveness of clients using callbacks
     or some other mechanism.

IMO, all of these are non-scalable, band-aid solutions, and all
"break" in some circumstances.  [The same applies to DCOM's pinging
mechanism too, IMO]  But they seem to work well enough that people
can use them.

-- Steve

Guys,

Check out the evictor pattern on IONA's web site - the idea is that if your
objects are recreatable ( that is have persistent state ) then you can
deactivate them at any time - ususally when inactive. The clever bit is that
if the ORB receives a request for a Persistent Object ( CORBA Object sense
not state!)  which has been deactivated, it will notify a locator /
activator object which is registered by the application and given the
opportunity to recreate the object on demand - this allows you to evict
inactive objects and recreate them totally transparently at the server. You
may also be interested in the CORBA 2.2 POA spec which defines support for
very similar functionality using Servant Activators.

Cheers,

Steve Moses.

P.S. I have implemented a system with this solution using Visigenic
Visibroker and it is definately not a band aid soultion IMHO.

couldn't you solve the problem you describe here by architecting your
server+clients in such a way that when a server wants to check if some object
is still alive it 'pings' the client (callback?) and requires a response.

no response/error/exception - client is gone and its objects should be removed
from server
response - still ticking, let it live.

what am I missing? :)

Otis

-----== Posted via Deja News, The Leader in Internet Discussion ==-----
http://www.dejanews.com/rg_mkgrp.xp   Create Your Own Free Member Forum

On Fri, 3 Jul 1998 o...@my-dejanews.com wrote:

Nothing. However, the callback pattern is fraught with danger, whether
you use it for garbage collection or anything else. In particular, it
does not scale. Once you have more than a few clients, the server spends
most of its time calling back instead of doing real work.

Also, with callbacks, the server is forced to deal with failures in
the clients.  For example, if a client is slow or catatonic, the server
gets stuck completely if it is single-threaded. So, to make the
callback pattern even vaguely reliable, you have to use threads. You can
make each callback in a different to avoid getting completely stuck in
the server. But what do you do with very slow clients? If clients are
non-cooperative and take a very long time to return control from a callback,
a thread is lost to the server for each callback. What is worse, the server
cannot recover the lost threads. Thread cancellation may not be supported
by the threads package, and even if it is supported, the effects of
cancelling a thread in the middle of an outstanding invocation are
completely undefined.

If you try to use the callback pattern for purposes other than garbage
collection, the picture gets even worse, because now the server must
have strategy for dealing with failed callbacks. What should the server
to if it gets COMM_FAILURE, TRANSIENT, INV_OBJREF, etc? Especially
TRANSIENT is hairy, because that is not a hard error. How often should
the server retry the callback (if at all) and at what intervals?

If that isn't enough, for single threaded clients and/or servers,
the callback pattern can result in deadlock, depending on the ORB.
For example, if as part of a callback, the client sends an operation
to the server, with most ORBs, client and server will both deadlock if
they are single threaded.

And on top of it all, as soon as a client wants to be able to receive
a callback, it typically must run a separate event loop in a separate
thread, or explicitly poll the event loop (some ORBs don't require
either threading or polling of the event loop to avoid deadlock, but most
don't).

And architecturally, callbacks suck big time. Why? Because they create
an extremely tight coupling between clients and server. Every client
must know about the server, and the server must know about all its
clients. When you think about it, callbacks create what John Lakos
calls a physical circular dependency, and he documents the negative
effects of such dependecies extremely well (I highly recommend his
book "Large-Scale C++ Software Design").

It turns out that if you want to use the callback pattern for anything
like industrial-strength software, you run into implementation problems
big-time. My personal opinion is that the callback pattern is evil in
distributed systems. It is one of those patterns that work fine in
the non-distributed case, and create havoc in the distributed case. I would
use callback only if:

        - there is only a single client that needs to be called back,

        - there is no chance that while executing a callback, the client may
          attempt to invoke another operation on the server via some
          call chain (at least if I care about portability),

        - the server is threaded anyway (otherwise I end up adding
          threads just to support the callback properly),

        - the server can afford to permanently lose a thread to a
          client that goes catatonic,

        - I trust the client to be called back a lot (in other words,
          I have implemented both client and server myself).

If these conditions are not met, it is probably far easier and more
reliable to use an event service instead of callbacks.

                                                        Cheers,

                                                                Michi.
Copyright 1998 Michi Henning. All rights reserved.
--
Michi Henning              +61 7 33654310
DSTC Pty Ltd               +61 7 33654311 (fax)
University of Qld 4072     mi...@dstc.edu.au
AUSTRALIA                  http://www.dstc.edu.au/BDU/staff/michi-henning.html

In article <899419864.20008.0.nnrp-01.c2dee...@news.demon.co.uk>,

Unfortunately, most CORBA objects can't be automatically recreated
from thin air.  Instead, most objects have state that needs to be made
persistent before the activations can be evicted.  And if you have
persistent state, then you need to know when you can safely delete it.

Thus you have only solved the problem of garbage collecting object
activations, not the objects themselves.  While this is useful, it is
not what people normally mean when they talk about garbage collection
for CORBA.

-- Steve

Evictors work well if the object is in some quiescent state, yet still
considered in-use by some client.  This can occur if, for example, the
client got temporarily disconnected, but wishes to re-establish the
connection at a later time.

The problem with Evictors is they don't address garbage collection per-se.
It's still difficult to know when it is safe to destroy the object.  So you
end up with resources being consumed when it would be possible to reclaim
them.  In this case the resource is (likely) a disk and not RAM, but it's
still finite so there needs to be some way to clean it by hand.

--
Regards,
          Greg

---
Intelligy Advanced Software
gp...@intelligy.com

<899419864.20008.0.nnrp-01.c2dee...@news.demon.co.uk>...

In article <Pine.OSF.3.96.980703175125.30077D-100...@tigger.dstc.edu.au>,
                Michi Henning <mi...@dstc.edu.au> wrote:

Another one of these sweeping statements :-)

Every design is fraught with danger, yet we have to use one to get anything
done. So let us re-phrase your concern:

   Applications for which scalability to huge numbers of observers is
   important must be careful to avoid a situation where servers spend
   most of their time doing callback processing.

Scalability will be a tricky issue for most design patterns, I don't think
it is unique to the Observer (aka MVC, callback, ...) pattern. How can it
be made scalable? Well, this will depend on the nature of the application.
If there are many independent "Model" objects one can distribute the Model
among as many servers as possible. If there are many views per model one
can put "mediators" between servers and observers, i.e. the server notifies
one or a few mediators, which are then responsible to perform the actual
callbacks to observers.

The server will have to deal with this issue whenever it maintains
per-client state. The Observer pattern is not the only design that requires
per-client state. Only a completely stateless design (which may be
appropriate for certain applications, but not for all) can avoid the
problem.

Funny, I'm working on a project right now where this had to be solved :-)
We simply delegated the task of dealing with "clients" to a multi-threaded
mediator, which allocates a small number of threads per client.

Without threads it would be difficult in a CORBA environment. It can be
done, however, and has been done in the past with the help of Unix select()
or poll() system calls in pre-CORBA days...

These issues are very application-dependent. If there are really huge
numbers of (unreliable) clients the thread budget will be a problem. One
can make the callbacks "oneway" (and let us not repeat the argument about
how useless or useful this is!) to reduce the problem. Mediators also help.
Not all applications need to worry too much about this.

Very application-specific. Depending on requirements one can either abandon
clients quickly, or make more or less elaborate efforts to determine
whether the client is really gone before cleaning up.

Yes, this is an interesting issue. In fact, it surprises me that there is
so much discussion about "oneway" being underspecified, yet one never hears
any complaints about the lack of any requirements about distributed,
recursive, method invokations! In normal, non-distributed programs
recursion is a common design pattern. Yet some CORBA implementations lock
up when this is attempted between processes! I know Orbix behaved this way
when we last evaluated it, which was one reason why we decided against
using it. Funny also, that the much maligned oneway feature can often be
used as a workaround to avoid these deadlocks in such ORBs.

No.  Whether or not it introduces tight coupling depends on how the
interfaces are designed. Abstraction is the key here, one can easily
achieve a design with very weak coupling, although it is, of course, easy
to get tight coupling as well.

In big software projects you run into big problems, what else is new? The
Observer pattern is one tool of many, which can be used to advantage, and
can also be misused.

Don't agree.

If there are many clients you must work hard to achieve scalability no
matter what your design pattern is. See suggestions above for ways to make
callbacks work in such an environment.

Choose ORBs that don't have this problem. I won't go into the portability
issue again here. Usable systems are being built with less than perfect
tools and standards. Not every application must be interoperable with every
ORB on the market...  Applications that do have this portability
requirement are another story and will have to restrict themselves to the
subset of portable features.

Or it is shielded by mediators.

See above.

Both sides need to agree on an interface, and on reliability requirements.
Depending on how much you can rely on the other side behaving well, you
have to code your side more or less defensively. Whether I have implemented
both sides is irrelevant. In a big project you often don't have the luxury
to implement both sides.

Joachim

---
joac...@kraut.bc.ca   (http://www.kraut.bc.ca)
joac...@mercury.bc.ca (http://www.mercury.bc.ca)

In article <jQ9n1.106459$zu1.12655...@news.rdc1.bc.wave.home.com>,

I disagree.  If you use a pervasive automatic garbage collector, it
will naturally be aware (sooner or later) that a client has died or
has "forgotten" about an object reference by some other means.  If
your system is non-transactional, the server probably needs to do
nothing more than let the collector reclaim the object.

If your system is transactional, then maybe you could rely on garbage
collection of the transaction context object in the server to trigger
a transaction abort.  Or more likely, the transaction system itself
would try to detect client failure.  Sure, there is always the problem
that one can't reliably detect some kinds client failure.  But if you
roll-back the client's transaction when you think it has died, then
getting it wrong occasionally is often acceptable.

-- Steve

In article <jQ9n1.106459$zu1.12655...@news.rdc1.bc.wave.home.com>,

I agree that it would be very nice if CORBA was specified so that
recursive method invocation never blocked.  I'm not sure of the
implications for ORB implementations though.  Would this mean that
the ORB had to be internally multi-threaded?

More broadly, it would be nice if the concurrency behaviour of a CORBA
object was specifiable in the IDL "contract".  This would make it
easier to develop portable clients, and maybe would make it feasible
to do static and dynamic deadlock detection.

The big problem is that both of these would be high impact changes
on the current CORBA world.

-- Steve

Or, to paraphrase:

        Once you have more than a few clients, the server spends most
        of its time calling back instead of doing real work ;-)

The fact is that the callback pattern doesn't scale, and you don't need
many clients to encounter that problem. Once you are up around ten clients,
the performance degradation becomes very noticable.

At the end of my original post, I suggested that many of the callback
problems can be solved by using the event service. The "mediator" you
talk about is the event service in that case. Big advantage is that I
don't have to build it myself.

I agree -- if the server maintains per-client state, it needs to address
how to clean up that state. However, the callback pattern makes this
harder to do than an event channel.

As I said, the role of the mediator can be taken by an event service.

Sure. Given enough effort, I can do just about anything. However, the whole
point is that CORBA is supposed to get me away from things like poll().

Yes, oneway can be used to get around this. But it is just as ORB-dependent
as is whether recursive callbacks deadlock or not. Personally, I feel
that the CORBA spec should be tighten to require that recursive callbacks
should not deadlock, simply because otherwise, location transparency is
broken.

In the normal callback pattern, the server stores an object reference to
an object in the client. That is tight coupling, full stop. If you want
weaker coupling, you need to use your mediator or an event channel.

Right. I've become very wary of the callback pattern, however. It is
universally tossed around as the standard solution to sending a conceptually
asynchronous notification to a client. However, the people who advocate
callbacks almost never mention the problems associated with it. Implementing
a reliable and scalable mechanism for callbacks is very difficult. And by
"scalable", I don't mean hundreds of clients to be called back. A dozen
is easily enough to require major implementation effort if reliability
is required in the presence of clients that can be slow or can crash.

I would like to go even further. The callback pattern is evil in many
circumstances in the non-distributed case too. Think about X11, which is
completely callback-driven. Why? Because a callback is a poor man's thread.

Programs that rely extensively on callbacks typically end up being a total
mess. Why? Because in a single-threaded implementation, I have to drop
back to the event loop to receive a callback. However, that forces me
to store all data that associates the state that present on the initial
in global variables, so I can reconstruct that state when the corresponding
callback arrives. The end result is that callbacks lead to programs that
are essentially a big state machine where all state is kept in global
variables or on the heap.

Sure, you can use encapsulation techniques to make all this a bit more
palatable, but the basic fact remains that callbacks in single-threaded
programs lead to global state.

As soon as I use threads instead, many of these problems disappear. In
particular, state can be localized in each thread. Of course, the price
I pay is that I need to pass messages between threads, but that I can
achieve a lot cleaner than I can with callbacks.

Sure. However, reliable callback is hard to implement and has a severe
performance penalty for even small numbers of clients.

And if you use an event channel instead of callbacks, none of these issues
every arise. Seems easier to me.

Which have to be threaded themselves to be reliable.

Exactly. And that is where callback raises even more problems. Let's face
it: callbacks are attractive when I am restricted to a synchronous RPC
mechanism like CORBA, but really want to do asynchronous things. Callbacks
suffer from the fact that true asynchronous behavior is difficult to
simulate with only synchronous calls (unless I make substantial effort).

Ultimately, the problem appears to be the lack of asynchronous messaging --
hopefully the Asynch Messaging spec will come to the rescue eventually.

                                                        Cheers,

                                                                Michi.
Copyright 1998 Michi Henning. All rights reserved.
--
Michi Henning              +61 7 33654310
DSTC Pty Ltd               +61 7 33654311 (fax)
University of Qld 4072     mi...@dstc.edu.au
AUSTRALIA                  http://www.dstc.edu.au/BDU/staff/michi-henning.html

In fact, none.

It seems unlikely. An RFP once reached draft stage (that is, the Request
For Proposals was drafted, not any proposals), but I guess someone heard
that distributed garbage collection is hard.

No standard -> no (effective) interoperability.

You have hit a hard problem in distributed computing. There is no
general solution to this. Even in the more restricted case of CORBA's
object model, there is no general solution.

By way of some food for thought, what if a client hands an object
reference to another client, how does the server know?

What if the client prints its stringified form on paper, inserts it into
a bottle corks it and casts it adrift in the ocean, only to retrieve it
months later (and OCR's it and destringifies it)? What if some human has
intercepted the paper form and photocopied it? (This example is not
original, my apologies to the originator, I've forgotten who you are.)

The example is ridiculous, but it alludes to a more relevant point:
CORBA object references are pieces of data which aren't tracked (in the
sense that copies can be made without notice being given to the creator
of the object reference), so there is ultimately no way for a server to
know that no copies of the object reference still exist. On this basis
alone, the consequence is that, within CORBA, completely automated
distributed garbage collection is infeasible.

You could take a far less general object model and provide distributed
garbage collection over it, but you would no longer be using CORBA.

- Raz

In article <35A001E7.13C83...@arrakis.com.au>,
Roland Turner  <r...@arrakis.com.au> wrote:

It came from the "twisted mind" of Michi Henning.  [Insiders joke :-)]

I think this is far too pessimistic an assessment.

There has been a lot of research over a number of years into scalable
algorithms for distributed garbage collection.  A number of solutions
have been found with different properties and assumptions.  For
example, here is a set of assumptions that allows a solution:

  1) Every client or service process is capable of managing object
references locally; e.g. it has a competent local garbage collector.
(This must apply to both memory resident references and references
saved to stable store.)

  2) All passing of object references between processes must be done
in a way that is visible to the ORB runtime; i.e. no passing of IORs
embedded in strings, etc.

Under these assumptions, a number of published distributed garbage
collection algorithms will work.

Now, I acknowledge that current CORBA ORBs and applications do not fit
these assumptions.  Furthermore, in ORBs for C, C++ and some other
languages, they are hard to meet.  I also acknowledge that unless you
forbid Object::string_to_object(), assumption 2) depends on user code
behaving properly.  Thus garbage collection is never likely to work
for all CORBA-based systems.

However, if one restricts the set of implementation languages, it is
quite feasible to build a CORBA compliant ORB in which supports
garbage collection.  ILU is an example, though I'm not familiar with
the way it does it.

-- Steve

It was me. I routinely store my object references in the Pacific Ocean ;-)

Look for the October issue of CACM (special issue on CORBA). I've mentioned
the Pacific Ocean Problem there.

                                                        Cheers,

                                                                Michi.
Copyright 1998 Michi Henning. All rights reserved.
--
Michi Henning              +61 7 33654310
DSTC Pty Ltd               +61 7 33654311 (fax)
University of Qld 4072     mi...@dstc.edu.au
AUSTRALIA                  http://www.dstc.edu.au/BDU/staff/michi-henning.html

Two things spring to mind:

(1) The server can't open a connection to the client because the client
    does not have a listener (and can't have one due to firewall
    restrictions).

(2) The client's network connection has just died but the user is in
    the process of restoring it, after which they will continue using
    the object.