[tao-users] CORBA IDL versioning, evolution, backward compatibility
Malcolm McRoberts
complex data model which is primarily defined in CORBA IDL.
As our architecture evolves this IDL will change. We have a wide
range of software providers developing components and gateways
based on our IDL specification. Of course when the IDL changes
these folks have to rebuild all their code. We've tried to isolate
the interfaces, but when data types in the IDL change, it tends to
impact nearly everything (even if a component doesn't use the
type that changed). In general, there's no easy way to tell which
components are impacted by any change.
We could isolate our interfaces from some data type changes by
using "Any"s, but they are very expensive in terms of size and
processing requirements.
At some point, we may have a component that we need to run
but hasn't been upgraded to the IDL that we're using in the rest
of the system.
Does anybody know of a good approach to dealing with this issue?
In theory, we could write adapters that would allow us to provide
backward support for older components, but practically, I'm not
sure how you'd construct such a beast. Can I even link in stubs
for 2 versions of the same IDL?
I don't know if OBV would help, but since we're a mult-language
environment, we've chosen use only CORBA 2.3 features.
-malcolm
--
+-----+
|SPEED| "It's not just a good idea, it's the law"
| | - Albert Einstein (paraphrased)
| c |
| |
|LIMIT|
+-----+
Dale Wilson
Malcolm McRoberts wrote:
> I'm working on a large component-based architecture. We have a
> complex data model which is primarily defined in CORBA IDL.
> As our architecture evolves this IDL will change.
1) Add an additional argument to selected methods that takes a sequence of name/value pairs. In my case we used strings as the values rather than Any's to avoid the overhead you mention. This gives you some flexibility in adding "small tweaks" to the method's signature. The rule-of-thumb is that servers should ignore unknown names, and should provide appropriate default values for missing names. This should only be used for small changes -- ie.. if you end up passing everything through the name/value system, it's time to define a new method.
2) Add an extra attribute to every interface. In our case we used:
readonly attribute unsigned long capabilities;
This capabilities word was treated as a bit field. Each bit corresponds to a feature that might be implemented by the interface. (Bits were defined in a shared header file.)
One bit in the capabilities word is reserved to mean "capabilities2 (another attribute) is available from this interface". That allows the technique to be extended "forever."
Rules for servers:
- Undefined capability bits are zero. Defined bits are one, only if the feature is implemented.
- Once a method is public (i.e. released into the wild) it is cast in stone. It will stay in the interface "forever" and it's signature will never change.
- The same rule applies to structures, enums, etc. Once it's out there you live with it forever.
- It's ok to add new methods to an interface (I'm not sure that's
kosher according to OMG, but it works.) as long as you define a
capability bit for it.
- If you really have to rework a method, use a new name (i.e. niftyFunction becomes niftyFunction2) leaving the original name and signature untouched. You should, of course, define a new capabilities bit, too..
- Do your best to implement the old methods, but if this is not possible. provide a stub implementation (or throw an exception depending on what makes sense.)
Rules for clients:
- Retrieve and cache the capabilities attribute.
- For any "optional" or "new-improved" feature clients MUST test the appropriate capabilities bit before attempting to use the feature.
- Clients should move to the new-improved version as soon as
feasable. Avoid "hanging on" to old, obsolete methods (see definition
of "forever")
- If a capability bit is zero, the client must "do the right thing." This might be to tell the user, or might be to use an older, less desirable approach to achieving the desired functionality.
HTH,
Dale
We have a wide
range of software providers developing components and gateways
based on our IDL specification. Of course when the IDL changes
these folks have to rebuild all their code. We've tried to isolate
the interfaces, but when data types in the IDL change, it tends to
impact nearly everything (even if a component doesn't use the
type that changed). In general, there's no easy way to tell which
components are impacted by any change.
We could isolate our interfaces from some data type changes by
using "Any"s, but they are very expensive in terms of size and
processing requirements.
At some point, we may have a component that we need to run
but hasn't been upgraded to the IDL that we're using in the rest
of the system.
Does anybody know of a good approach to dealing with this issue?
In theory, we could write adapters that would allow us to provide
backward support for older components, but practically, I'm not
sure how you'd construct such a beast. Can I even link in stubs
for 2 versions of the same IDL?
I don't know if OBV would help, but since we're a mult-language
environment, we've chosen use only CORBA 2.3 features.
-malcolm
-- ----------------------------------------------------- Dale Wilson, Senior Software Engineer Object Computing, Inc. (OCI) http://www.ociweb.com/ http://www.theaceorb.com/ ----------------------------------------------------
Malcolm McRoberts
Thanks for all the suggestions. We have move a lot of our more volitile
data into a name-value pair scheme.
Our values are a union of primitive types rather than a plain string to
reduce overhead. We use XML
documents to define the ones in use at a given time (we can add a new
one by just updating the
XML). I wish we'd done more of this. As you pointed out, adapting
name-value data on the fly
to be compatible with newer software is not too hard.
It's clear that if you don't consider the brittleness of IDL-based
architectures early on, then you're
in for a lot of pain down the road.
We've also got some complex, compound data types that are hard to do
this way. Those are the ones
that really concern me. If a data type changes, then every other data
that uses it, and every interface
that reference it or data types that use it are impacted.
It's certainly true that clients don't require code change if you just
add new operations to interfaces.
I'm pretty sure that you do need to at least regen the stubs, recompile
and relink. That would work
well for components delivered as source with a documented build
process. I don't think it'll support
binary components.
I like your idea of keeping the old operations as is, and just adding
the new one with a new name.
COM does this at the interface level.
What I'd really like is an adaptor framework that would accept calls
from clients using IDL 1.0
and forward these calls to servers using IDL 1.1 (and vice versa), and
would provide default
handling for those parts that didn't change.
Relational databases and XML-based systems seem to have good ways to
deal with these
issues. One would hope that the OMG is doing some work in this area.
-malcolm
Dale Wilson wrote:
> * Undefined capability bits are zero. Defined bits are one, only
> if the feature is implemented.
> * Once a method is public (i.e. released into the wild) it is cast
> in stone. It will stay in the interface "forever" and it's
> signature will never change.
> * The same rule applies to structures, enums, etc. Once it's out
> there you live with it forever.
> * It's ok to add new methods to an interface (I'm not sure that's
> kosher according to OMG, but it works.) as long as you define a
> capability bit for it.
> * If you really have to rework a method, use a new name (i.e.
> niftyFunction becomes niftyFunction2) leaving the original name
> and signature untouched. You should, of course, define a new
> capabilities bit, too..
> * Do your best to implement the old methods, but if this is not
> possible. provide a stub implementation (or throw an exception
> depending on what makes sense.)
>
> [Note: "forever" is defined as until you can be absolutely sure no
> antique clients or servers are still around. In your case it might
> mean "forever."]
>
> Rules for clients:
>
> * Retrieve and cache the capabilities attribute.
> * For any "optional" or "new-improved" feature clients MUST test
> the appropriate capabilities bit before attempting to use the
> feature.
> * Clients should move to the new-improved version as soon as
> feasable. Avoid "hanging on" to old, obsolete methods (see
> definition of "forever")
> * If a capability bit is zero, the client must "do the right
Dale Wilson
> Dale,
>
<SNIP>
> It's certainly true that clients don't require code change if you just
> add new operations to interfaces.
> I'm pretty sure that you do need to at least regen the stubs,
> recompile and relink. That would work
> well for components delivered as source with a documented build
> process. I don't think it'll support
> binary components.
Actually you can get by without rebuilding old clients. It's a hack
that wouldn't pass muster with OMG. However it *does* work [which
counts for a lot :-) ]. Corba metthods are identified by name (text
string) in IIOP (other protocols -- particularly colocated
client/servers -- may do it differently) so old clients can still
call-by-name the same methods on updated interfaces as long as the
"cast-in-stone" rules are followed.
In any case, I agree that this is a serious issue that needs a more
systemic solution. I would love to never use this technique again.
Dale
Dale Wilson
Dale Wilson wrote:Malcolm McRoberts wrote:Dale,<SNIP>It's certainly true that clients don't require code change if you just add new operations to interfaces. I'm pretty sure that you do need to at least regen the stubs, recompile and relink. That would work well for components delivered as source with a documented build process. I don't think it'll support binary components.Actually you can get by without rebuilding old clients. It's a hack that wouldn't pass muster with OMG. However it *does* work [which counts for a lot :-) ]. Corba metthods are identified by name (text string) in IIOP (other protocols -- particularly colocated client/servers -- may do it differently) so old clients can still call-by-name the same methods on updated interfaces as long as the "cast-in-stone" rules are followed.
Yes, it does. The same method name will find the correct method in the new hash.Does this method work even if "perfect hashing" is used when stubs were generated ?
Dale
Milan.
In any case, I agree that this is a serious issue that needs a more systemic solution. I would love to never use this technique again. Dale
Milan Cvetkovic
> Malcolm McRoberts wrote:
>
>> Dale,
>>
> <SNIP>
>
>> It's certainly true that clients don't require code change if you just
>> add new operations to interfaces.
>> I'm pretty sure that you do need to at least regen the stubs,
>> recompile and relink. That would work
>> well for components delivered as source with a documented build
>> process. I don't think it'll support
>> binary components.
>
>
> Actually you can get by without rebuilding old clients. It's a hack
> that wouldn't pass muster with OMG. However it *does* work [which
> counts for a lot :-) ]. Corba metthods are identified by name (text
> string) in IIOP (other protocols -- particularly colocated
> client/servers -- may do it differently) so old clients can still
> call-by-name the same methods on updated interfaces as long as the
> "cast-in-stone" rules are followed.
Does this method work even if "perfect hashing" is used when stubs were
generated ?
Milan.