Interoperability of different Elixir-using codebases
Vinay Sajip
need to be integrated at deployment time to work together. Each component
potentially has data persisted using Elixir. Each deployment will consist of a
different, essentially arbitrary set of components which need to work together.
Since the components are independently developed (by third parties) there is the
possibility of database name clashes (since component developers may have used
non-default table naming). Making changes to the source code for these
components is not an option.
To achieve component interoperability, we need control over the database table
naming, to happen at a time later than import time, so that we can define table
names unambiguously across components - overriding any table names which may
have been specified in the entity sources.
We also need to be able to configure the naming for each component in a specific
way, i.e. operate on all entities in component A using strategy (or
configuration data) specific to A, then on all entities in component B using
strategy (or configuration data) specific to B, etc.
What's the best way to do this? Would it work to import all the entities in all
the components, change the EntityClass._descriptor.tablename, then do a
setup_all()? Would this handle relationships correctly? And what's an easy way
to distinguish the entities for each component, perhaps to get them into
separate collections? By default they would all end up in elixir.entities. I see
there's a __collection__ - which ISTM can't be used in our case, as we can't
change the component sources - but I couldn't find enough information on it to
be sure of this, and it seems little discussed in comparison with __session__
and __metadata__. Is it me or do those seem a bit hackish? (No offence intended.)
Thanks in advance for any help,
Vinay Sajip
P.S. Elixir does a great job, but sometimes it would be nice to get an idea of
what's happening behind the scenes, as it happens, on a specific project. In
other systems I've turned logging verbosity up to get an idea of what happens
when, but Elixir doesn't use logging. Is there any particular reason for this?
I'm especially interested, as I'm the author of Python's logging package :-)
Gaetan de Menten
>
> I'm working on a project where a number of components, developed independently,
> need to be integrated at deployment time to work together. Each component
> potentially has data persisted using Elixir. Each deployment will consist of a
> different, essentially arbitrary set of components which need to work together.
>
> Since the components are independently developed (by third parties) there is the
> possibility of database name clashes (since component developers may have used
> non-default table naming). Making changes to the source code for these
> components is not an option.
>
> To achieve component interoperability, we need control over the database table
> naming, to happen at a time later than import time, so that we can define table
> names unambiguously across components - overriding any table names which may
> have been specified in the entity sources.
>
> We also need to be able to configure the naming for each component in a specific
> way, i.e. operate on all entities in component A using strategy (or
> configuration data) specific to A, then on all entities in component B using
> strategy (or configuration data) specific to B, etc.
>
> What's the best way to do this? Would it work to import all the entities in all
> the components, change the EntityClass._descriptor.tablename, then do a
> setup_all()?
That's kinda ugly, but it should work as far as the tables are concerned.
> Would this handle relationships correctly?
This depends on several things: which version of Elixir you are using,
whether or not the individual components use the same class names
(within their own module), and whether or not the relationships use
"absolute paths" to other entities. If they don't use absolute paths,
and some of them use the same class name, you'll get clashes within
the global collection name-to-class map.
> And what's an easy way
> to distinguish the entities for each component,
I guess they are defined in different modules so entity.__module__ should do it.
> perhaps to get them into separate collections? By default they would all end up in elixir.entities. I see
> there's a __collection__ - which ISTM can't be used in our case, as we can't
> change the component sources
That's indeed the case. __collection__ is only useful when declared
within the module where the entities are (and before they are
declared).
> - but I couldn't find enough information on it to
> be sure of this, and it seems little discussed in comparison with __session__
> and __metadata__. Is it me or do those seem a bit hackish? (No offence intended.)
Depends what you call hackish. I don't see any problem with those. And
__collection__ is indeed much less discussed because AFAIK nobody
except me has ever had a need for them.
Now as for solving your potential relationship problem (ie class name
clash), you have several options: if the components are totally
independent of each other (there is no database relationship between
their tables), you could use different collections (as for the tables,
you could go through all entities and remove them from the global
entities collection, then add them to a specific collection for that
component, and update their entity._descriptor.collection pointer.
Otherwise, you can use one custom collection for all your entities and
change the lookup mechanism (eg it looks for the target entity only
within the current component module, except if an absolute path is
given) so that there is no name clash.
> P.S. Elixir does a great job, but sometimes it would be nice to get an idea of
> what's happening behind the scenes, as it happens, on a specific project. In
> other systems I've turned logging verbosity up to get an idea of what happens
> when, but Elixir doesn't use logging. Is there any particular reason for this?
> I'm especially interested, as I'm the author of Python's logging package :-)
I've started to document what happens behind the scene at:
http://elixir.ematia.de/trac/wiki/BehindTheScene
but never took the time to finish it.
And there is no particular reason Elixir doesn't do any logging,
except lack of time. It's been on my wishlist for ages (see ticket #65
and #66) but I haven't had the time (or motivation) to do it yet. A
patch is of course welcome...
Hope it helps,
--
Gaëtan de Menten
http://openhex.org
Vinay Sajip
> That's kinda ugly, but it should work as far as the tables are concerned.
Yes, but I couldn't see a less ugly way :-)
> > Would this handle relationships correctly?
>
> This depends on several things: which version of Elixir you are using,
> whether or not the individual components use the same class names
> (within their own module), and whether or not the relationships use
> "absolute paths" to other entities. If they don't use absolute paths,
> and some of them use the same class name, you'll get clashes within
> the global collection name-to-class map.
Hmm - you've given me food for thought. We'll be using Elixir 0.7 or later.
> Depends what you call hackish. I don't see any problem with those. And
> __collection__ is indeed much less discussed because AFAIK nobody
> except me has ever had a need for them.
Fair enough - I thought the lack of mention might be due to those things being
deprecated ...
> Now as for solving your potential relationship problem (ie class name
> clash), you have several options: if the components are totally
> independent of each other (there is no database relationship between
> their tables), you could use different collections (as for the tables,
> you could go through all entities and remove them from the global
> entities collection, then add them to a specific collection for that
> component, and update their entity._descriptor.collection pointer.
>
> Otherwise, you can use one custom collection for all your entities and
> change the lookup mechanism (eg it looks for the target entity only
> within the current component module, except if an absolute path is
> given) so that there is no name clash.
Thanks, that's helpful. I will investigate this approach further. When you say
"change the lookup mechanism", are there any specific hooks in Elixir I should
use?
> I've started to document what happens behind the scene at:
> http://elixir.ematia.de/trac/wiki/BehindTheScene
> but never took the time to finish it.
Yes, I've read that, but formed the impression that it might be out of date.
> And there is no particular reason Elixir doesn't do any logging,
> except lack of time. It's been on my wishlist for ages (see ticket #65
> and #66) but I haven't had the time (or motivation) to do it yet. A
> patch is of course welcome...
Okay, I might be able to help with that :-)
Regards, and thanks for the quick response.
Vinay Sajip
Gaetan de Menten
>
> Gaetan de Menten <gdementen@...> writes:
>> Now as for solving your potential relationship problem (ie class name
>> clash), you have several options: if the components are totally
>> independent of each other (there is no database relationship between
>> their tables), you could use different collections (as for the tables,
>> you could go through all entities and remove them from the global
>> entities collection, then add them to a specific collection for that
>> component, and update their entity._descriptor.collection pointer.
>>
>> Otherwise, you can use one custom collection for all your entities and
>> change the lookup mechanism (eg it looks for the target entity only
>> within the current component module, except if an absolute path is
>> given) so that there is no name clash.
>
> Thanks, that's helpful. I will investigate this approach further. When you say
> "change the lookup mechanism", are there any specific hooks in Elixir I should
> use?
Not really. You just provide your own collection modeled after
collection.EntityCollection, possibly inheriting from it, and you
override the resolve method, and possibly other methods as needed.
>> I've started to document what happens behind the scene at:
>> http://elixir.ematia.de/trac/wiki/BehindTheScene
>> but never took the time to finish it.
>
> Yes, I've read that, but formed the impression that it might be out of date.
Nope, the general principles are intact. AFAIK, the only out of date
information in there is that the attributes are not attribute as soon
as the property is attached to its class but later during the
setup_all phase.