Google Groups

Re: [silverstripe-dev] Database Selection for DataObjects and derivatives


Marcus Nyeholt May 2, 2012 10:35 PM
Posted in group: SilverStripe Core Development
Ah yes - I should have qualified my reply in context of 2.4, not 3.0, which I don't know as well internally.


There's probably a school of research out there about managing data joins across disparate data sets, which would probably be quite suited to a map/reduce type solution. Either way, not a trivial solution

Cheers,

Marcus

On Thu, May 3, 2012 at 3:19 PM, Sam Minnée <s...@silverstripe.com> wrote:

On 3/05/2012, at 2:05 PM, Marcus Nyeholt wrote:

Hi Guus,


At the moment most of the core of SS is very much single DB connection minded; most core classes uses the DB:: static method calls for database interaction, which all use a 'default' database connection. While it might be possible to manage the swapping around of what the 'defaut' connection is externally from the core, I'd imagine that it would only be successful for a limited number of scenarios. So the first hurdle to overcome would be to change core classes to use either instance methods, or make the DB:: static methods accept a different db connection identifier, either of which is non-trivial.

DataList could potentially have a dbConn property; this could be set to DB::getConn(), or to something else, with DI.
DataList::toArray() would need to $this->dbConn->query($this->dataQuery->sql().

The more significant change is to ensure that direct SQLQuery construction isn't used in the rest of core, but that should be achievable and is a worthwhile bit of housekeeping apart from this change.

A second big problem with using different DB connections per object type is that the ORM can no longer manage related data properly (through $has_one, $many_many etc) as it'd be trying to join across databases, and if the type was inherited, all types in the hierarchy would need to be stored in the same DB as joins occur when loading inherited types. While there are ways around all of this (doing joins in code, etc), it's not ideal.

The most straightforward solution would be to throw exception when you joined across different databases.  This would let you make use of multiple databases as long as you managed the partitioning yourself.  There are many situations where you might have a coherent but loosely coupled set of models running on another database, and being forced not to run joins across multiple databases would be an acceptable limitation.

Joining in PHP would be helpful in other contexts (e.g. linking a relational database to some kind of web API), but definitely a lot more work.  I would be inclined to approach its implementation from the perspective of "how can we execute these joins using only the SS_List API?"

However, back to the actual problem that it looks like you're trying to solve, because it's one I've frequently come across myself and have built an external content connector to help out https://github.com/nyeholt/silverstripe-external-content 

The idea is that you write an adaptor for the external content source, and a wrapper (or wrappers if needbe) for the external content items. The ExternalContentItem is a DataObject subclass, so can be used within SS much as you would use a data object. There's also mechanisms for importing the objects into a native SS data object representation - for example bringing through Wordpress content into SilverStripe. You could do something similar to wrap your external data sources

Some example connectors to get an idea of what's involved


Totally agree. :-)

--
You received this message because you are subscribed to the Google Groups "SilverStripe Core Development" group.
To post to this group, send email to silverst...@googlegroups.com.
To unsubscribe from this group, send email to silverstripe-d...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/silverstripe-dev?hl=en.