Multi database

[Jonas]

In 0.2.0 version you have added some support for multi-database where
each model can live in a different database. But when you scale big
there are all kinds of ways things might need to be partitioned:

* Some data gets accessed all the time while some other data is only
ever written.
* You could want that certain records are in certain databases.
E.g. users 1 to 1,000,000 , or news/articles/information stored by
year.

More info and examples here:
http://code.djangoproject.com/ticket/1142
http://groups.google.com/group/django-users/browse_thread/thread/d08f4975da831bb/

[Jonas]

Another possible example with a multi-blog application. To use a
separate database for each blog "app" that you mount within the server.

[David Pollak]

Jonas,

It is possible for each model instance to be associated with a specific database as well.

Here's how it works:

• To select a record or records from a specific database, use findDb or findAllDb and pass in a connection identifier.  The connection identifier tells the lower level machinery which database to use.
• If you do no specify a connection identifier, the connection identifier returned by Model.dbDefaultConnectionIdentifier will be used.  Note upper case Model means the MetaMapper object rather than the Mapper class instance.
  
• If you create a new model instance, you can specify which Connection Identifier is associated with that instance (model.connectionIdentifier(foo) )
  

I'll add a little Partial Function magic to allow you to auto-select the database for instances to be saved and for primary key searches.

Thanks,

David

http://liftweb.net

[Jonas]

Impressive! I like more Lift every time and is that Lift/Scala look
very powerfull.

On 11 jun, 14:38, "David Pollak" <feeder.of.the.be...@gmail.com>
wrote:

> It is possible for each model instance to be associated with a specific
> database as well.
>

[John D. Heintz]

Wow, that sounds very interesting especially when coupled with Actors.
A huge opportunity for app scalability.

Have you read the paper by Pat Helland?
http://blogs.msdn.com/pathelland/archive/2007/05/16/link-to-life-beyond-distributed-transactions-an-apostate-s-opinion.aspx

In that paper he discusses never using distributed transaction, and
making each "entity" the unit of transactional scope. In order to
support correct behavior in the face of partial failures he proposes:
* "at-least-once" messaging
* Idempotent messages
* Recording message histories inside entities to support idempotence

The ideas in that paper could help influence the distributed and
partitioned capabilities of lift.

John Heintz

--
John D. Heintz
Principal Consultant
New Aspects of Software
Austin, TX
(512) 633-1198

[David Pollak]

BTW, I checked code in to make the automagic database selection happen.

For Model.find, in your meta-mapper object override:
def dbSelectDBConnectionForFind: PartialFunction[Type, ConnectionIdentifier]

and implement a partial function that maps from the primary key to a connection.  For example:

override def dbSelectDBConnectionForFind = {
  case n if n < 100000L => db1Connection
  case n if n < 200000L => db2Connection
}

Also, to choose what database to put a newly created instance, in your mapper class override:
def dbCalculateConnectionIdentifier: PartialFunction[A, ConnectionIdentifier]

eg:

override def dbCalculateConnectionIdentifier = {
  case i if i.email.charAt(0) < 'n' => dbAthroughNConnection
  case _ => dbOthroughZConnection
}

Thanks,

David

On 6/11/07, Jonas <jona...@googlemail.com> wrote:

--
lift, the fast, powerful, easy web framework
http://liftweb.net