New Beginnings

[Timothy Perrett]

Hey All,

So I saw that no one had yet started any discussions and I couldn't
resist starting something ;-)

I read DPPs very interesting blog post about the launch of Goat Rodeo,
and I was left with some questions... Maybe its just me, but I like to
work back from a group of use cases and provide a solution; its not
clear to me (yet) what the possible use cases of Goat Rodeo are? The
post talks about social networks / applications in relatively vague
terms and IMHO, whilst thats great, its perhaps not an applicable
scenario for a lot of developers and it would be great to have
something more high level that described what Goat Rodeo could in
theory be used for.

The more people you can deliver a concrete understanding too the
speedier adoption will be!

Cheers, Tim

[David Pollak]

I'm planning to write (yet another) Twitter clone with Goat Rodeo as the business logic tier rather than having that tier as part of the web tier.  Do you think that would help with understanding?

--
You received this message because you are subscribed to the Google Groups "Goat Rodeo" group.
To post to this group, send email to goat-...@googlegroups.com.
To unsubscribe from this group, send email to goat-rodeo+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/goat-rodeo?hl=en.

--
Lift, the simply functional web framework http://liftweb.net
Beginning Scala http://www.apress.com/book/view/1430219890
Follow me: http://twitter.com/dpp
Surf the harmonics

[Josh Suereth]

Yes, the key is to give enough insight into how a complex system is built.

Sent from my iPhone

[David Pollak]

So, I've got my first example Worker up in a Skittr subdirectory on Assembla.  It's (surprise surprise) a Twitter clone:

final case class UserId(id: Long) extends WorkerId {
  type MyType = UserId
  type MsgType = UserMsg
  type IdType = Long

  def myType: Manifest[MyType] = manifest[UserId]

  def uniqueNameForFile: String = "user_info_"+id
}

sealed trait UserMsg extends QBase

/**
 * Sent to a User telling the user to follow another user
 */
final case class Follow(who: UserId) extends UserMsg

/**
 * Sent from one user to another to inform the recipient that
 * you want to follow them
 */
final case class AddMeAsAFollower(from: UserId) extends UserMsg

/**
 * Sent to a User telling the user to unfollow another user
 */
final case class Unfollow(who: UserId) extends UserMsg

/**
 * Sent from one user to another to inform the recipient that
 * you want to un follow them
 */
final case class RemoveMeAsAFollower(from: UserId) extends UserMsg


/**
 * The user has posted an update.  Reflect that in the users
 * updates and propgrate the update to followers
 */
final case class PostUpdate(updateId: Long, time: Long) extends UserMsg

/**
 * Sent from the updating User to all its followers
 */
final case class AddUpdateToTimeline(updateId: Long,
                     from: UserId,
                     time: Long) extends UserMsg

final case class ListOfUsers(users: List[UserId]) extends QBase

final case class GetFollowers() extends UserMsg with
MsgWithResponse[ListOfUsers]

final case class GetFriends() extends UserMsg with
MsgWithResponse[ListOfUsers]

final case class MessageInfo(updateId: Long) extends QBase
final case class ListOfMessages(msgs: List[MessageInfo]) extends QBase

final case class GetTimeline(first: Int, count: Int) extends UserMsg with
MsgWithResponse[ListOfMessages]

final case class GetUpdates(first: Int, count: Int) extends UserMsg with
MsgWithResponse[ListOfMessages]


class UserWorker(id: UserId, calcFunc: UserId => ConnectionManager) extends
WorkerImpl[UserId, UserMsg](id, calcFunc) {
  private var timeline: Array[Long] = Array()
  private var updates: Array[Long] = Array()

  override protected[this] def workerCreated() {
    super.workerCreated()
   
    def nullLogFunc(a: => AnyRef) {}

    Schemifier.schemify(true, nullLogFunc _, Relationship, Message)
  }

  override protected[this] def workerUnfrozen() {
    timeline = Message.findAll(OrderBy(Message.postTime, Descending),
                   MaxRows(20)).map(_.messageId.is).toArray

    updates = Message.findAll(By(Message.postedBy, id.id),
                  OrderBy(Message.postTime, Descending),
                  MaxRows(20)).map(_.messageId.is).toArray
  }

  def doFollow(msg: Follow) {
    if (Relationship.find(By(Relationship.who, msg.who.id),
                          By(Relationship.isFollowing, true)).isEmpty) {
      Relationship.create.who(msg.who.id).isFollowing(true).save
    }
   
    for {
      other <- WorkerMgr.find(msg.who)
    } other ! AddMeAsAFollower(id)
  }

  def doAddAFollower(msg: AddMeAsAFollower) {
    if (Relationship.find(By(Relationship.who, msg.from.id),
                          By(Relationship.isFollowing, false)).isEmpty) {
      Relationship.create.who(msg.from.id).isFollowing(false).save
    }
  }

  def doUnfollow(msg: Unfollow) {
    Relationship.findAll(By(Relationship.who, msg.who.id),
             By(Relationship.isFollowing, true)).foreach(_.delete_!)

    for (other <- WorkerMgr.find(msg.who)) other ! RemoveMeAsAFollower(id)
  }

  def doRemoveAFollower(msg: RemoveMeAsAFollower) {
    Relationship.findAll(By(Relationship.who, msg.from.id),
             By(Relationship.isFollowing, false)).foreach(_.delete_!)
  }

  def doUpdate(msg: PostUpdate) {
    updates = (msg.updateId :: updates.toList).take(20).toArray
   
    val toSend = AddUpdateToTimeline(msg.updateId, id, msg.time)

    this ! toSend

    Relationship.findMap(By(Relationship.isFollowing, true)) {
      row =>
      for (other <- WorkerMgr.find(UserId(row.who))) other ! toSend

      Empty
    }
  }

  def doAddUpdate(msg: AddUpdateToTimeline) {
    timeline = (msg.updateId :: timeline.toList).toArray

    Message.create.messageId(msg.updateId).
    postTime(msg.time).postedBy(msg.from.id).save
  }

  def doGetFollowers(msg: GetFollowers): ListOfUsers =
    getRelated(false)

  private def getRelated(isFollowing: Boolean): ListOfUsers =
    ListOfUsers(
      Relationship.findMap(By(Relationship.isFollowing, isFollowing)) {
    row => Full(UserId(row.who))
      })

  def doGetFriends(msg: GetFriends): ListOfUsers =
    getRelated(true)

  private def getMessages(mine: Boolean,
                          first: Int, count: Int): ListOfMessages = {
    val set = if (mine) updates else timeline

    if (first == 0 && count <= set.length)
      ListOfMessages(set.toList.take(count).map(MessageInfo.apply))
    else {
      val qp = List[QueryParam[Message]](OrderBy(Message.postTime, Descending),
                                         MaxRows(count),
                                         StartAt(first))

      ListOfMessages(Message.findMap(
          (if (mine) List(By(Message.postedBy, id.id)) else Nil) ::: qp :_*) {
      row =>
          Full(MessageInfo(row.messageId))
    })
    }
  }

  def doGetTimeline(msg: GetTimeline): ListOfMessages =
    getMessages(false, msg.first, msg.count)

  def doGetUpdates(msg: GetUpdates): ListOfMessages =
    getMessages(true, msg.first, msg.count)
}

class Relationship extends LongKeyedMapper[Relationship] with IdPK {
  def getSingleton = Relationship

  object who extends MappedLong(this) {
    override def dbIndexed_? = true
  }

  object isFollowing extends MappedBoolean(this)
}

object Relationship extends Relationship with
LongKeyedMetaMapper[Relationship]

class Message extends LongKeyedMapper[Message] with IdPK {
  def getSingleton = Message

  object messageId extends MappedLong(this)
  object postTime extends MappedLong(this)
  object postedBy extends MappedLong(this)
}

object Message extends Message with LongKeyedMetaMapper[Message] {
  private def createFunc(table: String,cols: List[String]): String =
    "CREATE INDEX msg_time_desc ON "+table+" ("+cols.head+" DESC)"

  override def dbIndexes = GenericIndex(createFunc _,
                    IHaveValidatedThisSQL("dpp",
                                  "2010/02/16"),
                    postTime) :: super.dbIndexes
}

The key take-aways from the code are:

• Each worker has its own SQL store
• There's no remoting or other explicit distribution-related calls in the business logic code
• The Worker caches the most frequently accessed information (the timeline).  This means no DB hit for that information.  I think this is the highest and best use of Goat Rodeo, pre-calculating stuff (e.g., the stuff you'd put in memcached), but deal with the stuff in an isolated, transactions way