Social network design

[Aran Mulholland]

I have been messing around with social network design. One of the things that I have taken a long time getting right is linked list insertion. For any feed based social network the feed query is the most important query in the system. This query will be executed the most and needs to be performant. The method I thought of pursuing was to have the feed in a linked list hanging off every user so in order to fetch the latest stories for the user you could do a query like

MATCH (user)-[:FEED*]->(story)

to return the feed for a user.

However in order to do linked list insertion you need to take a lock. The lock worries me for the following reason. 

When (User A) posts a story you will need to take a lock on every follower of (User A) in order to insert this story (or a link to it) in the followers feed. As you have to take a lock on every follower there is a high possibility that someone else that they are following may be posting at the same time leading to possible deadlock. This possibility of deadlock increases when the number of followers increases.

There are multiple ways to implement news feeds, there is another here -  http://docs.neo4j.org/chunked/snapshot/cypher-cookbook-newsfeed.html you will notice that this query would span multiple relationship types and have to do an order-by as well. The data model for it is a lot cleaner though.

I am wondering if there are people who have implemented news feed applications who would be willing to share some of the ways they have implemented this feature and the associated data structure that has been used.

[Mahesh Lal]

Hi Aran

On one of my projects we designed a simple social network using Neo4J.

The user posts used to be linked to whichever user has first posted. Comments on the post were a linked list. When a user went to his wall, we used to simply pull posts from all his friends and the comments on those posts and display.

Does this help? Need more details? Find it inefficient? Feedback welcome.

--
You received this message because you are subscribed to the Google Groups "Neo4j" group.
To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Aran Mulholland]

I suppose I would like to know:

Was it efficient? 

How many users did you end up getting using your social network?

I have played with linked lists in neo4j (there are a few questions I have asked on this list if you are interested) In order to get parallel insertion when more than one user comments at the same time and not have your linked list be messed up you need to take locks. Did you have to manage taking locks when you did inserts in the comments list?

I did some tests on insertingin in parallel here -  https://github.com/aranm/neo4j-linked-list if you are interested

[Mahesh Lal]

We didn't bother taking locks on the database since we knew that the user load wasn't going to be much. Last I've heard, the website was a failure with just approximately eighty thousand users. But the reasons for failure was the business model and not the technology.

It was more or less a first pass after which the client decided to take the project in house and further develop it (which never really happened)

As far as efficiency goes, it had an acceptable response time with the arbitrary depth query (how deep should the query search for). We used to show 10 comments at a time fetched from the linked list. In terms of pure numbers, I dont know how performant it was.