Diaspora's architecture & development

[Robert Campbell]

A question to the diaspora devs.

How is diaspora developed? 

That's a vague question so let me dissolve it.

How do you decide which features to implement?

Is there a framework that you use for development? (I'm not asking about ROR here now)

I am asking if you have a well designed & documented system architecture that allows for simple extendability of the core product. For example there is a status/photo publisher. Is the system designed in a way so that people who decide to implement the software can easily integrate other publishers e.g: a video publisher, event publisher, poll publisher etc. I've checked the schema file in the source and the database does not seem to be modeled in a way that allows this kind of easy integration. So back to the question: Is there a framework that you use for development or do you just add things as you go and rewrite/modify the code base  to support it?

 This product is intended to be distributed. Is the system designed in such a way that other people can easily customize the view layer or is it "themable"?

The selling point of diaspora is "privacy" but so far the only layer of privacy I have seen so far is the "aspect". Is this meant to be the only layer of privacy for diaspora or are there plans to provide a more granular method for preserving privacy?

Is there a documented protocol for communicating with diaspora nodes/pods from other systems?

Are you satisfied with the current pace of development? There was so much hype about diaspora last year that I decided to learn ruby so I could be a part of it, but after the initial release and some negative feedback from certain "reporting houses", the flames just died.

I have taken a look(not thorough) at source and I must say that I am not impressed with the progress so far. From what I understand, you guys are so students (as am I); I understand you may have been busy with school so that may be the cause. 

I have other questions/concerns but I would appreciate if you guys could address these for the time being. Thanks in advance.

[Tom Scott]

Keep in mind that Facebook has been around for 7 years, and Diaspora's alpha release was...what, last year? It definitely needs some time to mature. Open-source software that's not in common use tends to stagnate a little longer than software people are being paid to develop, for obvious reasons. No one's getting paid to write Diaspora, and anybody who contributes probably gets paid to develop something else. There are very few coders who can program all day and then come home and do it all night. I know I'm certainly not one of those people, songs don't write themselves!

 

On Thursday, May 5, 2011 at 4:23 PM, Robert Campbell wrote:

A question to the diaspora devs.

How is diaspora developed? 

That's a vague question so let me dissolve it.

How do you decide which features to implement?

Well, if you feel like writing a feature and it works well with the software, it can be integrated into the main codebase on request. Basically, if you fork the project and then write your feature (and it works), you can submit a "Pull Request" which will subject your fork for review by the maintainers, and if they like the feature and it works properly, it will be integrated.

I am asking if you have a well designed & documented system architecture that allows for simple extendability of the core product. For example there is a status/photo publisher. Is the system designed in a way so that people who decide to implement the software can easily integrate other publishers e.g: a video publisher, event publisher, poll publisher etc. I've checked the schema file in the source and the database does not seem to be modeled in a way that allows this kind of easy integration. So back to the question: Is there a framework that you use for development or do you just add things as you go and rewrite/modify the code base  to support it?

Are you looking at the same schema? :) Photos are extensions of the Post class, which has a database. What's stopping you from creating a Video or Poll class which extends Post? The idea is that anything you post to your feed is a Post and should be treated equally. Obviously, to the server these different media types are handled differently, but regardless of what is contained in the post it's still a post and should be treated as such. This also makes it easier to sort through and display one's activity feed, which is definitely a central part of any social networking site.

Diaspora was built using behavior-driven development (BDD). It's a concept that's an offshoot of test-driven development (TDD), wherein you write tests for your application first, watch them fail, and then write just enough code to make them pass. By attacking each problem separately, and before it becomes a "bug", you end up writing more bulletproof software which can be easily maintained. The best part about BDD is when you want to add new features. By writing test-first, you can quickly see how your new code is affecting the rest of the application because you're only writing enough code to make your new feature's tests pass.

This product is intended to be distributed. Is the system designed in such a way that other people can easily customize the view layer or is it "themable"?

I wouldn't say it's "easy" to change Diaspora's look & feel, but I feel like theming is something that's can be implemented pretty quickly, considering projects like Typo have already mastered it. Although I think we have some more pressing matters to deal with right now than the UI. Of course, you should feel free to contribute such a feature... :)

The selling point of diaspora is "privacy" but so far the only layer of privacy I have seen so far is the "aspect". Is this meant to be the only layer of privacy for diaspora or are there plans to provide a more granular method for preserving privacy?

Is there a documented protocol for communicating with diaspora nodes/pods from other systems?

It's already implemented! If the Diaspora pod you're trying to connect to doesn't require an invite, you can easily sign in to that pod with your original Diaspora account details. This works like Twitter/Facebook connect, wherein you register using your existing account and the pod simply migrates the information into its own database. So technically, you're creating another account but since it's all "in the family" we can safely transfer lots of account details to get you immediately started on the new pod. This method was chosen because it's secure, fast and easy to understand. In addition, your pod is self-sufficient and doesn't ever need to "phone home" to some centralized user database. The point is decentralization, after all.   

Are you satisfied with the current pace of development? There was so much hype about diaspora last year that I decided to learn ruby so I could be a part of it, but after the initial release and some negative feedback from certain "reporting houses", the flames just died.

I have taken a look(not thorough) at source and I must say that I am not impressed with the progress so far. From what I understand, you guys are so students (as am I); I understand you may have been busy with school so that may be the cause. 

It sounds like you need to learn a little more about Rails, Ruby and behavior-driven development.

[Robert Campbell]

On Thursday, 5 May 2011 20:34:38 UTC-5, Tom Scott wrote:

Keep in mind that Facebook has been around for 7 years, and Diaspora's alpha release was...what, last year? It definitely needs some time to mature. Open-source software that's not in common use tends to stagnate a little longer than software people are being paid to develop, for obvious reasons. No one's getting paid to write Diaspora, and anybody who contributes probably gets paid to develop something else. There are very few coders who can program all day and then come home and do it all night. I know I'm certainly not one of those people, songs don't write themselves!

 

On Thursday, May 5, 2011 at 4:23 PM, Robert Campbell wrote:

A question to the diaspora devs.

How is diaspora developed? 

That's a vague question so let me dissolve it.

How do you decide which features to implement?

Well, if you feel like writing a feature and it works well with the software, it can be integrated into the main codebase on request. Basically, if you fork the project and then write your feature (and it works), you can submit a "Pull Request" which will subject your fork for review by the maintainers, and if they like the feature and it works properly, it will be integrated.

I am asking if you have a well designed & documented system architecture that allows for simple extendability of the core product. For example there is a status/photo publisher. Is the system designed in a way so that people who decide to implement the software can easily integrate other publishers e.g: a video publisher, event publisher, poll publisher etc. I've checked the schema file in the source and the database does not seem to be modeled in a way that allows this kind of easy integration. So back to the question: Is there a framework that you use for development or do you just add things as you go and rewrite/modify the code base  to support it?

Are you looking at the same schema? :) Photos are extensions of the Post class, which has a database. What's stopping you from creating a Video or Poll class which extends Post? The idea is that anything you post to your feed is a Post and should be treated equally. Obviously, to the server these different media types are handled differently, but regardless of what is contained in the post it's still a post and should be treated as such. This also makes it easier to sort through and display one's activity feed, which is definitely a central part of any social networking site.

I have seen that but the attributes for photo are tied to the post table. Say I wanted to add a poll publisher, following the current design I would put the poll attributes in the post table also. Following this design will lead to a very long and ugly post table. A more pluggable design would be to create a table for the items you want to publish e.g videos, photos etc... let us call them "publishables", and relate them to post in a 1:n or n:n manner (depending on whether you want to allow one/many publishables per post).

 

Diaspora was built using behavior-driven development (BDD). It's a concept that's an offshoot of test-driven development (TDD), wherein you write tests for your application first, watch them fail, and then write just enough code to make them pass. By attacking each problem separately, and before it becomes a "bug", you end up writing more bulletproof software which can be easily maintained. The best part about BDD is when you want to add new features. By writing test-first, you can quickly see how your new code is affecting the rest of the application because you're only writing enough code to make your new feature's tests pass.

 That's all good and peachy from the perspective of the current developers, but from the perspective of someone on the outside looking in it may look like the project lacks direction. A documented architectural design would serve to guide the development of the project from the inside and to make it much easier for someone who is just starting to grasp how diaspora works internally. If someone really has to read through the source code to understand how the system works, then that might be a deterrent to contribution simple because they do not think they have enough time to review the entire code base. If there was a documented architecture with maybe a few diagrams to show process flow, then someone could take up the project at any point and within a short time they could understand how everything works or maybe a single unit/module they are interested and how it relates to/affects the rest of the system. Documentation is very important.

An ERD for the database along with schema documentation or maybe model documentation would also help. Take the /ging/social_stream project for example. They have a lot of things documented which makes it very easy to follow that project without having to watch each commit everyday.  

This product is intended to be distributed. Is the system designed in such a way that other people can easily customize the view layer or is it "themable"?

I wouldn't say it's "easy" to change Diaspora's look & feel, but I feel like theming is something that's can be implemented pretty quickly, considering projects like Typo have already mastered it. Although I think we have some more pressing matters to deal with right now than the UI. Of course, you should feel free to contribute such a feature... :)

I agree that there are more important things than UI to look at right now. However, that's not really an UI concern, it is more of an architecture concern. For diaspora to acheive it's goal, the ability for a pod to differenciate itself from another is an important factor and of this factor UI is a very important once. So designing the system to be easily customizable is this regard from the early stages may be better than hacking to code later to allow this. So the current architecture could allow for multiple themes but only works on a default one. If that is done I think anyone at all would volunteer to document how to create a customized theme (heck even I would probably do something as boring as that).

The selling point of diaspora is "privacy" but so far the only layer of privacy I have seen so far is the "aspect". Is this meant to be the only layer of privacy for diaspora or are there plans to provide a more granular method for preserving privacy?

Is there a documented protocol for communicating with diaspora nodes/pods from other systems?

It's already implemented! If the Diaspora pod you're trying to connect to doesn't require an invite, you can easily sign in to that pod with your original Diaspora account details. This works like Twitter/Facebook connect, wherein you register using your existing account and the pod simply migrates the information into its own database. So technically, you're creating another account but since it's all "in the family" we can safely transfer lots of account details to get you immediately started on the new pod. This method was chosen because it's secure, fast and easy to understand. In addition, your pod is self-sufficient and doesn't ever need to "phone home" to some centralized user database. The point is decentralization, after all.   e you satisfied with the current pace of development? There was so much hype about diaspora last year that I decided to learn ruby so I could be a part of it, but after the initial release and some negative feedback from certain "reporting houses", the flames just died.

Ok, it's already implemented, but I do not see any documentation for this. So again, I would have to go through the source code to see how it works no? 

I have taken a look(not thorough) at source and I must say that I am not impressed with the progress so far. From what I understand, you guys are so students (as am I); I understand you may have been busy with school so that may be the cause. 

It sounds like you need to learn a little more about Rails, Ruby and behavior-driven development.

I don't know much about ruby or ruby on rails but I do have a firm grasp of programming, oop and mvc so al there is to ruby is just a different syntax from my perspective. I do have an idea of behaviour driven development. We did apply it to a couple of school projects but we found it more efficient to design the system before we started coding during the last two semesters.

It may work for diaspora but it may not be the most efficient way to go about it.

Some of my other concerns were not addressed. Thank you for your reply.

[Sarah Mei]

Documentation, and other things that would help us more quickly grow
our developer base, haven't been a focus for us. Even without it,
we've had over 100 people contribute to the project, which is amazing.

But we'd like to do better, and we'd like to move more people from
one-time contributor to regular committer. Sadly we have very limited
bandwidth at the moment, and we're unlikely to focus on documenting an
architecture that is still in constant flux.

That said, all our best documentation to date is community-generated,
so please - take a stab at writing some of the things you say are
missing! :)

Sarah

--
------------------
Sarah Mei
sarahmei.com/blog

[Ilya Z]

Worth noting, all of our current contributors earned the "Open Source Rookie of the Year" award by Black Duck Software and placed Diaspora in the top 2% of all open source that have ever existed. That being said, there is definitely room to grow since there have existed many open source projects.

Best,

Ilya

P.S. Another somewhat funny tidbit that we noticed that someone noticed the other day, is when you google for "github" our repository is the third result right above their official Wikipedia and Twitter pages.