Re: LessWrong codebase

[John Salvatier]

Thanks, that is super helpful! I think that gets me at least most of the way to being comfortable hosting a prize. Could I ask your help in judging such a prize? 

On Sun, Sep 25, 2011 at 8:03 AM, Laurent Bossavit <lau...@bossavit.com> wrote:

Hi John,

Here are some bits of information from my perspective.

       • How different is LW from its parent code (Reddit)?

Less Wrong is literally a fork (in the Github sense) of the Reddit code, so you can actually get a factual answer to this question by looking at commit histories.

Less Wrong starts to diverge from Reddit after Jan 08, 2009
 https://github.com/reddit/reddit/commits/master?page=16
 https://github.com/tricycle/lesswrong/commits/master?page=41

The number of pages of commit histories suggests that LW has evolved a lot more than its ancestor code.

       • What would I have to know to begin hacking on LW assuming that I have generic programming experience? I assume I would need python, postdb, Pylons

One key question is "how are the objects perceived at user level (articles, comments, karma, etc) represented at the implementation level". For instance, karma is tricky because there isn't a "total karma" field in the DB, it's computed by summing comment and article karma scores each time you query it.

You need to know a bit about the MVC pattern to navigate your way around this kind of question for the LW codebase. Knowing something about other MVC-inspired frameworks, such as Ruby on Rails, will help.

The database isn't very visible, mostly hidden behind a low-level ORM.

Pylons provides the templating language (Mako), plus things which make the design somewhat obscure, such as a "context" global variable.

       • What frameworks, tools is LW/Reddit based on?

On top of Reddit, LW has some things added, like Capistrano (which is used for deployment and thus shouldn't affect anyone just wanting to hack around). I don't know what else. It uses memcached for caching (if you forget to start this up during development you'll see strange behaviour and wonder where it's coming from).

You also need to know about some client-side features; there is some Ajax, a client-side editor called TinyMCE, some Reddit-specific bits of JavaScript.

       • Is LW organized the same way as lots of other websites or some particular framework or is it idiosyncratic?

Reddit and thus LW suffer from an unfortunate absence of automated unit or functional tests.

       • How long would it take me to be able to start hacking on LW effectively if I started with python programming experience but no web programming experience but had a good guide to the codebase? what if I started with web programming experience but in a different framework (say django)?

You can start hacking fairly quickly, but adding major features is far from trivial.

The best guide would be an experience report from someone who had in the past successfully added a non-trivial feature (say, polls) to LW documenting their difficulties and insights.

Cheers,
Laurent Bossavit
lau...@bossavit.com

[Laurent Bossavit]

> Thanks, that is super helpful! I think that gets me at least most of
> the way to being comfortable hosting a prize. Could I ask your help
> in judging such a prize?

Sure, count me in.

Cheers,
Laurent

[John Salvatier]

Hi John, Max

My apologies for not replying sooner. This email slipped off my radar. Answers to your questions follow.

On Fri, Sep 23, 2011 at 2:02 AM, John Salvatier <jsal...@u.washington.edu> wrote:

Hi Wesley,

Max Kesin and I are interested in possibly writing an introduction to the LW codebase or finding someone to do so. 

I picture something that:

• Lists the basic frameworks and tools that are not specific to LW and how central they are to the code along with places where you might go learn them.
• Outlines the basic organization of the site that is LW specific so that new people can get a general sense of the structure and can find out where to look when they want to look at some specific functionality 

Unfortunately, I don't know much about the LW codebase or even about web programming in general, so it's difficult for me to assess how easy or useful this would be. 

Could you answer a couple of basic questions? I don't need very detailed answers, just a general understanding of what things to look at to increase my understanding. Here are my main questions:

1. How different is LW from its parent code (Reddit)?

Reddit has obviously changed a bit since we forked the code but the changes to the version we forked are mostly around the appearance of the site and addition of Less Wrong specific features such as the concept of an Article, addition of Meetups, etc. So in general it isn't hugely different from Reddit.

1. What would I have to know to begin hacking on LW assuming that I have generic programming experience? I assume I would need python, postdb, Pylons 

Along with programming experience you should have a knowledge of web applications and web technologies in general (HTML, CSS, JS, HTTP). To a lesser degree some database knowledge (SQL). More specific things to know are Python (or another comparable scripting language such as Perl or Ruby, maybe PHP if the experience was with a modern object oriented PHP web framework) and Pylons (although Less Wrong/Reddit doesn't use Pylons model layer). Whilst the code uses PostgreSQL as the database it is not used as a typical relational database so familiarity with the concept of a key-value store (so called "NoSQL") is useful.

1. What frameworks, tools is LW/Reddit based on?

Noteworthy things the code uses are:

• Python
• Pylons
• SQLAlchmey 
• PostgreSQL
• Prototype (although jQuery is now also present)
• Memcached
• Mako Templates

1. Is LW organized the same way as lots of other websites or some particular framework or is it idiosyncratic?

The code follows a loose MVC (Model View Controller) paradigm and the code is organised to reflect that. MVC is certainly something to be familiar with. The r2/r2/ path (poorly named by the Reddit people representing their "release 2", to original Reddit was written in Lisp) contains the meat of the app. You will note 'models' and 'controllers' directories for those things and a 'templates' directory for the views plus other support code in 'lib'. This structure is typical of other MVC web apps such as Ruby on Rails.

1. How long would it take me to be able to start hacking on LW effectively if I started with python programming experience but no web programming experience but had a good guide to the codebase? what if I started with web programming experience but in a different framework (say django)? 

With Python and web experience you could be effective in a 1 - 3 days. Without web experience you're likely to feel quite lost for a bit as there's a bunch of conventions and technologies to learn. It would be more like 3 - 7 days. Keep in mind programmers (like myself) are notorious for underestimating though.

 

Cheers,

John

If there's anything else I can help with, let me know. I'll try to reply in a more timely manner next time.

Wes