Facebook rewriting PHP?

29 views
Skip to first unread message

Oscar Merida

unread,
Jan 14, 2010, 10:11:47 AM1/14/10
to DCPHP PHPDC
Morning all - I'm reading this article and its a great insight into
how Facebook uses the data it collects as well as PHP.

http://therumpus.net/2010/01/conversations-about-the-internet-5-anonymous-facebook-employee/?full=yes

I found this particularly interesting:

"Our site is coded, I’d say, 90% in PHP. All the front end —
everything you see — is generated via a language called PHP. He is
creating HPHP, Hyper-PHP, which means he’s literally rewriting the
entire language. There’s this distinction in coding between a scripted
language and a compiledlanguage. PHP is an example of a scripted
language."

as well as

"We’re going to reduce our CPU usage on our servers by 80%, so
practically, users will just see this as a faster site. Pages will
load in one fifth of the time that they used to."

I'm surprised that anyone would think its an easy undertaking to
rewrite PHP as a compiled language. I can understand than APC/opcode
caches don't provide a sufficient performance boost for a behemoth
site like Facebook. But I fail to see why rewriting the whole
language is a better option than moving a lot of their code from PHP
scripts to PHP extensions written in C. I'm also skeptical that
reducing CPU usage by 80% correlates to such a large improvement in
page load time. When loading a page the usual bottlenecks are the
network and the number of requests that the page has to make. To see
an improvement this large, FB must have already optimized that side of
the equation as much as it can.
--
Oscar Merida
* http://OscarM.org - random thoughts on my blog
* http://SoccerBlogs.net - your daily soccer feed
* http://FutBoliviano.com - bolivian soccer, futbol de altura

Ray Paseur

unread,
Jan 14, 2010, 10:40:27 AM1/14/10
to Oscar Merida, DCPHP PHPDC
Not having read the article, and accepting that the "80%" number is one of those things you say for public effect, my only first-blush comment is this: Whenever there is a performance problem, it is nearly always traceable to the I/O subsystem, most of the time to the data base, not the CPU consumption. It seems plain to me that a page-load time that is forecast to be this much faster is impossible to achieve with CPU optimization.

Unless, of course, 80% of the page-load time is 100% CPU-busy.

Typical comparisons of processor and memory access versus disk I/O access generate ratios of nanoseconds to milliseconds - several orders of magnitude. As Brandon S. has written elsewhere, there are some things you should not tune.

Best to all,
Ray

-----Original Message-----
From: washington-...@googlegroups.com [mailto:washington-...@googlegroups.com] On Behalf Of Oscar Merida
Sent: Thursday, January 14, 2010 10:12 AM
To: DCPHP PHPDC
Subject: [dcphp-dev] Facebook rewriting PHP?

Morning all - I'm reading this article and its a great insight into how Facebook uses the data it collects as well as PHP.

http://therumpus.net/2010/01/conversations-about-the-internet-5-anonymous-facebook-employee/?full=yes

I found this particularly interesting:

"Our site is coded, I'd say, 90% in PHP. All the front end - everything you see - is generated via a language called PHP. He is creating HPHP, Hyper-PHP, which means he's literally rewriting the entire language. There's this distinction in coding between a scripted language and a compiledlanguage. PHP is an example of a scripted language."

Oscar Merida

unread,
Jan 14, 2010, 10:37:40 AM1/14/10
to Ray Paseur, DCPHP PHPDC
On Thu, Jan 14, 2010 at 10:40 AM, Ray Paseur <rpa...@nationalpres.org> wrote:
> Not having read the article, and accepting that the "80%" number is one of those things you say for public effect, my only first-blush comment is this:  Whenever there is a performance problem, it is nearly always traceable to the I/O subsystem, most of the time to the data base, not the CPU consumption.  It seems plain to me that a page-load time that is forecast to be this much faster is impossible to achieve with CPU optimization.
>

Much less CPU optimizations on the server side of things. See
http://developer.yahoo.com/performance/rules.html
"80% of the end-user response time is spent on the front-end. Most of
this time is tied up in downloading all the components in the page:
images, stylesheets, scripts, Flash, etc. Reducing the number of
components in turn reduces the number of HTTP requests required to
render the page. This is the key to faster pages."

M Yilmaz

unread,
Jan 14, 2010, 10:42:39 AM1/14/10
to Ray Paseur, washington-...@googlegroups.com
I dunno Ray, the roadmap of tuning to the CPUs is fairly clear and the risk is pretty low.   Tuning I/O would be better, but that is (generally) a mission of kernel improvement, not PHP.  Facebook is challenged to squeeze all the performance they can make PHP have, so this seems to make sense for - plus the corporate adoption of the development would be good for all.  I am in agreement with you but it seems like your looking the gift horse in the mouth.  Let them tweak, they are facebook and they have so much dependence on PHP that its development is now part of the mission.  If that (CPU tweaking) isn't the best way to make PHP faster, I am sure they will figure out what will - its the result they MUST produce.

On Thu, Jan 14, 2010 at 10:40 AM, Ray Paseur <rpa...@nationalpres.org> wrote:
--
You received this message because you are subscribed to the Google
Group: "Washington, DC PHP Developers Group" - http://www.dcphp.net
To post, send email to washington-...@googlegroups.com
To unsubscribe, send email to washington-dcphp-...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/washington-dcphp-group?hl=en



--
Mehmet Yilmaz

Oscar Merida

unread,
Jan 14, 2010, 10:48:46 AM1/14/10
to M Yilmaz, Ray Paseur, washington-...@googlegroups.com
In the article, they're supposedly forking their own compiled version
of PHP(whatever that means exactly). I wouldn't assume that they
would release that to the outside world or that anything they learn in
the process will benefit the wider php community.

Tac Tacelosky

unread,
Jan 14, 2010, 10:49:36 AM1/14/10
to Oscar Merida, Ray Paseur, DCPHP PHPDC
Facebook also uses a lot of Javascript and AJAX calls.  I would think they would get a huge performance boost by using Google Gears and/or an HTML5 local database (SQLite).  Not only would their server load drop, they could design it so that the user experience was almost entirely local, and have the slower requests happen in the background.  Even if status updates were 15 seconds delayed, I would think that the user experience would be much better.

I never thought I'd use web-based email, but now I love gmail and wouldn't consider going back to the old Exchange / Outlook. 

Tac


Steve Roberson

unread,
Jan 14, 2010, 10:58:04 AM1/14/10
to DCPHP PHPDC
Could it be that their motivation is not so much to reduce the load time but to reduce the infrastructure needs associated with their (growing) user base? After all, if you have hundreds of millions of users, some of which treat your site like it's their new OS, you probably have thousands of machines supporting it.

Steve

M Yilmaz

unread,
Jan 14, 2010, 11:01:49 AM1/14/10
to Oscar Merida, Ray Paseur, washington-...@googlegroups.com
I think it actually is a fairly safe assumption that they will, but it will be on a delayed schedule in all probability.  Just my humble opinion.
--
Mehmet Yilmaz

Brandon Savage

unread,
Jan 14, 2010, 11:05:54 AM1/14/10
to Tac Tacelosky, Oscar Merida, Ray Paseur, DCPHP PHPDC
I don't think that Facebook is rewriting the entire language. The easiest way to write HPHP would be simply to rewrite the compiler so that PHP could be compiled and executed from a compiled binary.

While I'm not intimately familiar with the internals of PHP itself, my guess is that PHP compiles the source into opcodes, which is then translated into other computer-readable formats. Facebook's goal here is probably to have a compiler that renders the code executable for the machine it's running on, without having to figure out the details of the libraries. They're essentially standardizing it.

I'm typically one to argue against micro optimization. However, in Facebook's case, they need it. They need to optimize for CPU, I/O, network, etc. Their scale is so vast that every little bit makes an enormous difference.

Brandon

================================
Brandon J. Savage


"Courage is not the absence of fear, but rather the judgment that something else is more important than fear." ~ Ambrose Redmoon

Joshua Boyd

unread,
Jan 14, 2010, 11:07:44 AM1/14/10
to Brandon Savage, Tac Tacelosky, Oscar Merida, Ray Paseur, DCPHP PHPDC
A few PHP compilers already exist. phpc and roadsend. Mostly experimental though.


On Thu, Jan 14, 2010 at 11:05 AM, Brandon Savage <bra...@brandonsavage.net> wrote:
While I'm not intimately familiar with the internals of PHP itself, my guess is that PHP compiles the source into opcodes, which is then translated into other computer-readable formats. Facebook's goal here is probably to have a compiler that renders the code executable for the machine it's running on, without having to figure out the details of the libraries. They're essentially standardizing it.



--
Josh Boyd
Software Engineer
Endeavor Systems, Inc. - Securing the Business of Government
joshu...@endeavorsystems.com | 513.375.0157 | www.endeavorsystems.com

keith...@gmail.com

unread,
Jan 31, 2010, 8:44:58 AM1/31/10
to Washington, DC PHP Developers Group
On Jan 14, 11:07 am, Joshua Boyd <joshua.b...@endeavorsystems.com>
wrote:

> A few PHP compilers already exist. phpc and roadsend. Mostly experimental
> though.

Here's a little more public info:
http://www.sdtimes.com/blog/post/2010/01/30/Facebook-rewrites-PHP-runtime.aspx

Disclosure: We (Blue Parabola) got a technology preview recently and
we have some limited involvement in what they're working on and what
they may or may not be announcing eventually.

keith

Hans Lellelid

unread,
Feb 1, 2010, 8:28:23 AM2/1/10
to Washington, DC PHP Developers Group
This is interesting stuff. In general I like how Facebook/Friendfeed
have handled their open-source contributions thus far. We've been
playing with Tornado (their epoll python webapp server) and the more
technology-agnostic Thrift binary protocol. Both seem to be pretty
solid software contributions; I'm certainly interested to see what
they cough up here.

I wonder whether companies will start looking to languages like "Go"
when they need that completely different scale of performance. ... My
complaint with "Go" is no exceptions; if I were able to get over that,
it looks like it has some interesting qualities. (I don't think I
could get over that missing feature, though, for a software project of
any size.)

Hans

On Jan 31, 8:44 am, "mailingli...@caseysoftware.com"


<keith.ca...@gmail.com> wrote:
> On Jan 14, 11:07 am, Joshua Boyd <joshua.b...@endeavorsystems.com>
> wrote:
>
> > A few PHP compilers already exist. phpc and roadsend. Mostly experimental
> > though.
>

> Here's a little more public info:http://www.sdtimes.com/blog/post/2010/01/30/Facebook-rewrites-PHP-run...

keith...@gmail.com

unread,
Feb 2, 2010, 1:17:52 PM2/2/10
to Washington, DC PHP Developers Group
Alright, so the Facebook/PHP news is now public:

http://bit.ly/b1qNHL

And we're allowed to comment on it. Here are Blue Parabola and php|
architect's positions on things:

http://www.phparch.com/main/news/view/68/Announcing_our_support_for_Facebook_s_HipHop
http://blueparabola.com/blog/announcing-our-support-facebooks-hiphop

kc

jproffer

unread,
Feb 2, 2010, 1:26:01 PM2/2/10
to keith...@gmail.com, Washington, DC PHP Developers Group
interesting.  So, I'm assuming this means your site has to run via fastcgi.

jproffer

unread,
Feb 2, 2010, 1:29:50 PM2/2/10
to keith...@gmail.com, Washington, DC PHP Developers Group
Reminds me of C++ Server Pages (http://www.micronovae.com/CSP.html) - pretty sad that they never expanded beyond the Windows platform, or opensourced their product (considering its free to begin with).

D Keith Casey Jr

unread,
Feb 2, 2010, 1:53:31 PM2/2/10
to jproffer, Washington, DC PHP Developers Group, kin...@gmail.com
I'm getting deluged by questions... so to address a flock of them all at once:

* It's nothing like FastCGI or does anything with FastCGI at all. When you
run the "compilation" step it actually optimizes everything down to C++ and
then compiles that.

* Therefore, you don't deploy it with the normal Apache/PHP combo, you deploy
it instead of them normal combo. It creates one big binary that serves as its
own server.

* Yes, it runs independent of the Zend Engine. It's irrelevant to this
application - http://twitter.com/nateabele/status/8552203049

* Yes, Facebook is running this in production now and they have been for a
year+. They built this because APC, etc weren't fast enough for them.

* Yes, it will be released Open Source, most likely under the PHP License.

* Yes, it fundamentally changes your deployment process... now if we just had
a nice Ant-like tool to help us? *cough*Phing*cough* ;)

* No, once the binary is created, you can't modify it without rebuilding a
new version. So no more "editing files on production".

* No, it won't make you SQL queries any faster. If your queries suck now,
they'll suck after - http://twitter.com/rasmus/status/8552927895

* No, it doesn't support 100% of PHP currently. It supports upwards of 90% of
functions.. but this may be a good thing as "eval()" is currently in the
non-supported list.

* No, web2project is not compatible using this yet. That's on the agenda to
get prepared for our own offerings. Rumor has it that a smart WordPress guy is
looking at it too - http://twitter.com/technosailor/status/8553031318 ;)


Overall, what's the point?


You write PHP and it runs with the speed of C. Facebook (and others) have
already seen a 50% performance improvement -
http://twitter.com/rasmus/status/8552893110


kc

jproffer wrote:
> Reminds me of C++ Server Pages (http://www.micronovae.com/CSP.html) -
> pretty sad that they never expanded beyond the Windows platform, or
> opensourced their product (considering its free to begin with).

--
D. Keith Casey, Jr.
CEO, CaseySoftware, LLC
http://CaseySoftware.com

Joshua Boyd

unread,
Feb 2, 2010, 2:06:35 PM2/2/10
to D Keith Casey Jr, jproffer, Washington, DC PHP Developers Group, kin...@gmail.com
I'm assuming it will, however, make your SQL queries faster if you're using an ORM. Particularly the parsing parts of things like Doctrine, where DQL is transformed into SQL. Of course the actual query won't be faster, but the creation of data objects and working with them should be.

I'm anxious to run tests once this is released to see if Zend_Lucene stops being slow as molasses.

--
You received this message because you are subscribed to the Google
Group: "Washington, DC PHP Developers Group" - http://www.dcphp.net
To post, send email to washington-...@googlegroups.com
To unsubscribe, send email to washington-dcphp-...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/washington-dcphp-group?hl=en

Svemir Brkic

unread,
Feb 2, 2010, 2:08:23 PM2/2/10
to washington-...@googlegroups.com
> * No, it doesn't support 100% of PHP currently. It supports upwards of
> 90% of functions.. but this may be a good thing as "eval()" is currently
> in the non-supported list.

What are some other things it does not support? I know more official
info will be published soon, but it seems that you are already working
with it. Thanks!

--
Svemir Brkic
sve...@symplicity.com
Symplicity Corporation
http://www.symplicity.com

D Keith Casey Jr

unread,
Feb 2, 2010, 2:19:30 PM2/2/10
to Svemir Brkic, washington-...@googlegroups.com
Svemir Brkic wrote:
>> * No, it doesn't support 100% of PHP currently. It supports upwards of
>> 90% of functions.. but this may be a good thing as "eval()" is currently
>> in the non-supported list.
>
> What are some other things it does not support? I know more official
> info will be published soon, but it seems that you are already working
> with it. Thanks!

That's one of the things I can't get into yet. ;) In our testing, a scary
number of open source projects use "eval()" though... WordPress, Drupal are
just the most prominent ones.

Regardless, I missed the line in the post that HipHop is being released
tonight via GitHub and it is *officially* under the PHP license.

kc

Svemir Brkic

unread,
Feb 2, 2010, 2:20:37 PM2/2/10
to washington-...@googlegroups.com
> I'm anxious to run tests once this is released to see if Zend_Lucene
> stops being slow as molasses.

Frameworks that include zillions of files will probably see a huge
improvement - if this thing indeed results in a single binary.

Even when using opcode cache, and without spending much CPU time, your
ability to handle concurrent requests is likely to improve a lot.
Depending on the file system etc. you may not see any difference with
one, five, or ten requests, but keep increasing the number and you will
be pleasantly suprised :-)

P. S. Above statement is based on a comparision between "all classes in
one file" vs. "a bunch of included classes" and may not be entirely (or
at all) representativ of "regular zend framework page" vs. "hip hopped page"

Shaun Farrell

unread,
Feb 2, 2010, 2:22:24 PM2/2/10
to D Keith Casey Jr, Svemir Brkic, washington-...@googlegroups.com
So do you have to change anything other than deployment process.  I still code php the way php is coded and just compile and deploy?


Shaun J. Farrell
Washington, DC
(202) 713-5241
www.farrelley.com


Shaun Farrell

unread,
Feb 2, 2010, 2:29:05 PM2/2/10
to Svemir Brkic, washington-...@googlegroups.com
Yes! I am sure frameworks will speed up.  




Shaun J. Farrell
Washington, DC
(202) 713-5241
www.farrelley.com


twitter.png

John Yodsnukis

unread,
Feb 2, 2010, 3:06:59 PM2/2/10
to Washington, DC PHP Developers Group
On Tue, Feb 2, 2010 at 1:53 PM, D Keith Casey Jr
<ke...@caseysoftware.com> wrote:
> I'm getting deluged by questions... so to address a flock of them all at
> once:
>

<snip>

> *  Therefore, you don't deploy it with the normal Apache/PHP combo, you
> deploy it instead of them normal combo. It creates one big binary that
> serves as its own server.
>

<snip>

> Overall, what's the point?
>
>
> You write PHP and it runs with the speed of C. Facebook (and others) have
> already seen a 50% performance improvement -
> http://twitter.com/rasmus/status/8552893110
>

Additionally, you can now write and distribute php single end-user
desktop-type applications as one or two files in a zip archive.  Just
add SQLite.  This is HUGE.

D Keith Casey Jr

unread,
Feb 2, 2010, 4:46:02 PM2/2/10
to Shaun Farrell, Svemir Brkic, washington-...@googlegroups.com
Shaun Farrell wrote:
> So do you have to change anything other than deployment process. I
> still code php the way php is coded and just compile and deploy?

Correct. Facebook's stated goal is to allow PHP developers to continue writing
*in* PHP but get all the benefits of compilation.

Not surprisingly, Marco talks about it here -
http://blog.tabini.ca/2010/02/hiphop-what-you-need-to-know/

jproffer

unread,
Feb 2, 2010, 5:06:19 PM2/2/10
to D Keith Casey Jr, washington-...@googlegroups.com
I imagine that while this is probably a great thing, it would see very limited use.  What real-life scennario would this prove useful, especially considering most hosting companies won't be able to run your standalone application unless you purchase a VPS or dedicated server.  I'm sure the distributed applications will also have some platform limitations, and still require you set up a database on the target machine, so its not like you can just compile your new CRM application and send it to a client to run in windows?  I also imagine this will have other problems like upgrading, port conflicts, etc.

D Keith Casey Jr

unread,
Feb 2, 2010, 5:20:04 PM2/2/10
to jproffer, washington-...@googlegroups.com

You're 100% correct there. This has *zero* value to people running on shared
hosting environments. Even small traffic sites with a single server aren't
going to have many benefits. This kicks in where you have at least a few servers.

Can CaseySoftware.com on Dreamhost use it? Not a chance.

Can the Library of Congress (internally hosted dedicated servers) use it?
Yes.. and then handle twice as much traffic with the same setup.

kc

jproffer wrote:
> I imagine that while this is probably a great thing, it would see very
> limited use. What real-life scennario would this prove useful,
> especially considering most hosting companies won't be able to run your
> standalone application unless you purchase a VPS or dedicated server.
> I'm sure the distributed applications will also have some platform
> limitations, and still require you set up a database on the target
> machine, so its not like you can just compile your new CRM application
> and send it to a client to run in windows? I also imagine this will
> have other problems like upgrading, port conflicts, etc.

--

Joshua Boyd

unread,
Feb 2, 2010, 5:48:27 PM2/2/10
to D Keith Casey Jr, jproffer, washington-...@googlegroups.com
This will work very well for our applications, where we primarily distribute our application as an appliance to our customers. Well, once Zend and Doctrine work properly with it, atleast.

--
You received this message because you are subscribed to the Google
Group: "Washington, DC PHP Developers Group" - http://www.dcphp.net
To post, send email to washington-...@googlegroups.com
To unsubscribe, send email to washington-dcphp-...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/washington-dcphp-group?hl=en

Barry Austin

unread,
Feb 2, 2010, 6:00:18 PM2/2/10
to DCPHP
Something like a JIT compiler could do the trick. Not easy though.

B

jproffer

unread,
Feb 2, 2010, 6:04:17 PM2/2/10
to D Keith Casey Jr, washington-...@googlegroups.com
what about sites that need read/write access to directories.  Like, image and video processing/distribution sites?  Since the app compiles everything into a binary, how does it translate paths that would normally be within directories in the web root.


D Keith Casey Jr wrote:

Joshua Boyd

unread,
Feb 2, 2010, 6:08:35 PM2/2/10
to jproffer, D Keith Casey Jr, washington-...@googlegroups.com
I'd assume that you could have your app access an external configuration file, pointing to the correct paths to write to or serve static files from.


On Tue, Feb 2, 2010 at 6:04 PM, jproffer <jpro...@gmail.com> wrote:
what about sites that need read/write access to directories.  Like, image and video processing/distribution sites?  Since the app compiles everything into a binary, how does it translate paths that would normally be within directories in the web root.



jproffer

unread,
Feb 2, 2010, 6:21:43 PM2/2/10
to Joshua Boyd, D Keith Casey Jr, washington-...@googlegroups.com
Sure, but since it seems likely that (considering the binary is a self-contained webserver) the end user doesnt have direct access to files on the server, e.g. uploaded images or video, I imagine that means your app also has to handle what would normally be the apache/iis process' file I/O responsibility? Or will the binary consider whatever path it is in, to be "document_root" so to speak, and allow you to just create subfolders which are directly accessible by the end-user?

Basically what I'm asking is, would I have to write file_io.php which basically does something like fread() or file_get_contents() to make things like uploaded images/videos/documents accessible to the end user, or does it, as in the latter, automatically assume current path as document_root, making everything within that path and subdirectories user-accessible (permissions allowing)?

D Keith Casey Jr

unread,
Feb 2, 2010, 6:23:56 PM2/2/10
to jproffer, washington-...@googlegroups.com
jproffer wrote:
> what about sites that need read/write access to directories. Like,
> image and video processing/distribution sites? Since the app compiles
> everything into a binary, how does it translate paths that would
> normally be within directories in the web root.

*Excellent catch*

That's actually the *one* thing that makes web2project incompatible with it.
We've put it on the agenda for our v2.0 release.

kc

keith...@gmail.com

unread,
Feb 2, 2010, 8:19:32 PM2/2/10
to Washington, DC PHP Developers Group

And to not be 100% biased, Ilia - php 5.2 release manager - has some
additional perspective on HipHop - http://ilia.ws/archives/213-My-Thoughts-on-HipHop.html#extended
- including a number of the drawbacks and existing problems.

kc

jproffer

unread,
Feb 3, 2010, 11:41:27 AM2/3/10
to keith...@gmail.com, Washington, DC PHP Developers Group
Did FB release HipHop to Github as promised?  I'm unable to find it.  Anyone find it?

Aaron Brazell

unread,
Feb 3, 2010, 12:51:34 PM2/3/10
to D Keith Casey Jr, Svemir Brkic, washington-...@googlegroups.com
On Tue, Feb 2, 2010 at 2:19 PM, D Keith Casey Jr <ke...@caseysoftware.com> wrote:
Svemir Brkic wrote:
* No, it doesn't support 100% of PHP currently. It supports upwards of
90% of functions.. but this may be a good thing as "eval()" is currently
in the non-supported list.

What are some other things it does not support? I know more official info will be published soon, but it seems that you are already working with it. Thanks!

That's one of the things I can't get into yet. ;) In our testing, a scary number of open source projects use "eval()" though... WordPress, Drupal are just the most prominent ones.


WordPress only has one instance of eval() in PHP with many instances of eval() as a JavaScript function. Though the JS part is unrelated. I'm looking more at the variable variable uses (there are 31 in WordPress trunk) and trying to figure out a way to abstract those without massive amounts of code edits. Ideas always welcome.

I haven't seen the source code yet to know what else won't be supported.

Aaron Brazell 

Hans Lellelid

unread,
Feb 3, 2010, 4:50:34 PM2/3/10
to Washington, DC PHP Developers Group
> Correct. Facebook's stated goal is to allow PHP developers to continue writing
> *in* PHP but get all the benefits of compilation.
>
> Not surprisingly, Marco talks about it here -http://blog.tabini.ca/2010/02/hiphop-what-you-need-to-know/

This HipHop thing is interesting, perhaps in much the same way as
HipHop music: it feels like a hack. -- And I mean that respectfully
in both cases; I like hip-hop music, and appreciate how it pays homage
to R&B roots, remixing/reinterpreting them, etc; and I think that the
idea of taking one language and building it out to something else is
also something I should support. After all, I've embroiled myself in
code generation tools (e.g. Propel) that are operating on the same
philosophical groundwork. But I also believe that there's a general
rule like "if you need code generation, there's something wrong [in
your design or in the tools you've chosen or ...]" ... so those tools
also feel like hacks.

I guess the "hack" here is implicit in anything that's trying work
with what ya got. Another analogy is the DVR, the hack to fix the
problem that people want to watch TV programming when they want, not
when it's played. All indications in content delivery trends suggest
that we will look back and laugh about days when we needed DVRs. But
they provide a valuable service now (and probably for a number of
years to come). Similarly, I would hope (and expect) that the HipHop
project would become obsolete in the future too. It doesn't seem
unreasonable to expect that someone could write a PHP virtual machine
that performed as well as the compiled code. There are benchmarks
that show the JVM runs [some code] faster than optimized C++, so this
seems like a theoretically attainable goal. After all, since there's
nothing being added here, it seems like if there was a virtual machine
that could transparently compile PHP into machine bytecode, that'd
have the same effect (if not better) -- and skip the clumsy C-code
middleman.

I think also interesting (more interesting, honestly) is the idea of
compiling PHP (and other dynamic languages) to Java bytecode. There
at least, there's not a language rewrite happening first -- or,
rather, the rewrite is at a much lower level. (The JVM certainly
supports certain language paradigms better than others -- which can
make things weird when someone wants to do multiple inheritance [legal
in CPython] in Jython, for example.)

I think at some point, there are basic questions of why one chooses
one language over another that have to be answered. I would suggest
-- humbly ;) -- that PHP is not really chosen for its language
features. I would further suggest that it's not really chosen for its
huge library of code. I would suggest that it's actually chosen for
its accommodation of simple development patterns and its drop-dead
simple deployment paradigm. I recently wanted to write up a server-
side component to demonstrate http-only cookies. This is a few lines
of code in PHP that I just drop into any PHP server; done. I can even
mix my HTML in the same page. That is what has always been great
about PHP. Doing that in Python would have taken me some Apache
config, some sort of server script and probably a separate template
file (and then a template language) if I didn't want to print out HTML
from my wsgi app (a la CGI). Sure, I vastly prefer Python as a
language and its extensive built-in library support, but it is not
nearly as simple to develop or deploy. Well, HipHop sorta ruined the
main reason I would write something in PHP.

So, I concur with the suggestion that Brandon made in his blog and
that jproffer is making on list that this is probably of limited
attraction. Sure, if you *have* to make something perform balls-out-
fast and your developers refuse to learn a more appropriate technology
(or, in FB's case, I'd imagine they can't afford the redesign time),
this seems like a viable option. But I still think it's a hack.

Hans

Sandy Smith

unread,
Feb 3, 2010, 5:28:29 PM2/3/10
to Hans Lellelid, Washington, DC PHP Developers Group
See Terry Chay's (as usual, still-evolving-) blog post about it:


One of the key advantages to something like HipHop is that it lets your business scale all the way up from garage startup to FaceBook with the same language, minus some slight changes. Retooling was thought to take a year or two, and would likely require new engineers and restructuring much of the pre-deployment workflow.

Changing technologies (and l likely a not-insignificant percentage of your development talent) is extremely hard: not only do you have to restructure how things are built and how code is produced, but you also inevitably lose some valuable insight into the application and why it behaves as it does.

It's not impossible to manage that, but it is sometimes much harder than changing one aspect of your stack/workflow.

So to the extent it's a hack, it's a hack in the Hacker's Dictionary sense of the term: an elegant solution to a problem that solves a very large problem by changing a component in a very clever way.

-Sandy

Hans Lellelid

unread,
Feb 3, 2010, 6:48:21 PM2/3/10
to Washington, DC PHP Developers Group
On Feb 3, 5:28 pm, Sandy Smith <ssm...@forumone.com> wrote:
> See Terry Chay's (as usual, still-evolving-) blog post about it:
>
> http://terrychay.com/article/hiphop-for-faster-php.shtml

That's a good read; thanks for link.

> One of the key advantages to something like HipHop is that it lets your business scale all the way up from garage startup to FaceBook with the same language, minus some slight changes. Retooling was thought to take a year or two, and would likely require new engineers and restructuring much of the pre-deployment workflow.

Yeah, that makes sense. It certainly seems reasonable that FB would
have so much legacy code and there are clearly so many third-party
tools involved here that a platform change would be *really*
expensive. Heck, it could cost them their place in the market if it
didn't work well.

... but ...

> Changing technologies (and l likely a not-insignificant percentage of your development talent) is extremely hard: not only do you have to restructure how things are built and how code is produced, but you also inevitably lose some valuable insight into the application and why it behaves as it does.

That's all fair, but I feel like the problem here is that somewhere a
long, long time ago, Facebook *must* have realized that they were
going to have scaling problems. Long before they started having a
problem, someone *must* have thought "maybe a compile-at-runtime
language isn't the right solution here". I guess to me this cross-
compiler is just a public way to admit that PHP is not the right tool
for the job, but they're stuck with all these developers that only
know PHP so it was somehow cheaper to engineer a way to change PHP to C
++ than it was to retrain developers on C++ (or, probably more
realistic, Java).

I would imagine that these developers are going to (or already had to)
learn the quirks of HipHop. We know it won't work with some things at
all and I'm sure HipHop optimizes some things far more elegantly than
others. It's a different environment that has to be learned -- and
it's an odd, indirect way to learn it.

Obviously I have 0 experience designing/developing/managing
applications that have anywhere near the amount of code/traffic/users/
populatority/vested parties/etc. as Facebook, so I can't say "this is
dumb". I'm sure this makes perfect sense for them -- and I think
they've contributed some solid work, so I'm generally eager to see how
this goes. My more-limited experience has shown, however, that when
an application really is built as a collection of isolated components,
it does indeed become far simpler to constantly review technology as a
way to address scaling. What really locks a project to a technology
is kitchen-sink, monolithic application design. This is the beauty of
the loosely coupled approach Google has taken. I'd wager that if
Google decided that the new backend for Calendar was going to be
Erlang, it probably wouldn't be all that significant a development
effort. This is obviously apples & oranges and a bit of a tangent,
but a move like HipHop almost has a its-too-big-to-fail feel about
it. Maybe that's just me.

> So to the extent it's a hack, it's a hack in the Hacker's Dictionary sense of the term: an elegant solution to a problem that solves a very large problem by changing a component in a very clever way.

Well put. Whatever I may feel about its philosophical motivations, I
agree that it's certainly very clever. :)

Hans

Hans Lellelid

unread,
Feb 3, 2010, 7:44:41 PM2/3/10
to washington-...@googlegroups.com
Another perspective that may be worth reading:

http://alexgaynor.net/2010/feb/02/thoughts-hiphop-php/

-Hans

> effort. This is obviously apples& oranges and a bit of a tangent,

Aaron Brazell

unread,
Feb 3, 2010, 10:53:33 PM2/3/10
to Hans Lellelid, Washington, DC PHP Developers Group
Hans-

While I appreciate, and always have appreciated, your frank, honest, high level view of PHP, web security, web applications, etc., you strike me as somewhat naive and puritanical.

I don't know your background. I don't know your history. I know what I've read over the years.

What I can say is *I*, along with dozens of other technology people in and out of DC, in and out of PHP, never look at our initial ideas as scaling ideas. We look at them as ideas and experiments to see if they have legs. In fact, I'd go so far as to say it is counter-productive to think about scale before thinking of concievability (is that a word?).

There's a reason why Rails (God help us) is popular. It's a great prototyping tool. You stand up an app quickly and let it into the wild to see if it has legs. Does it go? What are the market influences? What are the pros and cons? Do we have to adjust?

After a concept is proven, then a solid dev team with solid tech leadership brings in their expertise to see if the idea can be built into something sustainable. As a sidebar, please take a read of Brad Feld's very awesome post from a few years ago "The first 25,000 Users are Irrelevant" (http://www.feld.com/wp/archives/2006/05/web-20-the-first-25000-users-are-irrelevant.html)

My point is, it's silly and a waste of resources for startup people to start thinking about how big they _might_ get _maybe_ 5 years down the road.

I think you'd find out that, in most cases, successful technology, web-based companies happened by some dumb luck. Twitter. Facebook. Name-the-popular-app. Dumb luck.

Hey, I'd even argue that when *too* much comp-sci brain energy goes into an app, you get things like Woofram Alpha. Cool. But useless. And not nimble enough to actually do the scaling necessary to need all that comp-sci engineering prowess.

Balance, my friend. Balance. Facebook (and others) start with PHP because PHP is fairly ubiquitous and, like you suggest, easy as pie to drop into production. However, there is a point of no return where you are committed to PHP and that's where HipHop comes in.

Personally, I wish we had HipHop when I was at b5media. We had a ton of scaling problems with PHP and we were running fully clustered Apache servers (25 deep, if I recall), sharded MySQL across 6ish database servers, and we had massive I/O bottlenecks. We ran eAccelerator and Memcached and had squid-based load balancing and damn if Grey's Anatomy or the Oscar's didn't pin our entire network on more than one occasion (See: http://technosailor.com/2008/12/04/be-nimble-be-quick/). What could have happened with an alternate to opcode caching. What could have happened if I had resources to put on developing C++ binaries of our frequently used PHP libraries.

I'll tell you. It would have rocked. We were already committed to PHP. We were already committed to WordPress. And when the company started, we were all volunteer resources. There was no assumption that our idea had legs or I think everyone on the team would have quit our jobs immediately and put everything into building that company. It took a year to get there.

This is, for better or for worse, the way companies get started in the real world. No offense. :)

Aaron 
 

jproffer

unread,
Feb 3, 2010, 10:58:26 PM2/3/10
to Aaron Brazell, Hans Lellelid, Washington, DC PHP Developers Group
Hmm dunno, I tend to always design with scalability in mind.  It's just how I'm wired, I guess.  Scalability, optimization.  I think its just good practice.  Then again I come from an ASM/C/C++ background.

Aaron Brazell wrote:

Aaron Brazell

unread,
Feb 3, 2010, 11:02:56 PM2/3/10
to jproffer, Hans Lellelid, Washington, DC PHP Developers Group
I don't want to imply that I think it's unimportant and not something to think about, but I also think a lot of cycles can be wasted preparing for something that never will and never should come at all. I think smart developers and engineers can see the need coming and prepare for it and spend the cycles when they know it will be needed. Chances are, most projects/apps/products will never need it. Just a statistical fact.

Aaron Brazell
CEO, Emmense Technologies
Lead Editor, Technosailor.com
Author, The WordPress Bible

e: aa...@technosailor.com
b: http://technosailor.com
t: http://twitter.com/technosailor
p: 443-455-1056Aaron Brazell
web:: www.technosailor.com
phone:: 410-608-6620
skype:: technosailor
twitter:: @technosailor

Joshua Boyd

unread,
Feb 4, 2010, 12:02:34 AM2/4/10
to Aaron Brazell, jproffer, Hans Lellelid, Washington, DC PHP Developers Group
Heh, if HipHop had been around 2 years ago, along with a few other of facebooks opensource projects, there's a chance the startup I was at would still be in existence, the infrastructure costs did a good job of killing capital when we had to scale out on AWS and CPU time is what was killing us.

Hans Lellelid

unread,
Feb 4, 2010, 11:32:56 AM2/4/10
to Aaron Brazell, Washington, DC PHP Developers Group

> While I appreciate, and always have appreciated, your frank, honest,
high
> level view of PHP, web security, web applications, etc., you strike me
as
> somewhat naive and puritanical.

;) Well, that's my missionary-kid upbringing, I'm sure.



> I don't know your background. I don't know your history. I know what
I've
> read over the years.
>
> What I can say is *I*, along with dozens of other technology people in
and
> out of DC, in and out of PHP, never look at our initial ideas as scaling
> ideas. We look at them as ideas and experiments to see if they have
legs.
> In
> fact, I'd go so far as to say it is counter-productive to think about
scale
> before thinking of concievability (is that a word?).

Yes, it's obvious we've been tasked with building differnet kinds of apps
over the years. I don't think I've ever had the pleasure (?) of building
an app to float out there & see if it works or catches on. There's always
been some concept of where it could grow and there's always been a need to
identify exactly what the scaling plan would be.

> Balance, my friend. Balance. Facebook (and others) start with PHP
because
> PHP is fairly ubiquitous and, like you suggest, easy as pie to drop into
> production. However, there is a point of no return where you are
committed
> to PHP and that's where HipHop comes in.

Well, that "point-of-no-return" is *exactly* my problem with HipHop. I
have had to deal with maintaining some huge codebases and that "point of no
return" is what kills a project, in my opinion. Smaller pieces, looser
coupling, is IMO a much, much better strategy for building an app that will
last.

> Personally, I wish we had HipHop when I was at b5media. We had a ton of
> scaling problems with PHP and we were running fully clustered Apache
> servers
> (25 deep, if I recall), sharded MySQL across 6ish database servers, and
we
> had massive I/O bottlenecks.

I don't see how HipHop would have helped if you had I/O bottlenecks. The
only thing it's gonna help with is CPU bottlenecks.

I also have run into big scaling problems with PHP in its traditional
hosting paradigm. This is partly why we looked to other technologies. The
epoll servers (+nginx) look like they're going to blow the socks off
anything we could tune PHP + Apache to handle. And when we needed a custom
SSL proxy server we wrote that in Java; sure we *could* have used Python,
but it wouldn't have been the right solution.

> This is, for better or for worse, the way companies get started in the
real
> world. No offense. :)

None taken. It may be the way companies get started in the real world,
but it's not necessarily the way software projects are run in the real
world. We don't start as volunteers; we start as paid software engineers.
We don't stick with tools because it's what we know, we look at benchmarks
in industry research to figure out what the right way to solve this problem
is. We do think about how things scale, about where we the decoupling
points are. I'm not just saying this because I'm naive; I'm saying it
because I've certainly been burned before and there's an important balance
to be struck between planning enough to get started and planning ahead.

Hans

Aaron Brazell

unread,
Feb 4, 2010, 11:40:47 AM2/4/10
to Hans Lellelid, Washington, DC PHP Developers Group
Fair enough. I did translate this discussion into a post though because I think there's value to the world outside of this Google Group (and since it's public...)...



Aaron Brazell
CEO, Emmense Technologies
Lead Editor, Technosailor.com
Author, The WordPress Bible

e: aa...@technosailor.com
b: http://technosailor.com
t: http://twitter.com/technosailor
p: 443-455-1056Aaron Brazell
web:: www.technosailor.com
phone:: 410-608-6620
skype:: technosailor
twitter:: @technosailor


Sandy Smith

unread,
Feb 4, 2010, 11:55:43 AM2/4/10
to Washington, DC PHP Developers Group

On Feb 4, 2010, at 11:32 AM, Hans Lellelid wrote:

> None taken. It may be the way companies get started in the real world,
> but it's not necessarily the way software projects are run in the real
> world. We don't start as volunteers; we start as paid software engineers.
> We don't stick with tools because it's what we know, we look at benchmarks
> in industry research to figure out what the right way to solve this problem
> is. We do think about how things scale, about where we the decoupling
> points are. I'm not just saying this because I'm naive; I'm saying it
> because I've certainly been burned before and there's an important balance
> to be struck between planning enough to get started and planning ahead.
>
> Hans


I think you're describing a scenario typical of large organizations with large budgets and stable, predictable use projections. Maybe that's not what all those projects were, but it sounds like the process typical of them.

That is not the environment of bootstrapped development with rapid success. Remember, the Facebook team has managed to handle all this growth successfully. There are not many organizations who can say the same given similar conditions (see Twitter or Boeing's newairplane.com), so I'm not sure criticizing them without a much better understanding of exactly how their components are put together (you can have loosely coupled PHP) is warranted.

You're also speaking as if every engineer is able to fluidly switch technologies at the drop of a hat with zero transition cost and be equally effective in the new paradigm, and that those costs can never outweigh the benefits of the shift. You also assume they have time to calmly research and plan, rather than put out the next fire while trying to do what they can to scale without running into the mythical man month problem. Human systems are a bit more squishy and hard to benchmark than engineering systems. ;)

The fact that you've been burned should tell you that software development doesn't always happen as it should, and if you've been able to have most of your projects work in the ideal way, you should be extremely thankful for your luck. :)

-Sandy

Hans Lellelid

unread,
Feb 4, 2010, 11:57:35 AM2/4/10
to Aaron Brazell, jproffer, Washington, DC PHP Developers Group
> I don't want to imply that I think it's unimportant and not something to
> think about, but I also think a lot of cycles can be wasted preparing
for
> something that never will and never should come at all. I think smart
> developers and engineers can see the need coming and prepare for it and
> spend the cycles when they know it will be needed. Chances are, most
> projects/apps/products will never need it. Just a statistical fact.

I think this is also highly dependent on your industry. I can't think of
any apps I've built or managed over the past 10 years that have been used
less than we anticipated. I can think of lots of them that ended up
getting used a lot *more* than we anticipated. And sure, we've rewritten
code, hacked things around to make them scale; we've done whatever it took.
We also learned that scaling is expensive; for us, thinking ahead is worth
it.

Here's my [final] attempt to clarify my point. I think we would all agree
that HipHop is a kludge. It's a necessary kludge for Facebook because they
committed themselves to a technology that could not scale to meet _their_
needs. I personally value cleanliness and the "right" solution. This is
often impractical, but it's still a valid opinion, I think. Given the
limitations of the system, I don't think rescuing PHP by cross-compiling it
to C++ is the "right" way to do anything (other than keep using PHP). I
think it's too bad that the Facebook web application couldn't have been
written in a way that would have made this unnecessary. While I know it's
not the same, I would again point to Google and say that their deliberate
move to start using more Java (instead of Python) feels like it's exactly
the other story -- the "right" way to evolve your technology platform.
Their point was that Python isn't always the right choice for a project
(e.g. when performance/scalability is of primary concern). I (obviously)
love Python, but I think this demonstrates Google's maturity; they deserve
credit for being flexible about their technology choices.

Hans

Marcel Esser

unread,
Feb 4, 2010, 11:59:22 AM2/4/10
to Washington, DC PHP Developers Group
I don't have much to add to this particular discussion except the
following: I've met some of the people who work as engineers at
Facebook, and they are some of the smartest people I know. They work
in an extremely competitive space, and adapt incredibly quickly. Until
proven otherwise, we should err on the side of competence and assume
HipHop to be A Good Thing.

- M.

Aaron Brazell

unread,
Feb 4, 2010, 12:02:18 PM2/4/10
to Sandy Smith, Washington, DC PHP Developers Group
On Thu, Feb 4, 2010 at 11:55 AM, Sandy Smith <ssm...@forumone.com> wrote:
Yeah, exactly. And keep in mind that Facebook *was* one of those companies that started in a Harvard dorm room by a student who wanted to create a social network for his school (before social networking really existed). This was not something that was developed because of a large degree of engineering experience, nor was it something that Zuck really anticipated would go beyond Harvard.

Number Two Site in the World. From a Dorm Room. At Harvard. This is the Real World. :)

Aaron Brazell 

Hans Lellelid

unread,
Feb 4, 2010, 12:02:43 PM2/4/10
to Aaron Brazell, Washington, DC PHP Developers Group
Fair enough. While I think we're approaching this from very different
directions, I think you've got lots of valid points. (And, to be fair, I
think your perspective is one much more in line with startups like Facebook
than more traditional software engineering projects. In *my* world HipHop
may be an indicator of bad design. In *your* world, it may be a perfectly
legitimate hail-mary when all else has been thrown at PHP scaling.)

I am still a little puzzled by how your I/O problems were going to be
solved by HipHop ... but I suppose that point is rather peripheral to the
main points in your discussion.

Hans


On Thu, 4 Feb 2010 11:40:47 -0500, Aaron Brazell <emmen...@gmail.com>
wrote:

Ray Paseur

unread,
Feb 4, 2010, 12:14:56 PM2/4/10
to Washington, DC PHP Developers Group
As someone who is currently engaged in software development by the "open
outcry" method - implementing marketing ideas as fast as possible to get
a first-to-the-field advantage, I have to admit that sometimes I just
build it and then look to see what I have built. Fred Brooks wrote in
his seminal book, The Mythical Man Month, "plan to throw one away." His
idea was that you are going to throw your first implementation away one
way or another, so you might be wise to build the inevitable into the
budget and timeline. Over the years I have found that no amount of
careful planning has enabled me to circumvent this almost immutable
discard step on anything but the smallest of projects. I now treat the
first implementation as a planning phase.

Great discussion, BTW.

Best to all, Ray

-----Original Message-----
From: washington-...@googlegroups.com
[mailto:washington-...@googlegroups.com] On Behalf Of Sandy
Smith
Sent: Thursday, February 04, 2010 11:56 AM
To: Washington, DC PHP Developers Group
Subject: Re: [dcphp-dev] Re: Facebook rewriting PHP?

-Sandy

--

Oscar Merida

unread,
Feb 4, 2010, 12:28:31 PM2/4/10
to Hans Lellelid, Aaron Brazell, jproffer, Washington, DC PHP Developers Group
> Here's my [final] attempt to clarify my point.  I think we would all agree
> that HipHop is a kludge.  It's a necessary kludge for Facebook because they
> committed themselves to a technology that could not scale to meet _their_
> needs.  I personally value cleanliness and the "right" solution.  This is
> often impractical, but it's still a valid opinion, I think.  Given the
> limitations of the system, I don't think rescuing PHP by cross-compiling it
> to C++ is the "right" way to do anything (other than keep using PHP).  I
> think it's too bad that the Facebook web application couldn't have been

Not much to add here except to say that it may not be a valid
conclusion to say that HIpHop means that PHP could not scale to meet
their needs. Obviously, they've gotten it to scale to this point, and
likely they can do so fairly easily by simply adding more servers
horizontally. In theory, they could do this forever but its really
expensive.

I think they're especially happy that they've gotten a near 2x
performance increase cause it sounds like they just cut their scaling
costs in half.

I wonder if they tried compiling any of their code into php
extensions. If so, the performance increases must not have been worth
the effort. Supposedly, C++ extensions were/are meant to speed up
any PHP app bottlenecks. See
http://www.sitepoint.com/blogs/2008/08/29/rasmus-lerdorf-php-frameworks-think-again/#

--
Oscar Merida
* http://OscarM.org - random thoughts on my blog
* http://SoccerBlogs.net - your daily soccer feed
* http://FutBoliviano.com - bolivian soccer, futbol de altura

Marcel Esser

unread,
Feb 4, 2010, 12:36:27 PM2/4/10
to washington-...@googlegroups.com
I don't think it's at all true that PHP didn't do the trick. It's not
like facebook didn't work. They just wanted to spend less, and they
obviously had the manpower available to Make Things Better.

I don't know where this utter myth comes from at all that PHP "isn't
scalable". It's probably one of the most scalable things you can
choose. It has healthy separation of concerns overall, and if you
wrote your app write, overcoming the problem of the actual PHP not
being fast enough is very easily solved by adding another web node.
It's also not as if it requires to this to unreasonable degrees. You
can, of course, get much better performance out of other
language/platform combinations, but there is an undeniable ease to
working with PHP. I would imagine this is why Facebook likes it. The
notion that you can easily get the same development pace from stricter
languages speaks either of inexperience with these issues, of
revanchism, or of sheer idiocy.

- M.

jproffer

unread,
Feb 4, 2010, 12:39:58 PM2/4/10
to Oscar Merida, Hans Lellelid, Aaron Brazell, Washington, DC PHP Developers Group
According to Keith, FB has been using this (or some variant of it) for the past year.  AFAIK, FB has been slow as molasses, at least in my experience.  Especially the past year. 

Not saying that's any kind of insight on their method's true performance when scaled to FB levels, as they may have been using it on just a few select servers rather than all. 

Just saying, just because it's from a monolith like FB, doesnt make it any kind of holy grail.  Will certainly look forward to people thoroughly testing its performance and limitations, and putting out benchmarks.

Some additional thoughts: I got a response to my concerns on relative paths since it was a compiled app, which basically indicated that it's a problem that would be addressed later.  However in hindsight I'm wondering if this isnt already solved, since apparently FB is already using this on their own servers? Everyone uploads tons of video/photos on a daily basis.  So they must have solved this already, otherwise I'm going to assume only part of their site is actually compiled.

Thoughts/insights?

Hans Lellelid

unread,
Feb 4, 2010, 1:01:16 PM2/4/10
to Marcel Esser, washington-...@googlegroups.com
> I don't think it's at all true that PHP didn't do the trick. It's not
> like facebook didn't work. They just wanted to spend less, and they
> obviously had the manpower available to Make Things Better.

I probably should have clarified that by "PHP", I mean what we know as
PHP. To me PHP involves assumptions about deployment, about the dynamic
features, about the Zend engine. HipHop is not the same thing; it just is
mostly compatible with PHP syntax. (I don't consider projects like Quercus
to be the same as "PHP" either.)

> I don't know where this utter myth comes from at all that PHP "isn't
> scalable". It's probably one of the most scalable things you can
> choose. It has healthy separation of concerns overall, and if you
> wrote your app write, overcoming the problem of the actual PHP not
> being fast enough is very easily solved by adding another web node.
> It's also not as if it requires to this to unreasonable degrees. You
> can, of course, get much better performance out of other
> language/platform combinations, but there is an undeniable ease to
> working with PHP. I would imagine this is why Facebook likes it. The
> notion that you can easily get the same development pace from stricter
> languages speaks either of inexperience with these issues, of
> revanchism, or of sheer idiocy.

I never said (or certainly didn't intend to) that PHP isn't/wasn't
scalable, just that it wasn't scalable to Facebook's needs. I think that
is affirmed quite simply by the fact that they had to write a
cross-compiler.

I don't think I ever said you could get the same development paces from
other languages (though, certainly, claiming that PHP is the most
productive language is very subjective -- and to me, wrong). You have to
factor in the time it took for them to write their cross-compiler into the
equation here. *That*, I think, is a critical part of discussing which was
the most efficient choice here.

Hans

Hans Lellelid

unread,
Feb 4, 2010, 1:11:44 PM2/4/10
to Sandy Smith, Washington, DC PHP Developers Group
> I think you're describing a scenario typical of large organizations with
> large budgets and stable, predictable use projections. Maybe that's not
> what all those projects were, but it sounds like the process typical of
> them.

Yeah, I agree. I'm talking about something different (and probably more
ideal than I can even really claim).

> That is not the environment of bootstrapped development with rapid
> success. Remember, the Facebook team has managed to handle all this
growth
> successfully. There are not many organizations who can say the same
given
> similar conditions (see Twitter or Boeing's newairplane.com), so I'm not
> sure criticizing them without a much better understanding of exactly how
> their components are put together (you can have loosely coupled PHP) is
> warranted.

Yeah, that's certainly fair. I'm sure they're an extremely smart group.
I think my suggested criticism is more a criticism of design choices that I
have witnessed (and often that I have been a part of making), which is
obviously in a different environment. Other than using Facebook and having
looked at some of their other, non-PHP open-source projects, I really don't
have any insight into their application. For me Facebook loads fast enough
and while on the surface it feels like a monolithic webapp, I do know that
they have different technologies driving different components.

> You're also speaking as if every engineer is able to fluidly switch
> technologies at the drop of a hat with zero transition cost and be
equally
> effective in the new paradigm, and that those costs can never outweigh
the
> benefits of the shift. You also assume they have time to calmly research
> and plan, rather than put out the next fire while trying to do what they
> can to scale without running into the mythical man month problem. Human
> systems are a bit more squishy and hard to benchmark than engineering
> systems. ;)

Well, that's all true. I'm not able to fluidly switch paradigms; there
are certainly context switching costs at a minimum. It would have been
expensive for Facebook to teach their employees to develop for a new
language/platform. I concur that this was a good strategy for them to
increase their performance. I think my original point, which I'm now
starting to forget, was just that they must have been planning for success
a lot earlier than they ultimately proved to be successful.

> The fact that you've been burned should tell you that software
development
> doesn't always happen as it should, and if you've been able to have most
of
> your projects work in the ideal way, you should be extremely thankful
for
> your luck. :)

Yes, that's fair. And I'm sure I have selective memory :) -- Not
everything has ever gone right, but I think that there's merit to planning
ahead, to reconsidering technology choices. Sometimes you do have to
"throw one away", that's part of an evolving product. I think it's hard
for a company to do that; it's certainly hard for me to look back on
something I wrote and say "you know, that wasn't a good idea", but I think
that's important. I see HipHop as an example of desperately trying *not*
to "throw one away".

Hans

Marcel Esser

unread,
Feb 4, 2010, 1:12:39 PM2/4/10
to Washington, DC PHP Developers Group
They didn't need to write a cross-compiler. That was an exploratory
project by one of their engineers that happened to turn out favorably
for cost effectiveness. I don't know where you got that impression. If
you read Haiping's post, he makes it perfectly clear that they are
aware of the performance trade-off - and they are ok with it for the
benefits. As I said before, it's not like Facebook didn't work before
they used hiphop. It worked fine. This is basically a bonus for them.

That being said, I question there being a trade-off at all. There is
one project I am working on right now where PHP is the actual
bottleneck, and not the database, I/O, the network, RAM, et cetera.
This was a hard situation to get into accidentally, and it's mainly
because we're doing a fair deal of math in PHP. I don't have the
manhours available right now to move that into C extensions, so that
in that sense hiphop will actually help me. But, back to my point, for
the other projects, and trust me, I'm working with some serious weight
in iron here, PHP was always the lesser problem. It's really easily to
write PHP code that runs slow, but it's also equally easy to write PHP
code, and build a stack that, runs really, really fast. It's largely a
question of competence. Miserable execution speeds of 30 requests per
second from a box aren't unheard of; but it's equally common to get
that to 1,000 requests a second with some optimization effort, or to
much higher numbers.

The truth of it is that most of the heavy lifting of complicated web
apps happens outside the frontend layer, and the frontend layer is
rarely heavily stressed.

Hans Lellelid

unread,
Feb 4, 2010, 1:25:24 PM2/4/10
to Marcel Esser, Washington, DC PHP Developers Group
Hi Marcel,

Good points. That's certainly true that FB worked fine before. I just
assume that the effort of rewriting the PHP engine was a significant effort
and that they wouldn't have funded it if it weren't important to them. So
maybe *need* is the wrong word, but clearly it was worth a lot to them.
And I assumed that they've been putting it into production because they
felt PHP without it wasn't good enough (or cheap enough) for their needs.
I suppose this is somewhat of a semantic game.

Yeah, it does sound like HipHop will help you in the case you describe.
And I certainly concur that being CPU-bound is a pretty rare thing in web
applications. Most of my challenges with PHP & scaling have had more to do
with the serving platform (Apache, mod_php, etc.) than PHP itself.

I don't think we're arguing about anything :)

Hans

On Thu, 4 Feb 2010 13:12:39 -0500, Marcel Esser <marcel...@gmail.com>
wrote:

D Keith Casey Jr

unread,
Feb 4, 2010, 1:55:19 PM2/4/10
to jproffer, Washington, DC PHP Developers Group
jproffer wrote:
> According to Keith, FB has been using this (or some variant of it) for
> the past year. AFAIK, FB has been slow as molasses, at least in my
> experience. Especially the past year.

Actually, I was a bit misinformed on that one. As an amendment to that
statement. It turns out that it was under development for the past year, in
production use the last 6 months or so.


In terms of site performance, poke around with YSlow and/or some of the API's
and you'll find where the problems are. The vast majority of it is at the
browser level with rendering and the requests themselves.

kc

--
D. Keith Casey, Jr.
CEO, CaseySoftware, LLC
http://CaseySoftware.com

Ray Paseur

unread,
Feb 4, 2010, 3:27:08 PM2/4/10
to Washington, DC PHP Developers Group
Let me be the first to plow a new field. Anybody have any experience
with CakePHP? War stories, gotchas, etc.? Just looking for a rapid
prototyping tool. Best to all, Ray

Joshua Boyd

unread,
Feb 4, 2010, 3:23:47 PM2/4/10
to Ray Paseur, Washington, DC PHP Developers Group
I played with Cake some, I ended up using symfony as it seemed to have less of a learning curve and more usage. Also, Doctrine integrates really easily.

Now I'm playing with lightvc and coughphp, but that's probably not what you're looking for.

--
You received this message because you are subscribed to the Google
Group: "Washington, DC PHP Developers Group" - http://www.dcphp.net
To post, send email to washington-...@googlegroups.com
To unsubscribe, send email to washington-dcphp-...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/washington-dcphp-group?hl=en

Zvi Band

unread,
Feb 4, 2010, 3:25:49 PM2/4/10
to Ray Paseur, Washington, DC PHP Developers Group
CakePHP is easy to develop with, but the performance is miserable. I've used CodeIgniter with great success.

On Thu, Feb 4, 2010 at 3:27 PM, Ray Paseur <rpa...@nationalpres.org> wrote:
--
You received this message because you are subscribed to the Google
Group: "Washington, DC PHP Developers Group" - http://www.dcphp.net
To post, send email to washington-...@googlegroups.com
To unsubscribe, send email to washington-dcphp-...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/washington-dcphp-group?hl=en



--
---
Zvi Band
http://zviband.com
http://hi.im/zvi
202-683-7803
ASAP? https://awayfind.com/zvi

Shaun Farrell

unread,
Feb 4, 2010, 3:31:57 PM2/4/10
to Zvi Band, Ray Paseur, Washington, DC PHP Developers Group
I 2nd the CodeIgniter recommendation by Zvi.   This was our framework of choice before moving to ZF.

FYI - We will not be discussing the Framework wars at the next DCPHP.  It's a war that will never be won!  Maybe over some beers we can all do this.

Shaun J. Farrell
Washington, DC
(202) 713-5241
www.farrelley.com

Aaron Brazell

unread,
Feb 4, 2010, 3:44:18 PM2/4/10
to Ray Paseur, Washington, DC PHP Developers Group
On Thu, Feb 4, 2010 at 3:27 PM, Ray Paseur <rpa...@nationalpres.org> wrote:
Let me be the first to plow a new field.  

Ohhhhhh.... bad choice of words for a pre-18-24" snowstorm, Ray. Bad choice of words. Maybe you need to hire a publicist? ;)
 

Dan Drinkard

unread,
Feb 4, 2010, 3:44:19 PM2/4/10
to Shaun Farrell, Zvi Band, Ray Paseur, Washington, DC PHP Developers Group
Wait, Facebook is REWRITING PHP??? WE NEED TO TALK ABOUT THIS!!

...sorry.
Dan Drinkard
703.298.3014

D Keith Casey Jr

unread,
Feb 4, 2010, 4:09:26 PM2/4/10
to Shaun Farrell, Zvi Band, Ray Paseur, Washington, DC PHP Developers Group
Shaun Farrell wrote:
> I 2nd the CodeIgniter recommendation by Zvi. This was our framework of
> choice before moving to ZF.

The CakePHP team has some weird stuff going on right now... it looks like
CakeDC was taking the reigns in terms of running things, doing training, and
generally commercializing things. In terms of the team itself, Nate Abele,
Joel Perras, and the project lead (don't remember his name) all split and
started a new project called Lithium.

I'm not speaking ill of the project... just seems like some stuff is odd.

jhilgeman

unread,
Feb 5, 2010, 8:30:49 PM2/5/10
to Washington, DC PHP Developers Group
CakePHP is okay. Pretty bad performance, though. A couple months ago,
I threw together a site on my personal testing domain, www.devunit.com,
using the latest CakePHP at the time. It's about as lightweight as can
be and switching pages still takes 1-2 seconds, which is pretty bad
considering that it's really not doing almost anything except loading
up a semi-blank page and connecting to a local DB server that doesn't
get any traffic.

I read that you can use accelerators to help speed it up, and there
are some tweaking tips, but like someone else said, CodeIgniter
usually performs better out of the box. I'm starting to see some CI on
some decently-large sites, too, with pretty good performance.

The difference in syntax between cake and CI is negligible - I don't
really see any benefits to Cake.

jhilgeman

unread,
Feb 5, 2010, 9:12:39 PM2/5/10
to Washington, DC PHP Developers Group
> > While I appreciate, and always have appreciated, your frank, honest,
> high
> > level view of PHP, web security, web applications, etc., you strike me
> as
> > somewhat naive and puritanical.
>
> ;)  Well, that's my missionary-kid upbringing, I'm sure.

I'm an MK as well, but I'm reserving judgment on this whole thing so I
can later claim I was on the "winning" side the whole time. :)

SK Rana

unread,
Feb 6, 2010, 9:51:50 AM2/6/10
to washington-...@googlegroups.com

class extends Rays_question
{

echo 'Why MVC should be used at first place? Some really key benefits are: ' . $answer1;
# single point of entry or SEO, at least not important for me

echo 'How learning involved is worth adapting any MVC (say CakePHP): ' . $answer2;
# Authentication or DB_connection or whatever - one can perhaps do faster directly than learning new culture/tricks/style.

echo 'Will one be hostage to MVC provider till life of web project? They upgrade you upgrade: ' . $answer3;
# I remember phpbb upgrade issues

echo 'Code maintenance (by different developer down the road) - would that be easier? ' . $answer4;
# I tried to fix/enhance code in MVC framework of predecessor - couple of years ago though

echo 'Is MVC framed Code easy to understand? ' . $answer5;

D Keith Casey Jr

unread,
Feb 6, 2010, 11:57:01 AM2/6/10
to SK Rana, washington-...@googlegroups.com
SK Rana wrote:
> class extends Rays_question

E_FATAL

You have to name your class. ;)

Hans Lellelid

unread,
Feb 6, 2010, 12:07:27 PM2/6/10
to washington-...@googlegroups.com
# Dear subclass,
#
# I appreciate your dedication to PHP syntax. And your use
# of shell-style comments, despite being discouraged by PEAR.
# Take that, silly PEAR!
#
# I'm going to answer post-question, so I fear it won't parse. :)

> echo 'Why MVC should be used at first place? Some really key benefits are: ' . $answer1;
> # single point of entry or SEO, at least not important for me

$answer1 = <<< EOF

Technically, single-point-of-entry and MVC are not the same thing,
though typically in PHP they go hand-in-hand (in that MVC kinda requires
a single-point-of-entry). Front controllers in PHP tend to be a
discussion topic (and debate) in and of themselves since PHP's natural
deployment paradigm is not to use a single entry point -- so you have to
do something that feels a little unnatural in PHP to get there.

See http://www.phppatterns.com/docs/design/the_front_controller_and_php
for more on front controllers in PHP.

For me, the front controller is important in PHP frameworks because
there is going to be a lot of boilerplate setup/framework code that is
going to be identical for every request. Even "framework-free" PHP apps
will some sort of "require 'init.php';" at the top of the script to
initialize the environment. A front controller, though, makes it
possible to have a *single* request/response pipeline. Now all your
framework setup, request filtering, response rendering logic, and error
handling can happen in *one* place (well, at least the errors you can
catch/handle). So that "one place" is the key reason [that I would] use
a front controller: DRY.

MVC is just an organization paradigm. Once you've got your single
request/response system, you are probably itching to think about how to
efficiently organize your application code. Typically that's where some
form of "MVC" comes in. Different frameworks tend to disagree on
exactly where the lines get drawn between those different letters. Some
frameworks have different acronyms, but the basic idea is separation of
concerns.

EOF;

> echo 'How learning involved is worth adapting any MVC (say CakePHP): ' . $answer2;
> # Authentication or DB_connection or whatever - one can perhaps do faster directly than learning new culture/tricks/style.

$answer2 = <<< EOF

Um, sure it's often faster to just hack something together rather than
think about it. Where frameworks save time is when you reuse
components. For example, maybe you come to discover that the code
needed to "edit user" has a lot of duplication with the code needed to
"save user" -- perhaps they both have to parse and filter request data,
potentially query database for an existing user. So, for example,
having a SaveUserAction extend a EditUserAction would probably be an
effective way to reuse all that initialization code. Or what if you
decide that you'd like to be able to return your list of articles as an
RSS feed instead of an HTML template? The strength of frameworks is
that they [can] greatly lessen the amount of code needed to add new
features; simplify maintainability; and (by virtue of fewer code paths)
make it easier to debug, test and secure your application.

EOF;

> echo 'Is MVC framed Code easy to understand? ' . $answer5;

# Yes? I have no idea. "Are books easy to understand?"
# It may depend on the author?

Hans

Joshua Boyd

unread,
Feb 6, 2010, 12:41:38 PM2/6/10
to washington-...@googlegroups.com
php_value_auto_prepend_file and php_value_auto_append_file are always fun hacks when you inherit a project without an MVC =P

--
You received this message because you are subscribed to the Google
Group: "Washington, DC PHP Developers Group" - http://www.dcphp.net
To post, send email to washington-...@googlegroups.com
To unsubscribe, send email to washington-dcphp-...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/washington-dcphp-group?hl=en



--

jproffer

unread,
Feb 6, 2010, 1:04:26 PM2/6/10
to Hans Lellelid, washington-...@googlegroups.com
That's a very good explanation.  I would add that, if you haven't already had experience working with or designing a MVC model, some people find it to be very frustrating and confusing.  But stick with it - as with anything, it gets easier, and you'll appreciate the benefits.
Also keep in mind, every model has its place.  The MVC model isn't really beneficial to small projects - it starts to show its colors in large and very large projects. (sort of what Hans already said in a roundabout way ;))
Reply all
Reply to author
Forward
0 new messages