Some tech questions

28 views
Skip to first unread message

Chris Graham

unread,
Feb 5, 2010, 7:30:12 AM2/5/10
to HipHop for PHP Dev
Hi,

Static analysis of PHP is something I've worked on in the past, and
there were a few problems I came across that made me realise how
difficult it can be.

I have a few questions OTTOMH...

1) How do you identify type? Clearly you can validate against the PHP
API functions, but what about functions within a codebase? Do you use
phpdoc comments to help with this?
2) Is a variable taken out of an array always a variant? It's not
AFAIK possible to statically identify something coming out of an array
as there's no good way to annotate them (they'd be a struct or a class
in C++, which would be nicely annotated).
3) Does Hip Hop PHP compile cleanly on all platforms? (Mac, Windows,
Linux)
4) A few people have commented that 50% performance speedup seems very
low for what should be achievable. Is there any more that can be said
about this?

Thanks! Really looking forward to seeing this...

chri...@gmail.com

unread,
Feb 5, 2010, 7:37:05 AM2/5/10
to hiphop-...@googlegroups.com
Hi Chris,
If you watch the lecture by haiping you will find that he actually covers the issues regarding type inference.
While he doesn't explicitly answer you second question, he does cover the over all jist of how its achieved.

Chris.
Sent from my BlackBerry® wireless device

Dharmendra Tolani

unread,
Feb 5, 2010, 8:57:51 AM2/5/10
to hiphop-...@googlegroups.com
I still don't know, how to get started with hiphop?
Are there any binaries which I could download?
On this page:
http://developers.facebook.com/news.php?blog=1&story=358#
there is a link which should apparently point to a wiki,
but it simply points to :
http://github.com

Sikander Iqbal

unread,
Feb 5, 2010, 9:44:05 AM2/5/10
to hiphop-...@googlegroups.com
It's not out yet.

--
Sikander Iqbal
siq...@gmail.com

Igor MILOVANOVIC

unread,
Feb 5, 2010, 9:48:08 AM2/5/10
to hiphop-...@googlegroups.com
This thing is about to be open sourced... We all are waiting for this
to happen...
Patience, my friend... ;)

On 5 February 2010 14:57, Dharmendra Tolani <dharmend...@gmail.com> wrote:
>
> I still don't know, how to get started with hiphop?
> Are there any binaries which I could download?
> On this page:
> http://developers.facebook.com/news.php?blog=1&story=358#
> there is a link which should apparently point to a wiki,
> but it simply points to :
> http://github.com

--
Igor Milovanović
http://www.linkedin.com/in/igormilovanovic

http://feeds.feedburner.com/f13o
http://www.inforepublic.org/author/pletisan/

Christopher Jones

unread,
Feb 5, 2010, 10:35:23 AM2/5/10
to hiphop-...@googlegroups.com
> From: Chris Graham <ch...@ocportal.com>
> Date: Fri, 5 Feb 2010 04:30:12
> To: HipHop for PHP Dev<hiphop-...@googlegroups.com>
> Subject: Some tech questions
>
> Hi,
>
> Static analysis of PHP is something I've worked on in the past, and
> there were a few problems I came across that made me realise how
> difficult it can be.
>
> I have a few questions OTTOMH...
>
> 1) How do you identify type? Clearly you can validate against the PHP
> API functions, but what about functions within a codebase? Do you use
> phpdoc comments to help with this?

FB decided against phpdoc early on. It's not particularly accurate nor
specific enough.

My notes from the PHP Summit say:
"Type inference is iterative, stopping when reached static point. Avg
seven passes. References are variants. Inside arrays are variants."
(A few other notes are at
http://blogs.oracle.com/opal/2010/02/facebooks_hphp_initial_comment.html
but Terry Chay covers more at
http://terrychay.com/article/hiphop-for-faster-php.shtml )

> 2) Is a variable taken out of an array always a variant? It's not
> AFAIK possible to statically identify something coming out of an array
> as there's no good way to annotate them (they'd be a struct or a class
> in C++, which would be nicely annotated).
> 3) Does Hip Hop PHP compile cleanly on all platforms? (Mac, Windows,
> Linux)

Linux only.

> 4) A few people have commented that 50% performance speedup seems very
> low for what should be achievable. Is there any more that can be said
> about this?

I'm sure FB is working hard on improving optimization techniques.
At the launch you could see Haiping's brain thinking about optimization
techniques when the topic of V8 came up.

My personal view is that a lot of page generation time is consumed by, for
example, database access.

Chris Graham

unread,
Feb 5, 2010, 10:43:55 AM2/5/10
to HipHop for PHP Dev
Thanks Chris, interesting!

With regard to phpdoc, I developed a system for my company that does
static type checking (for a lint) using that, and you're right, it's
not accurate. What I then did is I made a new version of the Zend PHP
that gave errors when types didn't match up, which allowed me to fix a
lot of the problems. I also set up a call listener function for when
functions were executed and added in some run-time guards to check
against the phpdoc-defined types. End result was a pretty reliable
base.
I can't wait to see what Facebook have done, but I am a bit worried my
code will be unnecessarily littered with Variant's and thus
performance affected. I've worked to strict typing in my 300k codebase
pretty much hoping we'd get to the point we're now at (but also it
helped me weed out a few security holes when I was assuming integers
for what were actually strings, for example) so it's fantastic to see
all this.

I'm also wondering if they are going to allow casting. I had
contemplated the idea of abusing PHP's language-level cast feature to
work as type-hinting for code conversion. E.g. you type hint when you
extract from an array, to help the static analysis along.

On Feb 5, 3:35 pm, Christopher Jones <cjbjo...@gmail.com> wrote:
> > From: Chris Graham <ch...@ocportal.com>
> > Date: Fri, 5 Feb 2010 04:30:12
> > To: HipHop for PHP Dev<hiphop-...@googlegroups.com>
> > Subject: Some tech questions
>
> > Hi,
>
> > Static analysis of PHP is something I've worked on in the past, and
> > there were a few problems I came across that made me realise how
> > difficult it can be.
>
> > I have a few questions OTTOMH...
>
> > 1) How do you identify type? Clearly you can validate against the PHP
> > API functions, but what about functions within a codebase? Do you use
> > phpdoc comments to help with this?
>
> FB decided against phpdoc early on.  It's not particularly accurate nor
> specific enough.
>
> My notes from the PHP Summit say:
> "Type inference is iterative, stopping when reached static point.  Avg
> seven passes.  References are variants.  Inside arrays are variants."

> (A few other notes are athttp://blogs.oracle.com/opal/2010/02/facebooks_hphp_initial_comment.html

chri...@gmail.com

unread,
Feb 5, 2010, 11:24:34 AM2/5/10
to hiphop-...@googlegroups.com
There are too many people called Chris in this mailing list.
I'm already getting confused! Ha.
Chris.

Sent from my BlackBerry® wireless device

-----Original Message-----

Sergey Chernyshev

unread,
Feb 5, 2010, 11:52:14 AM2/5/10
to hiphop-...@googlegroups.com
That's if you still have database calls which is not true for some who
already optimized that far.

Sergey

--
Sent from my mobile device


Thank you,

Sergey


--
Sergey Chernyshev
http://www.sergeychernyshev.com/

Mike Curry

unread,
Feb 5, 2010, 11:42:02 AM2/5/10
to hiphop-...@googlegroups.com
Try being Mike... I was "Mike CU" in grade 5... there was also a Mike C, Mike CA. and a Mike Y

John Mullanaphy

unread,
Feb 5, 2010, 3:30:57 PM2/5/10
to hiphop-...@googlegroups.com
Greetings everyone,

Definitely intrigued about HipHop although don't think my company is quite ready to worry about PHP optimization right now. Currently getting everyone going forward with optimizing MySQL and utilizing memcached better. With HipHop very curious on seeing the conformity my code. I don't use eval() however one of my coworkers does so that'd be a task for another day.

In relation to Chris and Mike,
As a John personally just go by my lastname mostly, Mullanaphy.

Thanks,
John Mullanaphy

P.S. Have to give a big thank you to the developers at Facebook as well. Must be a hard albeit fun project to work on.

sgerrand

unread,
Feb 6, 2010, 7:37:14 AM2/6/10
to HipHop for PHP Dev
Why not just port the code base to a language that actually supports
strong data typing and inference like, say, Python?

It sounds like you're building a lot of scaffolding around an
inherently weak foundation.

Chris Graham

unread,
Feb 6, 2010, 8:15:04 AM2/6/10
to HipHop for PHP Dev
In my case, my company develops a "for the joe public" CMS (http://
ocportal.com/) that has to run on any random end-users web hosting,
and most web hosting doesn't support python. I'm not 100% happy with
the PHP language myself, but we need to work within the environment
that's out there.
Besides, I wouldn't use the weak foundation analogy - we now develop
on our own stricter custom version of PHP, in a sense we actually
replaced the foundation, to make it type strict.

Chris Graham

unread,
Feb 7, 2010, 4:54:11 AM2/7/10
to HipHop for PHP Dev
I remembered a '5' which was also a big issue for me when thinking
about this stuff in the past...

In PHP you often are intentionally weak typed. For example, a function
will return 'false' or 'NULL' often, which is kind of like an error
signal. In C, of course, it would be done with a reference parameter,
or a constant set within the bounds of the normal expected return
type.
I hope it's smart enough to handle this without just reverting to
variants, although I'm not sure how it could do it.

Le Quoc Viet

unread,
Feb 7, 2010, 5:52:05 AM2/7/10
to hiphop-...@googlegroups.com
Hi all,

There was not a single talk and thought about it, so I think I would
kick off one. Here goes the scenario:

1. I have a PHP library in its full meaning:
1.1. The code only has classes and functions
1.2. No embedding text or HTML that disrupts PHP code by closing tag
?>. Only pure PHP
1.3. There is a single <?php tag only in each file (please look at
Zend Framework or Code Igniter code
1.4. There might be inc/ini/xml/yaml files for configuration

2. There is no invocation of CGI features at all ($_SERVER, $_POST,
$_GET, ...)

3. I want to convert it to C++ and use generated code as a C++ library
without HipHop built-in server

4. Let's take ZendPdf or TCPDF for example. It would be unnecessary and
weird to have a server as a hindrance. I don't need a server to handle
I/O. What I need is just an application to run background or GUI
application without opening any socket or port.

Is that possible with HipHop? Please kindly advise. Thank you very much!

Best regards,
Viet.

Martijn

unread,
Feb 7, 2010, 6:35:49 AM2/7/10
to HipHop for PHP Dev
Of what is currently said, this is *not* possible. Its only possible
to
compile complete applications where a framework is included but not
only
the framework. I dont know or a CLI frontend is already support but
probally isnt it very hard to add such frontend. If you want only
convert a library to a php extension PHC suits more your needs but as
i
have seen its not (much) faster then the php version of the code. The
problem is only that PHC is never finished. It was a one man job and
is
currently missing active contributors. It works but didnt reach it
goals
such as be faster then the ZE.

Martijn

Le Quoc Viet

unread,
Feb 7, 2010, 6:59:56 AM2/7/10
to hiphop-...@googlegroups.com
Hi,

I don't want a PHP extension. I want a library written in pure PHP to be
converted to C++. Not sure what HipHop developers will say. Thanks anyway.

Regards,
Viet.

Platonides

unread,
Feb 7, 2010, 9:35:54 AM2/7/10
to hiphop-...@googlegroups.com
Le Quoc Viet wrote:
> Hi,
>
> I don't want a PHP extension. I want a library written in pure PHP to be
> converted to C++. Not sure what HipHop developers will say. Thanks anyway.
>
> Regards,
> Viet.

They said that you could edit the generated C++, so I guess you can also
remove the web server from that and compile it with a different frontend.

Haiping Zhao

unread,
Feb 7, 2010, 3:35:05 PM2/7/10
to hiphop-...@googlegroups.com
Not exactly sure what you want, but we have what we called "FFI", foreign
language interface, that allows to build .a and .so from PHP
files/libraries. Then other languages can call into your functions. We made
it available to Java and Python and any other languages supporting SWIG. Of
course, calling from C++ is automatically supported ;-)

-Haiping

Haiping Zhao

unread,
Feb 7, 2010, 3:49:21 PM2/7/10
to hiphop-...@googlegroups.com
Very quickly,

1) We can't count on comments being accurate :-)
2) Yeah.
3) Right now, only Linux.
4) That 50% is Facebook's number and it really depends on portion of work in
extensions (which are not optimized that much at all) vs. portion of work
spending in programming constructs. So, it's hard for anyone to predict
what's the number it will give to your web site/app.

-Haiping

Samuel ROZE

unread,
Feb 7, 2010, 4:18:44 PM2/7/10
to hiphop-...@googlegroups.com
Can we understand that it will be possible to make PHP extensions
(loaded using the "extension" ini configuration) from a PHP code,
compiled by your "FFI", which is a part of HipHop ?

These PHP extensions could also be used in another C++ project?

If I'm right, that's extremely interesting! Am I right?

-Samuel

Haiping Zhao

unread,
Feb 7, 2010, 5:25:04 PM2/7/10
to hiphop-...@googlegroups.com
That's theoretically possible, but we don't have support for that yet. Why
though? Why not just use those PHP files as is in another PHP project?
Better distribution with just one single binary?

-Haiping

Manuel Lemos

unread,
Feb 7, 2010, 5:32:24 PM2/7/10
to hiphop-...@googlegroups.com
Hello,

on 02/07/2010 08:25 PM Haiping Zhao said the following:


> That's theoretically possible, but we don't have support for that yet. Why
> though? Why not just use those PHP files as is in another PHP project?
> Better distribution with just one single binary?

The PHC PHP compiler generates PHP extensions from compiled PHP code
that can be used with a regular PHP distribution. I suppose that is what
Samuel wants.

--

Regards,
Manuel Lemos

Find and post PHP jobs
http://www.phpclasses.org/jobs/

PHP Classes - Free ready to use OOP components written in PHP
http://www.phpclasses.org/

Haiping Zhao

unread,
Feb 7, 2010, 7:29:06 PM2/7/10
to hiphop-...@googlegroups.com
Ahh. We can't do the same thing yet. This is a similar problem to bridging
PHP and HipHop compiler. Phc uses Zend, so it's a lot easier to do this
work, as they share the same execution environment. Globals and resources
can be maintained back and forth. That wouldn't be the case in HipHop. While
we can compile some PHP into Zend-compatible extensions, we may not be able
to keep globals and resources in-sync.

-Haiping

Le Quoc Viet

unread,
Feb 7, 2010, 8:20:38 PM2/7/10
to hiphop-...@googlegroups.com
Hi,

Thanks for your reply. I thought I wrote it very clearly. Ok what I want is converting these:


<?php

require('some_file.php');

function add($a, $b)
{
    return (
a + b);
}

class SimpleClass
{

   

   

    static $count = 0;

   
    public __construct()
    {
    }

   
    public __destruct()
    {
    }

   

   

   

    protected draw()
    {
    }
}


#include "some_file.h"

double add(
double a, double b)
{
    return (a + b)
}

class SimpleClass
{
    public:
   
    static long $count = 0;

   
   
SimpleClass()
    {
    }

   
   
~SimpleClass()
    {
    }
   
    protected:
   
    draw()
    {
    }
}

long SimpleClass::count = 0;



By having this, I could translate pure PHP functions and classes to pure C++ functions in classes. Is that possible with current HipHop? Please kindly advise. Thanks.

Regards,
Viet.


Haiping Zhao wrote:

Manuel Lemos

unread,
Feb 7, 2010, 8:30:49 PM2/7/10
to hiphop-...@googlegroups.com
Hello,

on 02/07/2010 10:29 PM Haiping Zhao said the following:


> Ahh. We can't do the same thing yet. This is a similar problem to bridging
> PHP and HipHop compiler. Phc uses Zend, so it's a lot easier to do this
> work, as they share the same execution environment. Globals and resources
> can be maintained back and forth. That wouldn't be the case in HipHop. While
> we can compile some PHP into Zend-compatible extensions, we may not be able
> to keep globals and resources in-sync.

Yes, phc uses Zend because it is intended to generate extensions to the
regular PHP engine, but the actual PHP compiled code runs about 40%
faster than the same code executed by Zend Engine.

It seems that PHC performs clever optimization similar to those there
were mentioned in HipHop presentation. So I think it would probably not
be so hard to add whatever is necessary to make HipHop do the same as PHC.

Haiping Zhao

unread,
Feb 7, 2010, 8:32:23 PM2/7/10
to hiphop-...@googlegroups.com
Possibly. We should explore into this...

Haiping Zhao

unread,
Feb 7, 2010, 8:31:28 PM2/7/10
to hiphop-...@googlegroups.com
Yes. Converted code has some prefix. “add” becomes “f_add”, and “SimpleClass” becomes “c_simpleclass”, just to avoid collisions. But you can easily typedef yourself to their original names.

-Haiping

Le Quoc Viet

unread,
Feb 7, 2010, 10:02:05 PM2/7/10
to hiphop-...@googlegroups.com
That sounds good. Thanks for clarification.

Xin Qi

unread,
Feb 7, 2010, 8:50:13 PM2/7/10
to HipHop for PHP Dev
Hi Viet,

The generated binary by HipHop doesn't have to be run as a server,
although it has the server capability linked in. If you run it not in
the server mode, it would be just like any normal command-line
application.

Hope that answers your question.

Best,
-Xin

Le Quoc Viet

unread,
Feb 8, 2010, 10:03:54 AM2/8/10
to hiphop-...@googlegroups.com
Hi Xin Qi,

Thank you very much for further clarification. I hope when HipHop is
released, I can convert pure PHP libraries to C++ without a sweat.

Regards,
Viet.

Craig Lance

unread,
Mar 10, 2010, 9:04:36 AM3/10/10
to hiphop-...@googlegroups.com
Regarding #1: Are there many PHP library functions that return either FALSE or an array?  If this is common, we might think of a special solution as apposed to using a variant.  I was thinking of returning a NULL, but I know that's not a cut and dry solution.

-=- Craig A. Lance
--
No trees were destroyed in the sending of this message,
however, a significant number of electrons were terribly inconvenienced.

-- unknown

Chris Graham

unread,
Mar 10, 2010, 9:13:57 AM3/10/10
to hiphop-...@googlegroups.com
Because PHP often uses false as a special signal for failure most PHP functions will return a possible false instead of the normal data type.
I like what you say Craig. I don't see a big reason why HPHP could not identify that a certain type is never used natively as NULL and hence remap any comparisons of it as false booleans to NULL instead, and hence not have to use a variant. I am assuming that HPHP uses object pointers for it's data types, I'm not looking at the generated code right now to check that. One problem would be that if the value is passed on outside code scope (e.g. to another function) then it could get nasty and hence HPHP couldn't do the optimisation - but...
a) There will be lots of optimisable cases where that does not happen
b) If a programmer type-hinted that call using a cast, it wouldn't happen either


Chris Graham, Managing Director, ocProducts Ltd

Experts in dynamic and interactive websites

Phone (switchboard): 0114 209 6208 (9am-5pm, UK time)

Mark

unread,
Mar 10, 2010, 12:21:57 PM3/10/10
to HipHop for PHP Dev
On Mar 10, 6:13 am, Chris Graham <ch...@ocproducts.com> wrote:
> Because PHP often uses false as a special signal for failure most PHP functions will return a possible false instead of the normal data type.
> I like what you say Craig. I don't see a big reason why HPHP could not identify that a certain type is never used natively as NULL and hence remap any comparisons of it as false booleans to NULL instead, and hence not have to use a variant. I am assuming that HPHP uses object pointers for it's data types, I'm not looking at the generated code right now to check that.

It depends on the type. Array vs FALSE you can use NULL. But you could
have a function that returns Array, or NULL or FALSE. Or even int or
FALSE. And then there's no out of band value available.

Even so, a struct containing an int, and an isFalse flag, is /much/
cheaper to manipulate than a Variant.

Work on this type of optimization is in the pipeline... stay tuned.

Mark

Reply all
Reply to author
Forward
0 new messages