New automatic build script for pyv8 and v8

[Flier Lu]

As you know, Google v8 disable RTTI and C++ exception by default after
v3.0, but pyv8 and boost::python need it.

So, I just try to update the setup.py, which will automatic check out
the v8 code, patch its SConstruct file, and build it as a private
build.

Please try to use it from the SVN trunk, and give me your feedback :)

[mcot]

Hi, one thing missing is debug mode. V8 needs to be compiled with
mode=debug. I think the previous build process already tries to link
against v8_g if debug is set to True.

[mcot]

Also, my pyv8 build failed complaining about PIC even though I had
done:
export CCFLAGS=-fPIC

But it worked when I patched v8's SConstruct with:

'gcc': {
'all': {
'CCFLAGS': ['$DIALECTFLAGS', '$WARNINGFLAGS', '-fPIC'],
'CXXFLAGS': ['$CCFLAGS']
},

I am not sure why this happened. If someone else could verify on
Linux x64 with v8 build options: scons mode=debug snapshot=on arch=x64
library=static, PyV8 buildconf options: debug = True.

As long as you are modifying v8's SConstruct from python it might be
more reliable to just add -fPIC from there.

[Flier Lu]

Got it, I have added some code to support the debug and x64 mode,
please verify the modification with SVN trunk after r333.

Thanks

[smthmlk]

There are lots of things wrong with the setup.py since this update,
and I've created a patch that tries to do what you intended a bit more
gracefully. I tested this on Ubuntu 10.10 x86_64 and Fedora 14 x86_64
and issuing simply `python setup.py build' runs flawlessly. There is a
new feature here designed to help users avoid running into linking/
compiler issues by using the latest v8 + the latest pyv8, where the
two are seemingly independent--there is a V8_REVISION global variable
that you, Flier, should set to the V8 revision of the source code that
you know your code compiles successfully against (runtime issues are
another issue...).

Applies from inside the pyv8 trunk, `patch -p1 < pathToPatch.patch'.
Patch file contents:

diff -urN pyv8-orig/setup.py pyv8/setup.py
--- pyv8-orig/setup.py 2011-02-08 13:23:41.871401003 -0800
+++ pyv8/setup.py 2011-02-08 13:23:55.701401002 -0800
@@ -17,6 +17,7 @@
PYTHON_HOME = None
V8_HOME = None
V8_SVN_URL = "http://v8.googlecode.com/svn/trunk/"
+V8_REVISION = 6583
INCLUDE = None
LIB = None
DEBUG = False
@@ -43,9 +44,8 @@

if V8_HOME is None or not os.path.exists(os.path.join(V8_HOME,
'include', 'v8.h')):
print "WARN: V8_HOME doesn't exists or point to a wrong folder, ",
- print "we will try to checkout and build a private build from
<http://code.google.com/p/v8/>."
-
- V8_HOME = os.path.join(PYV8_HOME, 'build', 'v8')
+ print "we will try to export and build a private build from <%s>" %
V8_SVN_URL
+ V8_HOME = os.path.join(PYV8_HOME, 'v8_r%d' % V8_REVISION)
else:
print "Found Google v8 base on V8_HOME <%s>, update it to latest SVN
trunk" % V8_HOME

@@ -174,58 +174,110 @@
else:
print "ERROR: unsupported OS (%s) and platform (%s)" % (os.name,
sys.platform)

+
+
class build(_build):
+
def checkout_v8(self):
+ global V8_HOME
+
+ # Do we need to checkout?
if not os.path.isdir(V8_HOME):
- os.makedirs(V8_HOME)
+ # Yes.

- cmd = "co"
- else:
- cmd = "up"
+ # We have to checkout from SVN, so we have some additional
dependencies: pysvn
+ # To avoid alienating curious developers, lets fail gracefully if
they don't have this--they probably don't.
+ try:
+ from pysvn import Client, Revision, opt_revision_kind
+ except Exception, e:
+ print "ERROR: checkout_v8(): could not import pysvn. Ensure you
have the pysvn package installed.\nOn debian/ubuntu, this is called
'python-svn'; on Fedora/RHEL, this is called 'pysvn'."
+ sys.exit(-1)

- proc = subprocess.Popen(["svn", cmd, V8_SVN_URL, V8_HOME],
- stdout=sys.stdout, stderr=sys.stderr)

- proc.communicate()

- if proc.returncode != 0:
- print "WARN: fail to checkout or update Google v8 code from SVN,
error code: ", proc.returncode
+ # Exporting the specified revision of the v8 source from its svn
repo
+ # Why export? Because we will almost certainly NOT be committing
to this from here, nor will we be updating our
+ # local copy, so to hell with it.
+ svnClient = Client()
+
+ print "INFO: Exporting v8 source, revision %d, to %s" %
(V8_REVISION, V8_HOME)
+ r = svnClient.checkout(V8_SVN_URL, V8_HOME,
revision=Revision(opt_revision_kind.number, V8_REVISION))
+
+ # the return value is a Revision instance;
+ if r is None:
+ print "ERROR: checkout_v8(): Failed to export from V8 svn
repository"
+ # We can't continue. Exit here
+ sys.exit(-1)

- def build_v8(self):
- scons = os.path.join(V8_HOME, "SConstruct")
+ else:
+ print "INFO: Successfully exported v8"

- with open(scons, 'r') as f:
- build_script = f.read()
+ else:
+ # No.
+ print "INFO: V8 is already present in '%s'" % V8_HOME

- fixed_build_script = build_script.replace('-fno-rtti',
'').replace('-fno-exceptions', '')

- if build_script == fixed_build_script:
- print "INFO: skip to patch the Google v8 SConstruct file "
+ def build_v8(self):
+ # We'll have to compile using `scons': do we have scons installed?
+ try:
+ p = subprocess.Popen(['scons', '--version'], shell=False,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+ sconsOUT, sconsERR = p.communicate()
+
+ if sconsOUT and 'Knight' in sconsOUT:
+ print "INFO: scons is installed; we can at least attempt to build
v8"
+ else:
+ print "ERROR: checkout_v8(): scons doesn't appear to be installed
or work correctly: %s" % sconsERR
+ sys.exit(-1)
+ except Exception, e:
+ print "ERROR: checkout_v8(): scons doesn't appear to be installed:
%s" % e
+ print "You must have the 'scons' package installed to build v8
(called 'scons' on debian/ubuntu/fedora/RHEL)"
+ sys.exit(-1)
+
+ # Next up, we have to patch the SConstruct file from the v8 source
to remove -no-rtti and -no-exceptions
+ # CXXFLAGS.
+ sconstructFilePath = os.path.join(V8_HOME, "SConstruct")
+
+ # Check if we need to patch by searching for rtti flag in the data
+ sconstructFileContents = open(sconstructFilePath, 'r').read()
+
+ if '-fno-rtti' in sconstructFileContents or '-fno-exceptions' in
sconstructFileContents:
+ print "INFO: Applying patch to v8's SConstruct file"
+
+ try:
+ p = subprocess.Popen(['patch','-p', '0', '-i', 'v8.patch',
sconstructFilePath], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+ sout, serr = p.communicate()
+ except Exception, e:
+ print "ERROR: build(): caught exception trying to patch v8
SConstruct file: %s" % e
+ sys.exit(-1)
+ else:
+ if serr:
+ print "ERROR: build(): couldn't apply patch to v8's SConstruct
file: %s" % serr
+ else:
+ print "INFO: patched successfully"
+
else:
- print "INFO: patch the Google v8 SConstruct file to enable RTTI
and C++ Exceptions"
+ print "INFO: patch already applied to v8's SConstruct file"

- os.rename(scons, scons + '.bak')

- with open(scons, 'w') as f:
- f.write(build_script)
+ # Continue with setting up parameters for scons
+ x64 = [k for k, v in macros if k == 'V8_TARGET_ARCH_X64']

- x64 = [k for k, v in macros if k == 'V8_TARGET_ARCH_X64']
+ mode = 'debug' if DEBUG else 'release'
+ arch = 'x64' if x64 else 'ia32'
+ snapshot = 'on'

- mode = 'debug' if DEBUG else 'release'
- arch = 'x64' if x64 else 'ia32'
+ if x64 and os.name != 'nt':
+ os.putenv("CCFLAGS", "-fPIC")

- if x64 and os.name != 'nt':
- os.putenv("CCFLAGS", "-fPIC")
+ print "INFO: building Google v8 with SCons for %s platform,
snapshot=%s" % (arch, snapshot)

- print "INFO: building Google v8 with SCons for %s platform" %
arch
+ proc = subprocess.Popen(["scons", "arch="+arch, "mode="+mode,
"snapshot="+snapshot],
+ cwd=V8_HOME, shell=False, stdout=sys.stdout, stderr=sys.stderr)

- proc = subprocess.Popen(["scons", "arch="+arch,
"mode="+mode], cwd=V8_HOME, shell=True,
- stdout=sys.stdout, stderr=sys.stderr)
-
- proc.communicate()
+ proc.communicate()

- if proc.returncode != 0:
- print "WARN: fail to build Google v8 code from SVN, error code: ",
proc.returncode
+ if proc.returncode != 0:
+ print "WARN: fail to build Google v8 code from SVN, error code:
", proc.returncode

def run(self):
self.checkout_v8()
diff -urN pyv8-orig/v8.patch pyv8/v8.patch
--- pyv8-orig/v8.patch 1969-12-31 16:00:00.000000000 -0800
+++ pyv8/v8.patch 2011-02-08 13:24:00.041400975 -0800
@@ -0,0 +1,22 @@
+--- SConstruct.bak 2011-02-08 12:44:52.921401002 -0800
++++ SConstruct 2011-02-08 12:48:35.471400999 -0800
+@@ -135,8 +135,7 @@
+ },
+ 'gcc': {
+ 'all': {
+- 'CCFLAGS': ['$DIALECTFLAGS', '$WARNINGFLAGS'],
+- 'CXXFLAGS': ['$CCFLAGS', '-fno-rtti', '-fno-exceptions'],
++ 'CCFLAGS': ['$DIALECTFLAGS', '$WARNINGFLAGS']
+ },
+ 'visibility:hidden': {
+ # Use visibility=default to disable this.
+@@ -467,8 +466,7 @@
+ },
+ 'gcc': {
+ 'all': {
+- 'LIBPATH': ['.'],
+- 'CCFLAGS': ['-fno-rtti', '-fno-exceptions']
++ 'LIBPATH': ['.']
+ },
+ 'os:linux': {
+ 'LIBS': ['pthread'],

[Flier Lu]

Sorry for the immature setup script, I have merged part of it to the
SVN trunk code

There are some differences between your patch and the trunk code

1. The SVN trunk code of pyv8 will build with the trunk code of v8 by
default, but it could be set to build with a special revision with
V8_SVN_REVISION, I will update the number when pyv8 release a new
version or made a new tag
2. pyv8 will checkout/update SVN code with pysvn first, but it still
try to use the system 'svn' command when pysvn is missing
3. pyv8 still use string.replace to apply the patch, because the patch
could be different for x86/x64 and linux/windows

Thanks very much for your patch, please verify it with SVN trunk code