A new model for UXF development
7 views
Skip to first unread message
Mark Birbeck
Dec 10, 2009, 3:04:36 PM12/10/09
to ubiquity-...@googlegroups.com
Hello all,
I wonder if anyone who has a free moment might take a look at backplanejs:
<http://backplanejs.googlecode.com>
It's essentially a merge of all of the different projects that we
(Backplane) had under the Ubiquity banner -- our SMIL, Yowl, RDFa, and
backplane modules are now all under this one roof.
But it contains many new tools, as well as some source reorganisation,
that I'd like to bring back to UXF, so would appreciate it if people
would start looking at the changes.
BACKGROUND
backplanejs came about mainly because someone recently said to me that
they had no idea where they were supposed to start with all of our JS
libraries. :)
But it's also been motivated by work we are doing for a customer that
is using UXF in a real live solution; we needed to be able to make
changes more quickly than we can in UXF, so we felt it was important
to have our own area.
For example, a big concern I've had for a while is that if someone is
going to deploy UXF on the web, then UXF needs to be fast. To this end
I needed to be able to experiment with different techniques, to make
sweeping changes to the tools, to reorganise directories, and so on.
I'm sure I don't need to say this, but I hope nobody fears we may be
creating a fork. We're absolutely going to bring the changes back to
the UXF project -- assuming everyone agrees on them, of course -- and
this email is really about me starting a process of discussion going,
about some of the changes that could be coming down the line.
MERCURIAL
The first big change you'll notice is that we're using Mercurial.
Be warned...we're now evangelists...the Kool-aid has been tasted!
Being able to have your own local version of the source, that you can
experiment within, take your own branches from, and generally do
whatever you like with, is incredibly liberating.
(And don't even get me started on the joys of patch queues.)
As discussed on one of our telecons, we'd like to move UXF to
Mercurial at some point, too. We wouldn't do that until everyone feels
comfortable, but it might be worth using the backplanejs project as a
way to start playing with the tools. (Which is essentially what we
have done.)
THE ELEVATION OF ROLL-UP FILES...
The next big change is that roll-up files are not just something we
create at deployment time in order to speed up loading -- they are now
an integral part of the development cycle.
You create a roll-up file every time you make a change (see TOOLS,
below), and then you retry your forms. Even the unit-tests are
rolled-up into a single file, which incidentally has the advantage of
meaning that we don't miss out any tests -- I discovered a few test
files that aren't being loaded in the current UXF unit-test framework.
Whilst the rolled-up file is used for testing, when it comes to
deployment, we create a compressed version.
(Actually, we create two roll-up files -- there's also one for CSS,
which really helps with performance too.)
...AND THE DEMISE OF YUI LOADER
Consequently, we don't need a run-time loader. This means I've
completely removed the YUI loader from the equation, and many of those
timing issues that we used to have are gone.
TOOLS
Everything now goes through Ant -- no more of those annoying batch
files that only work on Windows. (Or was it only me that was annoyed?)
There's a new build.xml that has tasks for creating the roll-up files,
compressing them, combining them into a zip file, uploading them to
Google Code, starting a local test-server (see GOOGLE APP ENGINE,
below), and so on.
Most of the tools work without installation, because the key
components have have been included in a special tools directory, which
developers get automatically -- the two that do require installation
are Sprockets and Google App Engine.
SPROCKETS
The best tool I found for creating concatenated files is Sprockets,
which requires Ruby. It's not a big job to install this, if you're on
Windows, so I think it's reasonable to require this. (If you're on a
Mac you have Ruby already.) However, I do have some ideas about how to
remove this requirement in the future.
GOOGLE APP ENGINE
The other tool that does need to be installed, is the Google App
Engine SDK. Although you could use any local server, such as an Apache
one, the Google App Engine has some powerful advantages.
Google's App Engine is actually a cloud-based service which is free to
use, up to a certain limit of bandwidth. But the reason I've
incorporated it into backplanejs, is because the SDK provides a local
version of this cloud environment, which is very easy to configure.
Once the App Engine is installed, it's possible to launch a local
web-server from the command-line, and this command-line can specify a
path to use as the root directory for the server. This makes it very
easy to include scripts and files within the VCS that will be picked
up when the server runs, and makes it exceptionally easy to run this
from Ant.
For example, the current configuration file adds a path to the
unit-tests, so tests can be run, like this:
<http://localhost:8080/unit-tests/xforms/main.html>
<http://localhost:8080/unit-tests/rdfa/main.html>
and so on.
(It may be possible to do this with other servers, such as Apache, but
I couldn't find information on how to do it, without adding something
to the Apache configuration files.)
I should mention that the reason I've introduced a server into the
heart of the development process, is because I think it's important to
develop in an environment that is as close as possible to where your
forms will be running. This means rolling-up the files and then
serving them up over a web-server. It means that the effects of
caching and file size are there from the beginning, rather than
appearing just at the last minute, before you deploy an important
project.
DEPLOYMENT FOR AUTHORS
For people who aren't working on the library, and simply want to use
it, they will go to this page:
<http://code.google.com/p/backplanejs/wiki/DeployingBackplanejs>
You'll see that authors are referred to the download page, where there
is an archive that contains the compressed JS and CSS files, plus the
supporting assets (images, XBL and HTC files).
Even this is a big step forward, since in the past we didn't actually
know which files were needed to get a deployment to work!
I'd appreciate it if people could look at this wiki page and
subsequent download, as if you were deploying the library, and tell me
if anything is missing. (I realise that the zip file should probably
include a simple form, a readme and so on, and I'll sort that out
soon.)
SIZE AND PERFORMANCE
And hopefully you'll be pleased to see that the final, compressed,
file is 680 Kb, which is not unreasonable, given (a) that it includes
RDFa and SMIL support, and (b) that it includes the YUI library.
(I've also compared it to other XForms JS libraries, and it's not too bad. :))
As you're probably aware, loading happens much more quickly if you
include everything in one file, rather than loading the files
separately, hence the inclusion of the YUI files in the roll-up.
I'm certain that the size of this can be reduced quite a bit, once we
get into that mode of working, and of course a UXF build of the
library wouldn't need RDFa and SMIL.
NEXT STEPS
The next thing I'm going to do is set the tools up so that the
concatenation step takes a parameter of the platform to use (YUI,
Dojo, etc.). This will allow people to start work on supporting Dojo.
Also, we should look at adding some Python scripts to the App Engine
area, to allow all testing to take place on an individual's machine,
rather than relying on the web. For example, we could create local
versions of the xformstest.org echo scripts.
And then of course, I'd really like to feed back this framework, and
the associated tools, to UXF so that we can make use of these changes
'over here'.
I'd appreciate any feedback, as we start to prepare for convergence.
Thanks.
Regards,
Mark
--
Mark Birbeck, webBackplane
mark.b...@webBackplane.com
http://webBackplane.com/mark-birbeck
webBackplane is a trading name of Backplane Ltd. (company number
05972288, registered office: 2nd Floor, 69/85 Tabernacle Street,
London, EC2A 4RR)
I wonder if anyone who has a free moment might take a look at backplanejs:
<http://backplanejs.googlecode.com>
It's essentially a merge of all of the different projects that we
(Backplane) had under the Ubiquity banner -- our SMIL, Yowl, RDFa, and
backplane modules are now all under this one roof.
But it contains many new tools, as well as some source reorganisation,
that I'd like to bring back to UXF, so would appreciate it if people
would start looking at the changes.
BACKGROUND
backplanejs came about mainly because someone recently said to me that
they had no idea where they were supposed to start with all of our JS
libraries. :)
But it's also been motivated by work we are doing for a customer that
is using UXF in a real live solution; we needed to be able to make
changes more quickly than we can in UXF, so we felt it was important
to have our own area.
For example, a big concern I've had for a while is that if someone is
going to deploy UXF on the web, then UXF needs to be fast. To this end
I needed to be able to experiment with different techniques, to make
sweeping changes to the tools, to reorganise directories, and so on.
I'm sure I don't need to say this, but I hope nobody fears we may be
creating a fork. We're absolutely going to bring the changes back to
the UXF project -- assuming everyone agrees on them, of course -- and
this email is really about me starting a process of discussion going,
about some of the changes that could be coming down the line.
MERCURIAL
The first big change you'll notice is that we're using Mercurial.
Be warned...we're now evangelists...the Kool-aid has been tasted!
Being able to have your own local version of the source, that you can
experiment within, take your own branches from, and generally do
whatever you like with, is incredibly liberating.
(And don't even get me started on the joys of patch queues.)
As discussed on one of our telecons, we'd like to move UXF to
Mercurial at some point, too. We wouldn't do that until everyone feels
comfortable, but it might be worth using the backplanejs project as a
way to start playing with the tools. (Which is essentially what we
have done.)
THE ELEVATION OF ROLL-UP FILES...
The next big change is that roll-up files are not just something we
create at deployment time in order to speed up loading -- they are now
an integral part of the development cycle.
You create a roll-up file every time you make a change (see TOOLS,
below), and then you retry your forms. Even the unit-tests are
rolled-up into a single file, which incidentally has the advantage of
meaning that we don't miss out any tests -- I discovered a few test
files that aren't being loaded in the current UXF unit-test framework.
Whilst the rolled-up file is used for testing, when it comes to
deployment, we create a compressed version.
(Actually, we create two roll-up files -- there's also one for CSS,
which really helps with performance too.)
...AND THE DEMISE OF YUI LOADER
Consequently, we don't need a run-time loader. This means I've
completely removed the YUI loader from the equation, and many of those
timing issues that we used to have are gone.
TOOLS
Everything now goes through Ant -- no more of those annoying batch
files that only work on Windows. (Or was it only me that was annoyed?)
There's a new build.xml that has tasks for creating the roll-up files,
compressing them, combining them into a zip file, uploading them to
Google Code, starting a local test-server (see GOOGLE APP ENGINE,
below), and so on.
Most of the tools work without installation, because the key
components have have been included in a special tools directory, which
developers get automatically -- the two that do require installation
are Sprockets and Google App Engine.
SPROCKETS
The best tool I found for creating concatenated files is Sprockets,
which requires Ruby. It's not a big job to install this, if you're on
Windows, so I think it's reasonable to require this. (If you're on a
Mac you have Ruby already.) However, I do have some ideas about how to
remove this requirement in the future.
GOOGLE APP ENGINE
The other tool that does need to be installed, is the Google App
Engine SDK. Although you could use any local server, such as an Apache
one, the Google App Engine has some powerful advantages.
Google's App Engine is actually a cloud-based service which is free to
use, up to a certain limit of bandwidth. But the reason I've
incorporated it into backplanejs, is because the SDK provides a local
version of this cloud environment, which is very easy to configure.
Once the App Engine is installed, it's possible to launch a local
web-server from the command-line, and this command-line can specify a
path to use as the root directory for the server. This makes it very
easy to include scripts and files within the VCS that will be picked
up when the server runs, and makes it exceptionally easy to run this
from Ant.
For example, the current configuration file adds a path to the
unit-tests, so tests can be run, like this:
<http://localhost:8080/unit-tests/xforms/main.html>
<http://localhost:8080/unit-tests/rdfa/main.html>
and so on.
(It may be possible to do this with other servers, such as Apache, but
I couldn't find information on how to do it, without adding something
to the Apache configuration files.)
I should mention that the reason I've introduced a server into the
heart of the development process, is because I think it's important to
develop in an environment that is as close as possible to where your
forms will be running. This means rolling-up the files and then
serving them up over a web-server. It means that the effects of
caching and file size are there from the beginning, rather than
appearing just at the last minute, before you deploy an important
project.
DEPLOYMENT FOR AUTHORS
For people who aren't working on the library, and simply want to use
it, they will go to this page:
<http://code.google.com/p/backplanejs/wiki/DeployingBackplanejs>
You'll see that authors are referred to the download page, where there
is an archive that contains the compressed JS and CSS files, plus the
supporting assets (images, XBL and HTC files).
Even this is a big step forward, since in the past we didn't actually
know which files were needed to get a deployment to work!
I'd appreciate it if people could look at this wiki page and
subsequent download, as if you were deploying the library, and tell me
if anything is missing. (I realise that the zip file should probably
include a simple form, a readme and so on, and I'll sort that out
soon.)
SIZE AND PERFORMANCE
And hopefully you'll be pleased to see that the final, compressed,
file is 680 Kb, which is not unreasonable, given (a) that it includes
RDFa and SMIL support, and (b) that it includes the YUI library.
(I've also compared it to other XForms JS libraries, and it's not too bad. :))
As you're probably aware, loading happens much more quickly if you
include everything in one file, rather than loading the files
separately, hence the inclusion of the YUI files in the roll-up.
I'm certain that the size of this can be reduced quite a bit, once we
get into that mode of working, and of course a UXF build of the
library wouldn't need RDFa and SMIL.
NEXT STEPS
The next thing I'm going to do is set the tools up so that the
concatenation step takes a parameter of the platform to use (YUI,
Dojo, etc.). This will allow people to start work on supporting Dojo.
Also, we should look at adding some Python scripts to the App Engine
area, to allow all testing to take place on an individual's machine,
rather than relying on the web. For example, we could create local
versions of the xformstest.org echo scripts.
And then of course, I'd really like to feed back this framework, and
the associated tools, to UXF so that we can make use of these changes
'over here'.
I'd appreciate any feedback, as we start to prepare for convergence.
Thanks.
Regards,
Mark
--
Mark Birbeck, webBackplane
mark.b...@webBackplane.com
http://webBackplane.com/mark-birbeck
webBackplane is a trading name of Backplane Ltd. (company number
05972288, registered office: 2nd Floor, 69/85 Tabernacle Street,
London, EC2A 4RR)
Jack Jansen
Dec 21, 2009, 7:43:47 AM12/21/09
to ubiquity-...@googlegroups.com
On 10 dec 2009, at 21:04, Mark Birbeck wrote:
> Hello all,
>
> I wonder if anyone who has a free moment might take a look at backplanejs:
>
> <http://backplanejs.googlecode.com>
>
> It's essentially a merge of all of the different projects that we
> (Backplane) had under the Ubiquity banner -- our SMIL, Yowl, RDFa, and
> backplane modules are now all under this one roof.
I've been playing with this, since I finally have some time again to spend on the SMIL stuff.
I have two questions (so far):
- The smil animation examples don't work. I got the impression that the script and stylesheet references were incorrect, so I modified them to be like those in the other examples (which do work, sometimes after updating the backplanejs version number), but no go. Tried in both Safari and Firefox (both on Mac).
- What is the preferred way of extending backplanejs with third-party functionality, such as my SMIL engine? One possible path (which I think I prefer, because it keeps things separate) is to create my SMIL engine besides the backplanejs stuff, create my own sprockets sourcefile listing my private dependencies along with references to the backplanejs stuff and supply my own antfile to glue it all together. Another way is to insert my stuff into the third_party directory and add some references to build/backplane.js. What is best?
--
Jack Jansen, <Jack....@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma Goldman
Reply all
Reply to author
Forward
0 new messages