sagenb/data reorganization?

[Pat LeSmithe]

This is rough, especially since I'm not familiar with several
components. Absolutely no offense is intended toward any component or
its proponents.

I propose mapping sagenb/data/ to http://localhost:8000/data/ or a
similarly-named base resource (lib?). Inside sagenb/data/:

3d/
gears/
highlight/
jmol/
jquery/
plugins/
dimensions/
farbtastic/
jqueryui/
jsmath/
sage/
css/
html/
notebook/
worksheets/
images/
js/
sage3d/
sage3d.js
tiny_mce/
zorn/

Questions and suggestions (relative to sagenb/data):

* Equal treatment for templates and non-templates alike. In
particular, how about using sage/ as the base "template" directory?

* We have three copies of prettify.css. I suggest just keeping a copy
in highlight/, along with prettify.js.

* Do we use zorn?

* Can we reconcile sage3d.js and sage3d/sage3d.js? How do these relate
to 3d/? Can we house them all in sage3d/ or, perhaps, under sage/?

* Although we have almost every file under version control, I suggest
minimizing changes to third-party directories, to ease upgrades. For
example, we can put our TinyMCE and jsMath setup code in
sage/js/tinymce.js and sage/js/jsmath.js, respectively. The latter, if
it contains a list of macro inserted on-the-fly, we can serve virtually
(see twist.py) from that location. Of course, some changes, such as
those to jquery/plugins/jquery.event.extendedclick.js might be better to
do in place.

* Don't serve sage/js/*_lib.js as main.js (see twist.py). Instead,
serve just the necessary pieces separately. For example: async.js,
notebook.js, canvas3d.js, jmol.js. (Later, we might reorganize further
and load parts on-demand, possibly with Ajile: http://ajile.net/) There
are now two template macros in notebook.js:

* SAGE_URL: A place to get jsMath fonts. The link
(http://sage.math.washington.edu/sage/jsmath) is broken. For now, I
suggest hard-coding
http://www.math.union.edu/~dpvc/jsMath/users/welcome.html

* KEY_CODES: I suggest serving these, along with other potential
user customizations, separately/virtually (see twist.py) as
data/sage/js/keys.js (or config.js).

Does anyone have experience with the YUI compressor
(http://developer.yahoo.com/yui/compressor/)? We could use it offline
to compress our static sage/js files.

[Pat LeSmithe]

Pat LeSmithe wrote:
> Does anyone have experience with the YUI compressor
> (http://developer.yahoo.com/yui/compressor/)? We could use it offline
> to compress our static sage/js files.

Some numbers:

'min' : YUI compressor
'clean' :
sagenb.notebook.compress.compress.JavaScriptCompressor().getClean()
'pack' :
sagenb.notebook.compress.compress.JavaScriptCompressor().getPacked()

/bin/gzip-ped:

230 async_lib.min.js.gz
348 async_lib.js.gz
385 async_lib.clean.js.gz
487 async_lib.pack.js.gz
577 jmol_lib.min.js.gz
615 jmol_lib.js.gz
675 jmol_lib.clean.js.gz
868 jmol_lib.pack.js.gz
2192 canvas3d_lib.min.js.gz
2391 canvas3d_lib.clean.js.gz
2581 canvas3d_lib.pack.js.gz
4272 canvas3d_lib.js.gz
10324 notebook_lib.min.js.gz
11476 notebook_lib.clean.js.gz
11707 notebook_lib.pack.js.gz
29547 notebook_lib.js.gz

not-gzipped:

279 async_lib.min.js
606 async_lib.clean.js
689 async_lib.pack.js
729 async_lib.js
1004 jmol_lib.min.js
1183 jmol_lib.js
1196 jmol_lib.clean.js
1357 jmol_lib.pack.js
5394 canvas3d_lib.pack.js
6458 canvas3d_lib.min.js
7070 canvas3d_lib.clean.js
14804 canvas3d_lib.js
31506 notebook_lib.pack.js
40627 notebook_lib.min.js
48397 notebook_lib.clean.js
119798 notebook_lib.js

Overall, the YUI compressor + gzip appears to have a slight edge. The
YUI compressor is also actively developed and its output, according to
its home page, should work safely in multiple browsers.

There are some safety issues, presumably for old browsers, mentioned at
the top of

compress/JavaScriptCompressor.py

I couldn't find a more recent version of this compressor.

Details:

java -jar yuicompressor-2.4.2.jar file.js > file.min.js

from compress.JavaScriptCompressor import JavaScriptCompressor

def pack(base):
"""

returns one or more JavaScript code packed, using getClean and

obfuscating output

"""
s = open(base + '.js', 'r').read()
s = JavaScriptCompressor().getPacked(s)
open(base + '.pack.js', 'w').write(s)

def clean(base):
"""

returns one or more JavaScript code without comments, by default

removes some spaces too

"""
s = open(base + '.js', 'r').read()
s = JavaScriptCompressor().getClean(s)
open(base + '.clean.js', 'w').write(s)

files = [
'async_lib',
'notebook_lib',
'jmol_lib',
'canvas3d_lib'
]

map(pack, files)
map(clean, files)

[Pat LeSmithe]

Pat LeSmithe wrote:
> gears/
> highlight/
> jmol/
> jquery/
> plugins/
> dimensions/
> farbtastic/
> jqueryui/
> jsmath/
> sage/
> css/
> html/
> notebook/
> worksheets/
> images/
> js/
> sage3d/

> tiny_mce/
> zorn/

The patches at

http://trac.sagemath.org/sage_trac/ticket/7196

set up the hierarchy above for sagenb/data.

> * Although we have almost every file under version control, I suggest
> minimizing changes to third-party directories, to ease upgrades. For
> example, we can put our TinyMCE and jsMath setup code in
> sage/js/tinymce.js and sage/js/jsmath.js, respectively. The latter, if

I haven't created tinymce.js and jsmath.js, but I can make these part of

http://trac.sagemath.org/sage_trac/ticket/6459
http://trac.sagemath.org/sage_trac/ticket/4714

> * Don't serve sage/js/*_lib.js as main.js (see twist.py). Instead,
> serve just the necessary pieces separately. For example: async.js,
> notebook.js, canvas3d.js, jmol.js. (Later, we might reorganize further
> and load parts on-demand, possibly with Ajile: http://ajile.net/) There
> are now two template macros in notebook.js:

I haven't done this, although v2 at

http://trac.sagemath.org/sage_trac/ticket/7110

goes part of the way.

[William Stein]

Hi,

I've merged 7196 in, fixed some issues that arose (not your fault),
and pushed the result to
http://trac.sagemath.org/sage_trac/ticket/7196 and to the repo.

Anybody who is working on the notebook should upgrade to 0.3.1.
Note that you can't *just* do a pull from the repo since you have to
move the data/java/jmol directory to data/jmol manually, since jmol is
not under revision control. I think that is the only issue though.
Oh, probably there will be several empty directories if you just pull.
So you may want to just install 0.3.1 and pull from your own old
repo.

Note that 0.3.1 (i.e., the reorganized code) is *not* included in
sage-4.1.2, since I'm worried it may introduce bugs. However,
everybody should update to this, since it's a significant
reorganization of the data and we don't want to have patches against
the old data layout.

-- William

[Pat LeSmithe]

Still to do...

* Parens matching:

http://trac.sagemath.org/sage_trac/ticket/3646

* Canvas3d + Pre3d:

http://groups.google.com/group/sage-devel/browse_thread/thread/0315e138d5783377/742d9f9bb00b0cd7?#742d9f9bb00b0cd7

* Simplify/update jsMath loaders for notebook and docs:

http://trac.sagemath.org/sage_trac/ticket/4714
http://trac.sagemath.org/sage_trac/ticket/6673

* jQuery / UI upgrade:

http://trac.sagemath.org/sage_trac/ticket/5447

* Move TinyMCE setup code to data/sage/js/tinymce.js:

http://trac.sagemath.org/sage_trac/ticket/6459

[Pat LeSmithe]

Pat LeSmithe wrote:
> * Parens matching:
> http://trac.sagemath.org/sage_trac/ticket/3646

There's now a rebased patch for #3646.

> * Move TinyMCE setup code to data/sage/js/tinymce.js:
> http://trac.sagemath.org/sage_trac/ticket/6459

There's now a rebased patch for #6459.

> * jQuery / UI upgrade:
> http://trac.sagemath.org/sage_trac/ticket/5447

This and #6459 both affect

sagenb/data/sage/html/notebook/head.tmpl

so it would be great to get a verdict or further guidance on #6459,
before I rebase #5447.

There's a rebased notebook registration CAPTCHA patch at

http://trac.sagemath.org/sage_trac/ticket/7158

There's a rebased notebook settings page patch at

http://trac.sagemath.org/sage_trac/ticket/4551

Also: There's a new, #7196-adjusted jsMath font spkg at

http://trac.sagemath.org/sage_trac/ticket/7229

[Pat LeSmithe]

An update on rebased / reviewable sagenb tickets:

Starting from #7196 (actually, revision 114), the queue is

Notebook registration CAPTCHA
http://trac.sagemath.org/sage_trac/ticket/7158

Notebook settings page
http://trac.sagemath.org/sage_trac/ticket/4551

Parens matching
http://trac.sagemath.org/sage_trac/ticket/3646

TinyMCE line break + move init to tinymce.js
http://trac.sagemath.org/sage_trac/ticket/6459

jQuery / UI upgrade
http://trac.sagemath.org/sage_trac/ticket/5447

Notes

* #3646 should commute with the others.

* #5447 depends on #6459 *only* to the extent that sagenb-specific
TinyMCE code is now in tinymce.js.

* Bug: The jsMath macros are *not* loaded in 4.1.2 because of circular
imports.

* Other jsMath tickets:

http://trac.sagemath.org/sage_trac/ticket/4714
http://trac.sagemath.org/sage_trac/ticket/6673

[William Stein]

On Fri, Oct 16, 2009 at 1:02 PM, Pat LeSmithe <qed...@gmail.com> wrote:
>
> An update on rebased / reviewable sagenb tickets:
>
> Starting from #7196 (actually, revision 114), the queue is
>
> Notebook registration CAPTCHA
> http://trac.sagemath.org/sage_trac/ticket/7158
>
> Notebook settings page
> http://trac.sagemath.org/sage_trac/ticket/4551
>
> Parens matching
> http://trac.sagemath.org/sage_trac/ticket/3646
>
> TinyMCE line break + move init to tinymce.js
> http://trac.sagemath.org/sage_trac/ticket/6459
>
> jQuery / UI upgrade
> http://trac.sagemath.org/sage_trac/ticket/5447

I've now reviewed and applied all of the above patches to sagenb, and
pushed them. I'm making a 0.3.2 release of sagenb that contains all
of the above.
It's especially good that the notebook settings/user management code
is finally in the notebook, finally we can insert newlines in tinymce,
and finally paren matching!

Bravo!

William

>
>
> Notes
>
> * #3646 should commute with the others.
>
> * #5447 depends on #6459 *only* to the extent that sagenb-specific
> TinyMCE code is now in tinymce.js.
>
> * Bug: The jsMath macros are *not* loaded in 4.1.2 because of circular
> imports.
>
> * Other jsMath tickets:
>
> http://trac.sagemath.org/sage_trac/ticket/4714
> http://trac.sagemath.org/sage_trac/ticket/6673
>
>

--
William Stein
Associate Professor of Mathematics
University of Washington
http://wstein.org