collaboratively developing javascript files in a packaged tiddlyweb vertical.

[simon mcmanus]

In our weekly TiddlySpace hackdays I have been working on of the TiddlyWiki client side macros.

So far I have been doing this is the following way :

1 .. git clone the tiddlyspace package repo into the folder "tiddlyspace_package" from here :

http://github.com/FND/tiddlyspace

2 ..  I then create a  simple TiddlyWebWiki instance using the devstore in the same folder that contains the tiddlyspace_package dir.

3 .. I then go in to edit the tiddlywebconfig.py file of the dev instance so that it points to the (tiddlywiki) recipe which exists in the (src) folder of the tiddlyspaces package.

4 .. At the same the same time i have the TiddlyWiki recipe listed in the spaces instance.py (http://github.com/FND/tiddlyspace/blob/master/tiddlywebplugins/tiddlyspace/instance.py)

This means that when others in my team build a spaces distribution and install it my changes are integrated.

While this works it does not allow me to test any spaces specific integration between the client and the server.  e.g. Calling the create space script on the server.

The primary focus of development for spaces will be javascript code. In the coming weeks there will be other developers working on the client side code for spaces. This will make it far more important that each developer is using the latest code in source control so  I would like to find an easy way for us all to keep the spaces client code up to date on our local machines.

At the moment we have created a build script which makes it slightly easier to deploy a version of spaces (http://github.com/FND/tiddlyspace/blob/master/build.sh).

Which means the following steps are required to see everyone elses changes in the browser :

1 .. git pull
2 .. ./build.sh
wait wait wait
3 .. refresh the browser

This works, but takes its time and also takes copies of all the files specified in instancer.py.

I wonder how I could take advantage of the benifits of the devstore (change .js/.tid/.recipe files and then immediately see the change in the browser after a refresh.) when working with a packaged tiddlyweb vertical?

Ideally I would do this with :

1 .. git pull
2 .. refresh the  browser

I guess we could do this by keeping a special spaces/devstore instance in git which contains a duplicate of all the required client and server functionality but  that sounds like a bad solution.

While in this case I am particularly talking about spaces I have the same problem with developing against the python TiddlyDocs package.

Thanks for any suggestions

Simon

[jnthnlstr]

Hi Simon,

For your packaged vertical, where is it looking for the static files,
such as JS?

It seems like what you could do is create a symlink between your
development directory and the static folder used for the vertical.

I have a situation with tiddlywebplugins.wikidata, where I am needing
to tweak JavaScript and test it, without going through a build and
deploy cycle. My "static" directory in the package git repo is
actually a symlink to /tiddlywebplugins/wikidata/static, which is
where the production server looks for all the static content.

Does this make any sense in your context?

J.

On Feb 10, 6:01 pm, simon mcmanus <mcmanus.si...@gmail.com> wrote:
> In our weekly TiddlySpace hackdays I have been working on of the TiddlyWiki
> client side macros.
>
> So far I have been doing this is the following way :
>
> 1 .. git clone the tiddlyspace package repo into the folder
> "tiddlyspace_package" from here :
>
> http://github.com/FND/tiddlyspace
>
> 2 ..  I then create a  simple TiddlyWebWiki instance using the devstore in
> the same folder that contains the tiddlyspace_package dir.
>
> 3 .. I then go in to edit the tiddlywebconfig.py file of the dev instance so
> that it points to the (tiddlywiki) recipe which exists in the (src) folder
> of the tiddlyspaces package.
>
> 4 .. At the same the same time i have the TiddlyWiki recipe listed in the

> spaces instance.py (http://github.com/FND/tiddlyspace/blob/master/tiddlywebplugins/tiddly...

[Michael Mahemoff]

At the moment we have created a build script which makes it slightly easier to deploy a version of spaces (http://github.com/FND/tiddlyspace/blob/master/build.sh).

It's also worth noting the build script has to do several things that seem like they could be improved if I knew how:

(a) cacher is run every time; this is something where I should probably add a flag to the script so it's not automatically run

(b) setting up symlinks and manually adding an "import mangler"

(c) running twanager userpass to add passwords, as the instancer doesn't let you set passwords. Automation is the goal here, to make developers' lives as easy as possible, and if passwords can't be set automatically, we're saying every time a developer wants to test anything authorisation-related (can I see the space someone else created?), they have to manually run twanager.

(d) run twanager twimport to populate with test content from a URL. Again, it would be good if we could put this into instancer, and idealy from a local source; I'm not sure if it's possible.

Our main goal with build.sh would be to remove all dependencies on remote sources (other than established pip packages, e.g. tiddlywebwiki) during the development cycle. Cutting down steps, i.e. twanager userpass, would also be a great help.

[FND]

> I wonder how I could take advantage of the benifits of the devstore
> (change .js/.tid/.recipe files and then immediately see the change in
> the browser after a refresh.) when working with a packaged tiddlyweb
> vertical?

I'll look into further simplifying the devstore, making it easier to
use with any vertical. (The twinstance_dev script currently only
supports tiddlywebwiki - that should be easy to fix, now that we have
some tangible use cases.)

There's a more basic review of instancer (which provides the framework
for "twebicality"-style verticals) on the agenda as well, which should
lead to further clarification. (Here, too, the requirements are much
clearer now than during the original conception.)

> It's also worth noting the build script has to do several things that
> seem like they could be improved if I knew how

While a slightly separate issue, this is all part of the bigger picture,
so I expect that the instancer review and related efforts will lead to
more clarity on this front as well.

I'll be working on these tasks over the next few days - will keep you in
the loop as things progress.

-- F.

[chris...@gmail.com]

On Wed, 10 Feb 2010, jnthnlstr wrote:

> For your packaged vertical, where is it looking for the static files,
> such as JS?
>
> It seems like what you could do is create a symlink between your
> development directory and the static folder used for the vertical.

The situation Simon's dealing with is the desire to have an active
development/test cycle of TiddlyWiki plugins, in a TiddlyWeb
environment, done in a way that interoperates nicely with packaging
and deployment.

There are several tools that help with all this, notably devstore, and
tiddlywebplugins.instancer but they haven't quite come together. In
part this is because the requirements are slippery, in part it is
because it is a hard problem.

This is slightly different from the situation you are describing,
where the javascript is not tiddlers, but static content used when
TiddlyWeb is operating as a sort of content mgt system, and that
static content is being served up directly, rather than from bags.
That process is pretty well worked out: symlink a dir containing the
static content where you need, and edit it as required.

What would be extremely useful throughout the process of improving
devstore, tiddlywebplugins.instancer, and develop/test/deploy in
general is if we can get some ordering into the bugs, feature requests
and requirements associated with those things. What tends to happen is
we start a discussion about what needs to happen and it quickly
diffuses into a bunch of general wishes for things to be better,
easier, more effective etc.

These general wishes are not actionable. To make progress there need
to be specific, detailed, granular requests that are, where possible,
placed into a generic context separate from whatever large overarching
concern may be involved. This is much like the persistent desire for
minimum test cases (MTC) over on the [tiddlywiki] group. For these things we
need, where possible, separated minimum feature requests (MFR). The large
overarching concern and context is valuable too (often extremely so)
but the MFR needs to be extracted out, made visible. This makes it
actionable.

Sometimes an MFR might be in the form of a user story (e.g. I want to
edit foo.js in my favorite editor, reload in tiddlyweb, and have the
foo.js changesbe there). Sometimes it might be in the form of a more
specific technical demonstration.

In either case imagine that the respondent to your request is going to
be writing a test for the request. It's possible to write a test for
the above example. It's not possible to write a test for "make it
better" (not to suggest people are saying that, just pointing out an
extreme example).

--
Chris Dent http://burningchrome.com/~cdent/
[...]

[Michael Mahemoff]

On Thu, Feb 11, 2010 at 11:39 AM, <chris...@gmail.com> wrote:

On Wed, 10 Feb 2010, jnthnlstr wrote:

For your packaged vertical, where is it looking for the static files,
such as JS?

It seems like what you could do is create a symlink between your
development directory and the static folder used for the vertical.

The situation Simon's dealing with is the desire to have an active
development/test cycle of TiddlyWiki plugins, in a TiddlyWeb
environment, done in a way that interoperates nicely with packaging
and deployment.

There are several tools that help with all this, notably devstore, and
tiddlywebplugins.instancer but they haven't quite come together. In
part this is because the requirements are slippery, in part it is
because it is a hard problem.

Worth mentioning that tools might not always be the answer, but an articulation of the patterns you're envisioning. We have a gap there right now. e.g. how to use symlinks.

I would love to see a small, real-world canonical application, sitting in GitHub, with instructions on how to build it and test it, ideally with one click, then how to update a tiddler and rebuild and test it again with one click and no downloading. That would solve most of the problem we face. To me, this is the end-goal.

It's a lot more difficult to write actionable requests, mostly because it feels like there's a lot of implicit knowledge and assumptions here that aren't apparent to me (e.g. the symlink thing) and because, frankly, it would inevitably bleed into implementation design or assume detailed architectural understanding, whereas for most of us, we don't actually care how it happens; we just have the uber-simple, easily stated, requirement that we want a one-click test-debug cycle. I change something, I run "make", and I reload my browser. (Some will say we shouldn't have to run "make" at all, but I can live with that compromise personally.)

Rather than writing generic tickets or stories, I think we could make things more actionable if the community worked on a canonical application together. I would be happy to contribute that way, in a colocated event.

[FND]

> I'll look into further simplifying the devstore, making it easier to
> use with any vertical.

I've added more Evil Magic to the devstore twinstance_dev instantiation
script, which enables the following process (at significant cost in
complexity, and brittleness, of the underlying code):

$ sudo pip install -U tiddlywebplugins.devstore

$ git clone g...@github.com:FND/tiddlyspace.git tiddlyspace
$ make remotes # also runs cacher
$ twinstance_dev tiddlywebplugins.tiddlyspace foo
$ cd foo
$ $EDITOR tiddlywebconfig.py # edit desired paths
$ twanager server

Regarding the desired paths: One example would be changing the _public
bag's file:// URI to cloneWizard.js to a relative path
("../src/cloneWizard.js").
Changes in that file in the repo will be reflected directly in the
instance - so adding 'alert("foo");', reloading the page in the browser,
adding 'alert("bar");', reloading etc.

-- F.

[FND]

> $ $EDITOR tiddlywebconfig.py # edit desired paths
> $ twanager server
>

> Regarding the desired paths [...]

We have since come up with a further abstraction, allowing individual
developers to specify paths in a separate file (here: devtiddlers.py):
http://github.com/FND/tiddlyspace/commit/e79292f4ae9fd22ff7c0c79daf942c695ccf3977

-- F.

[simon mcmanus]

Thanks Fred

-- F.

--
You received this message because you are subscribed to the Google Groups "TiddlyWeb" group.
To post to this group, send email to tidd...@googlegroups.com.
To unsubscribe from this group, send email to tiddlyweb+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/tiddlyweb?hl=en.

--
Simon McManus

blog : http://simonmcmanus.com