Getting Manila to run in OPML (and welcome Tom Clifton)

[Dave Winer]

I got an email last night from Tom Clifton saying he wanted to get
Manila running in the OPML Editor environment.

I would like to see this work too, but of course I have other stuff on
my plate too. :-)

Anyway, I took a few minutes this morning to try to get it up and
running on one of my servers, and I got to Hello World, was able to
create a new site, get to the Control Panel site, which worked.

http://worknotes.scripting.com/january2012/1812ByDw/settingUpManila/

Now in reviewing the settings on Control Panel, I realize that most of
them should be turned off because they represent earlier versions of
stuff that the OPML server environment does better. It's version 3.0
of a lot of the stuff in there.

Also to people who are new to this incarnation of Frontier, I put
together a site with pointers of where to look for new stuff.

http://hellofrontier.com/

Dave

[Tom Clifton]

Thank you, Yes the control panel is working. We need some init scripts
to fill in the preferences so they aren't pointing to Dave's values.
Next thing is to get the admin page up. Once that is running, I will
be a happy camper.

I think my biggest problem is that since the server verbs got
kernelized (sp?), I've never quite figured out how to walk through
them and figure out where I am and what needs to happen next. Not sure
if that makes sense, but that is were I am in making the connection
between the call to the admin page (127.0.0.1:5337) and the scripts
that create the admin page.

Tom

[Dave Winer]

Tom,

If I am to help i need to get a lot more specific about what exactly isn't working.

http://scripting.com/stories/2010/06/30/writingGoodBugReports.html

Please give that a read. It's generic advice about how to get this to go faster.

Ironically perhaps, the way to do it is to go slowly... :-)

Dave

[Tom Clifton]

Thanks Dave,

You are correct that I need to slow down and focus. Here is what I expected. In Manila and Radio the port was important because a specific location in the root structure was called as a result. So 127.0.0.1:5335 pulled up Radio and 127.0.0.1:5336 pulled up the Manila admin page. Calling 127.0.0.1:5335 yields the "allowdirectorylistings"error reported earlier. Calling 127.0.0.1:5336 produces an unable to connect message in Firefox because the OPML Editor is not listening to port 5336.

The OPML Editor is listening to four ports, 5335, 5337, 5338, and 8081. All four of these listens (user.inetd.listens) call system.verbs.builtins.webserver.server. Presumably this verb is where the port is matched to a database address. I don't know this for certain because the verb was kernelized in 1999.

That is the extent of my knowledge. From here on out, we are going to have to deal with conjecture.

I thought that the control panel works because it is part of the mainresponder which is now part of OPML.root. But when I created a copy of system.verbs.builtins.mainresponder.controlPanel called system.verbs.builtins.mainresponder.controlPanel2 and called it with the url http://127.0.0.1:5337/controlpanel2/.

It generates a 404.

So I don't know why the controlpanel works. I don't think that it is called through a responder as HelloWorld is. I don't know how the OPML server maps ports to database addresses. I don't even know if this is a useful bug report.

Tom

[Dave Winer]

What is "the Manila admin page?"

Not sure what you're looking for.

An old screen shot or docs page might help.

If anyone else here has any ideas, chime in! :)

On Sunday, January 8, 2012, Tom Clifton <trcl...@gmail.com> wrote:

[Tom Clifton]

This is what I am looking for. It lists the sites on my server and provides links to edit those sites. I think that it is located in [manila.root].manilaSuite.admin.adminPage. 

The question is how do we route the request in the OPML Editor from tools>Manila>Open Admin Site to that page. The port routing that worked in Frontier doesn't seem to work. I don't know the innards (of system.verbs.builtins.webserver.server) enough to suggest a solution. Would it be possible to specify an address in the user.inetd.listens table?

Tom

[Tom Clifton]

The screen shot didn't work, so let's try this:

http://www.flickr.com/photos/tclifton/6662893927/in/photostream

Tom

On Jan 8, 3:24 pm, Tom Clifton <trclif...@gmail.com> wrote:
> This is what I am looking for. It lists the sites on my server and provides links to edit those sites. I think that it is located in [manila.root].manilaSuite.admin.adminPage.
>
> The question is how do we route the request in the OPML Editor from tools>Manila>Open Admin Site to that page. The port routing that worked in Frontier doesn't seem to work. I don't know the innards (of system.verbs.builtins.webserver.server) enough to suggest a solution. Would it be possible to specify an address in the user.inetd.listens table?
>
> Tom
>
> On Jan 8, 2012, at 2:54 PM, Dave Winer wrote:
>
>
>
>
>
>
>
> > What is "the Manila admin page?"
>
> > Not sure what you're looking for.
>
> > An old screen shot or docs page might help.
>
> > If anyone else here has any ideas, chime in! :)
>

> > On Sunday, January 8, 2012, Tom Clifton <trclif...@gmail.com> wrote:
> > > Thanks Dave,
>
> > > You are correct that I need to slow down and focus. Here is what I expected. In Manila and Radio the port was important because a specific location in the root structure was called as a result. So 127.0.0.1:5335 pulled up Radio and 127.0.0.1:5336 pulled up the Manila admin page. Calling 127.0.0.1:5335 yields the "allowdirectorylistings"error reported earlier. Calling 127.0.0.1:5336 produces an unable to connect message in Firefox because the OPML Editor is not listening to port 5336.
>
> > > The OPML Editor is listening to four ports, 5335, 5337, 5338, and 8081. All four of these listens (user.inetd.listens) call system.verbs.builtins.webserver.server. Presumably this verb is where the port is matched to a database address. I don't know this for certain because the verb was kernelized in 1999.
>
> > > That is the extent of my knowledge. From here on out, we are going to have to deal with conjecture.
>

> > > I thought that the control panel works because it is part of the mainresponder which is now part of OPML.root. But when I created a copy of system.verbs.builtins.mainresponder.controlPanel called system.verbs.builtins.mainresponder.controlPanel2 and called it with the urlhttp://127.0.0.1:5337/controlpanel2/.

[Dave Winer]

We'll figure this out -- the site has to be in one of the roots somewhere, then it's just a matter of connecting it through config.mainresponder somehow.

I would stop thinking in terms of inetd. We connect sites up all the time, but never do it at such a low level.

If anyone knows where the site is, let me know -- otherwise I'll find it. Maybe not today or tomorrow, but soon.

Dave

[Dave Winer]

Tom, a possibly dumb question -- have you been to the Control Panel site on your server?

If so, are you sure it's not the site you're looking for?

I am able to access it on mine, as I explained in yesterday's worknote.

http://worknotes.scripting.com/january2012/1812ByDw/settingUpManila

Dave

[Tom Clifton]

Yes, I can get to the control panel page and I can use it to create a
Manila site.

I need to work through the functions of the control panel and see how
it behaves relative to the Manila environment.

I will report back when I have more data.

Tom

On Jan 9, 2:54 am, Dave Winer <dave.wi...@gmail.com> wrote:
> Tom, a possibly dumb question -- have you been to the Control Panel site on
> your server?
>
> If so, are you sure it's not the site you're looking for?
>
> I am able to access it on mine, as I explained in yesterday's worknote.
>
> http://worknotes.scripting.com/january2012/1812ByDw/settingUpManila
>
> Dave
>
>
>
>
>
>
>
> On Sun, Jan 8, 2012 at 8:35 PM, Dave Winer <dave.wi...@gmail.com> wrote:
> > We'll figure this out -- the site has to be in one of the roots somewhere,
> > then it's just a matter of connecting it through config.mainresponder
> > somehow.
>
> > I would stop thinking in terms of inetd. We connect sites up all the time,
> > but never do it at such a low level.
>
> > If anyone knows where the site is, let me know -- otherwise I'll find it.
> > Maybe not today or tomorrow, but soon.
>
> > Dave
>

> > On Sun, Jan 8, 2012 at 6:29 PM, Tom Clifton <trclif...@gmail.com> wrote:
>
> >> The screen shot didn't work, so let's try this:
>
> >>http://www.flickr.com/photos/tclifton/6662893927/in/photostream
>
> >> Tom
>
> >> On Jan 8, 3:24 pm, Tom Clifton <trclif...@gmail.com> wrote:
> >> > This is what I am looking for. It lists the sites on my server and
> >> provides links to edit those sites. I think that it is located in
> >> [manila.root].manilaSuite.admin.adminPage.
>
> >> > The question is how do we route the request in the OPML Editor from
> >> tools>Manila>Open Admin Site to that page. The port routing that worked in
> >> Frontier doesn't seem to work. I don't know the innards (of
> >> system.verbs.builtins.webserver.server) enough to suggest a solution. Would
> >> it be possible to specify an address in the user.inetd.listens table?
>
> >> > Tom
>
> >> > On Jan 8, 2012, at 2:54 PM, Dave Winer wrote:
>
> >> > > What is "the Manila admin page?"
>
> >> > > Not sure what you're looking for.
>
> >> > > An old screen shot or docs page might help.
>
> >> > > If anyone else here has any ideas, chime in! :)
>
> >> > > On Sunday, January 8, 2012, Tom Clifton <trclif...@gmail.com> wrote:
> >> > > > Thanks Dave,
>
> >> > > > You are correct that I need to slow down and focus. Here is what I
> >> expected. In Manila and Radio the port was important because a specific

> >> location in the root structure was called as a result. So 127.0.0.1:5335pulled up Radio and
> >> 127.0.0.1:5336 pulled up the Manila admin page. Calling 127.0.0.1:5335yields the "allowdirectorylistings"error reported earlier. Calling

[Jake Savin]

The Admin site was part of the 9.0 UserLand release. Mostly it's implemented in mainResponder.adminSite, with the actual table in mainResponder.adminSite.website. As I recall, the Admin site also supported Control Panel plugins from the 8.x timeframe, though I don't know how well that was tested.

The Admin site's Settings pages were implemented using the outline-based prefs system that was introduced with Radio 8 (radio.prefs.browser). The outline that renders the admin site is in mainResponder.adminSite.data.localization.languages.english.outlines.prefs. (As far as I know it was never localized.)

I haven't tried getting all this running in the OPML Editor, and my time is somewhat limited right now, but I'm happy to help with more pointers if I can. The mainResponder and manilaSuite hierarchies are still shockingly fresh in my mind, even though I haven't worked with them for more than a few minutes at a time for over 5 years. :)

Hope this helps,
-Jake

[Dave Winer]

>>Hope this helps,

Yes it does.

And the outline-based prefs system not only works in the OPML Editor, it's being maintained.

It's how we do prefs in the OPML Editor and in River2, Radio2 and World Outline.

I'd like to do more than just get Manila run in the new environment, I'd like to have it be *part* of it.

Dave

[Jake Savin]

Great to hear about outline-based prefs - as an implementor I love how fast and easy it made the work. (And I figured it was still being used. :)

 

Can you elaborate a little on what you mean by having Manila "be a part of it"?

 

Having one or a few near-term goals would help me set a context for whatever investigations I have time for.

 

Sadly, it's going to be very hard for me to commit serious time to this effort right now, but if I can help by contrubiting an hour or three at a time, I'm happy to do that.

 

Thanks,

-Jake

[Dave Winer]

In reviewing the Control Panel yesterday I realized that most of the things in there are functions that are already implemented in the core of the OPML Editor, and it's done better here than it was there. That's because this stuff is a generation ahead. So it's higher level. More stuff is buried in lower levels, so the higher level code is easier.

Also there are facilities that weren't there when Frontier/Manila was being worked on. We have clouds and Amazon S3, and really elegant interfaces for DNS. The updating process is way different (and a lot simpler).

That said, what Manila does is not done anywhere in the three-part suite that I have put together. So most of what's in Manila is strictly a plus.

Re how you can help, your first message is the best stuff. Make your knowledge available, within your time constraints.

Dave

[Tom Clifton]

Thanks Jake!

Yes this helps.

I edited the entry for spudlight.local in config.mainresponder.domains
mapped admin to mainresponder admin site:
mainResponder.adminSite.website

That yields this: http://www.flickr.com/photos/tclifton/6669412321/in/photostream

The menu at the top is broken, in that the relative urls point to
spudlight.local rather than to spudlight.local/admin/. But if you
manually enter these urls, they work.

The site config page appears to be working.

These are big steps in the right direction!

Thanks again, Jake!
Tom

[Jake Savin]

Great, Dave. Thanks for the guidance! -Jake

[Jake Savin]

Glad to hear it!

My guess is that the links may be relatively easy to fix. Let me know if you need any pointers there, and I'll have a look.

Cheers, -Jake

[Tom Clifton]

I got around the link problem by setting
mainResponder.adminSite.website as the top site for the local domain.
This won't work when I want to implement it in the wild, but will work
fine for testing locally.

Tom

[Jake Savin]

Tom, the links are generated by
mainResponder.adminSite.website.["#tools"].menu. They come from
mainResponder.adminStie.urls. Most of the strings in that table correspond
to pages in the admin site itself, and they're all relative links, with a
leading '/' character.

If you want to fix this, most likely the easiest way is to update the menu
script to append the string from the urls table to the URL of the admin site
homepage, as running in your context. The WSF page table /may/ have all the
data you need to construct the URL.

-Jake

Tom

> > > > > >> tools>Manila>that worked in

[Tom Clifton]

Ok, I have the admin page looking like it should. Not everything is
working, but the cosmetics are good. I had to work around one of
Jake's 2002 work arounds back to get the status center macro
mainResponder.adminSite.macros.statusCenter to work (apparently
system.environment.osVersionString now reports the correct version in
OS X and there is no need for a hex conversion). Thank you for a well
commented script.

Making progress,
Tom

[Dave Winer]

Tom -- just checking in -- were you able to get your Manila sites
running in the OPML Editor environment?

Dave

[Tom Clifton]

Dave,

The short answer is yes. I am able to take an existing
ManilaWebsites.root, move it to my OPML Editor environment, install
the site from the OPML environment, modify it in the Manila
environment and get it to run.

There are glitches to work out, but they don't seem unreasonable.

I need to play with it some more to give a fuller report.

Tom

[Dave Winer]

First, that's good! I'm glad you're able to get into Manila on the OPML Editor.

I'm interested in your progress for other reasons too...

1. I am running a bunch of Manila sites myself from the Old Days (TM) and have to keep a copy of Frontier running. This is actually fairly expensive for me. So if I could run Manila in OPML I would save money and be able to simplify things, which is always good.

2. If there are any bugs to fix in OPML Editor, I want to do that. Esp easy ones. :-)

3. I'd like to be able to say to people who ask (and it comes up a fair amount) that you can run Manila in OPML, and have a brief howto document that explains. Linked into its record in the Tools Catalog page.

4. Over time it may be possible to connect Manila up to the other tools that are now running in the OPML environment.

5. And other system-level capabilities that didn't exist when Manila was written.

It's a big piece of software, and it would be good to have it on board the train to the future. :-)

Dave

[Tom Clifton]

Dave,

As you might imagine, I was pleased to hear Amazons EC2 announcement today. I have created an instance and will be working to get Manila running there. I will let you know how it goes.

I have attached some notes from running Manila in the OPML editor on my local server. While the focus is on things that are broken, the majority of Manila is working fine. This should not be taken as a negative report. One thing to note, however, is that the Manila release in the OPML editor is older than the Manila 9.6 release that I have been using. That may be the root of some of the things that I see as missing.

[Tom Clifton]

An update: 

I have Manila running on a EC2forPoets instance. Most everything seems to be working as it should, though I haven't tested everything.

I can create new Manila sites. I have moved an existing Manila site to the server and have it running.

I ran into a problem when trying to edit a page. Clicking on the Edit this Page button brings up the edit form (no WYSIWYG editor though in FireFox 10. Clicking the post changes button generates the following server error:

500 INTERNAL ERROR

Can't evaluate the expression because the name "binaryServerFolder" hasn't been defined.

The changes are made, but you have to go back to the page and refresh it to see them.

There are some cosmetic glitches here and there, but I haven't worried about them. The picture upload may not be working correctly. I need to test this some more.

So there are some glitches but it is generally running well.

Tom