Discontinuities

[Brent Simmons]

I realized something important. What stops me from working on the kernel isn't so much lack of time or lack of an OS X 10.6 machine as it is this problem: I keep running into brick walls.

Here’s the thing. I’m not that interested in expedient fixes. I want to do work that will last for many years; I want to be able maintain and improve the software in the future; I want to rebuild it as a vigorous and vital platform with lots of users.

But there are these brick walls. So it finally occurred me: why not ask for help?

The help I need isn’t programming help: it’s conceptual help. I’ll describe the first issue.

#### Object database format

The database format is 32-bit. It makes all the sense in the world to move it to 64-bit, since 32-bit is going away.

Here are the problems:

1. A 64-bit database couldn't be opened using an older version of the kernel.

2. It would be necessary to convert a 32-bit database to 64-bit entirely upon opening it. This is an expensive operation with no turning back, and so it makes sense to have a manual import/conversion step.

3. A new database format is an opportunity to drop the ODB code entirely. Instead, a database should either be a SQLite file, or, perhaps better yet, a folders/files structure on disk. (Machines and hard drives these days — SSDs, especially — are fast enough for this. This would also mean you could put a database into source code control, which would be very cool.)

4. I could write such a new storage system in a day using Cocoa. Which makes it Mac-only. When would I turn around and do the Windows version? Could be a very long time. (But a new format would, at least, be compatible in principle with Windows and Linux.)

5. There’s only a limited period of time where we could convert old databases to new databases. The main issue is rich text, which is serialized using deprecated APIs that will stop working, and there’s no way to modernize those. (It should be replaced with RTF.) This could happen as soon as the OS X release after Mavericks. You could be in a position of having an old database that you can’t import to the new format.

#### Some options

You can see why this is confusing and why I get stuck. The issue isn’t writing code — it’s “what’s the right thing to do?”

One option is the greenfield option. It would have to have a new name. Do a Mac app, all-Cocoa, that’s as compatible as possible. Make it so you could import/convert old database files, make it so scripts would run, the same verbs are there, etc. Drawback: Compatibility would be elusive. And it’s a Mac app only.

The second option is similar, but it makes the promise of compatibility (as much as possible). You’d still have the import/convert step for databases, unfortunately. It would also be a Mac app only, mostly-Cocoa, though there would be the hope that, with a revitalized platform, someone or some people would step up who are willing and able to help port it to Windows.

With both options the Windows version is left out in the cold. Your new databases wouldn’t open there until a Windows programmer came along who could do the work.

On the bright side: the kernel would be about 1/4th (or less) the amount of code it is now, with either option. And languages, tools, and techniques have traveled a million miles since the late ’90s when we ported Frontier to Windows. New formats would use standard stuff (file system, JSON, RTF, UTF-8, OPML) and open source (SQLite possibly, if we went that way), so there would be no Mac lock-in for the data, and writing Windows and Linux code to handle the data would be relatively easy.

But still: any way forward leaves some people disappointed, and the best case still has people having to import/convert databases rather than just open them, which is a pain.

#### Yet more brick walls

There are some things we didn’t port to Windows and that would also be casualties in the Mac app: OSA component, MacBird cards, file aliases, PICTs. Printing support might live on the bug tracker forever.

In other words, I don’t see a painless way forward. There is no 100% compatibility. There is breakage even in the Mac app (though limited to things that are marginal for most users, assuming a replacement for the built-in MacBird cards). There is the potential that a Windows version would never happen.

[Dave Winer]

What a great message. This is the kind of stuff I love. ;-)

I've already dealt with these questions in my current work environment, I'm breaking with the past.

I would absolutely be delighted with a solid Mac version that's tracking Apple's releases. 

The best thing that would come with that are people, more developers. :-)

People to share code with. Man that would be great.

My existing code would work, but I'm converting most of it to JavaScript anyway, it's the new stuff that I care about.

Frontier has been miraculously backward compatible, and we've killed it with that, I'm afraid. 

Break it Brent. GIve us a path forward. Mac only is fine. Go go go.

Dave

--
You received this message because you are subscribed to the Google Groups "frontier-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to frontier-use...@googlegroups.com.
Visit this group at http://groups.google.com/group/frontier-user.
For more options, visit https://groups.google.com/groups/opt_out.

[Brent Simmons]

Cool. First thing is to write a little app that converts an object database to a folders-and-files structure.

One of the exciting things about this is that scripts, outlines, and menubars will be stored on disk as OPML files. It's obvious, right? And it makes sense that the app's native storage format would be OPML.

(wp-text will be RTF; scalar will be stored in a single JSON file per table/folder; binary objects will be binary files.)

-Brent

[Ted Howard]

I wonder if this should be written in UserTalk. ObjectDB access is built in. Is the file API rich enough to handle the export?

Ted

--
Ted C. Howard
http://tedchoward.com
http://twitter.com/tedchoward

[Brent Simmons]

Good thinking. I’m sure it could be written UserTalk.

If I don’t have to spend the time writing a special Mac app to do the conversion, then it means I can get going sooner on other things.

A UserTalk export might be slow compared to a C-based app. But maybe that’s okay? It’s something you’d only have to go through once.

I think the first step would be for me to write up what I think the file system version of the database should look like. I’ll do that and post it here. (Tomorrow or later this week.)

-Brent

[Karsten Wolf]

On Tuesday, November 19, 2013 6:45:34 AM UTC+1, Brent Simmons wrote:

Good thinking. I’m sure it could be written UserTalk.

If I don’t have to spend the time writing a special Mac app to do the conversion, then it means I can get going sooner on other things.

A UserTalk export might be slow compared to a C-based app. But maybe that’s okay? It’s something you’d only have to go through once.

Such a suite already exists. Thomas Creedon made it to put a root under version control.

It's available in the files section of the frontierkernel yahoo group.

files/creedon/vcsSync

It was slow but it worked. Perhaps it needs some polishing because it exported every table item as a file.

-karsten