CPDocument architecture in Cappuccino 1.0

[Daniel Stolzenberg]

I would like to discuss how we could improve the document architecture
towards a cappuccino 1.0 release. There are several reasons why I
think this is necessary:

1) The document architecture neither works out-of-the-box nor clearly
states what the developer has to do to make it work (in addition to
implementing document specific "dataOfType:" and
"readFromData:ofType:").

2) The document architecture is in contradiction to cappuccinos
overall "server agnostic" nature: There is a lot of specific code in
CPDocument almost useless to everybody and therefore replaced in most
cases anyway.

3) Moreover the design of the document architecture does not provide a
standard way to adapt the same application to different server
technologies.

It is a hard task for every developer new to cappuccino (even when
having a cocoa background) to use the document architecture and this
is not acceptible in my opinion. So I think this definitely needs to
be improved before releasing cappuccino 1.0.



After working around this limitations for a long time now, I want to
discuss my thoughts and solutions to get things started:

In general, we should cleanly extract and encapsulate all aspects of
document persistence that may vary between different server
technologies as well as different applications. The current problems
and source code comments within the document architecture suggests
extracting the following aspects:

a) Save- & Open-Panels
b) URL-based save and read operations (including authentication)



My current solution uses a delegation mechanism to encapsulate these
aspects: Every CPDocument instance has a "persistence agent" object,
which is responsible for executing the save/read operations and
providing an appropriate SavePanel.

Which kind of persistence agent is used is configured in the main
bundle: In Info.plist for every supported document type the matching
persistence agent class is given via an additional key
"CPDocumentPersistenceAgentClass" - this way an existing application
can easily be adapted to a different persistence stack (i.e. server
technology).

The persistence agent class itself is responsible for providing the
Open Panel for its associated document types (via class methods).

In my opinion the document architecture should be useable almost out-
of-the box, so default persistence agent and save/open panel
implementations should be provided by cappuccino itself (maybe using
HTML-5 local storage). This would enable new developers to focus on
application features first before implementing a custom persistence
mechanism. Moreover I suggest, that cappuccino provides a new template
for document based applications using this default persistence agent
class.


I would really like to hear your thoughts and ideas regarding the
future of document architecture as well as my approach to accomplish a
better encapsulation of document persistence.

Daniel

[Francisco Tolmasky]

So, the idea behind the current CPDocument open/save structure, which is incomplete no doubt, is that it should work out of the box give the server is RESTful. To give you an idea of how I internally implemented HTML5 local store, I came up with "local" URLs:

cappuccino-local://path/to/file

Our XMLHTTPRequests know internally that URLs of this nature should be put in the local store (or gotten from the local store) and return RESTful response codes. That way, we don't start having incredibly different logic for each different form of persistence. Additionally, this means that these documents can be opened in other parts of the app, again without worrying about the implementation details of how these things are *actually* stored. For example, say an app may open a document through the normal document process, OR open it with an XMLHTTPRequest and operate on the text directly. Additionally, this app has some documents that exist remotely and others that exist locally. In a world where URLs define this entirely, the code can treat all these documents identically, instead of having 2-4 code paths.

[Tom Robinson]

We should also explore built-in syncing online/offline documents. It should be relatively easy to do with unstructured documents.

CouchDB could be an inspiration (the document architecture should also be flexible enough to work with CouchDB almost out of the box, since it's RESTful)

> --
> You received this message because you are subscribed to the Google Groups "Cappuccino & Objective-J Development List" group.
> To post to this group, send email to objecti...@googlegroups.com.
> To unsubscribe from this group, send email to objectivej-de...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/objectivej-dev?hl=en.

[Randy Luecke]

Though perhaps out of the scope of this discussion I've Bren thinking
about how we could implement a file open/save window... It brings up a
lot of abstraction questions and the fact that most people would be
more interested in the interaction with their server while I think the
local storage stuff is much more interesting...

The biggest issues is that Html5 localstorage is limited to 5MB with
no option for expansion. I think indexedDB is much more interesting
for this (but we get into the browser issues)

Perhaps adding an API for optional folder structures and what not...
Keeping everything in sync would be particularly interesting. I think
being able to launch 1.0 with a complete "FS API" for documents would
be really nice.

The real pain is to make people understand that we ARE server
agnostic, and that it will work with any restful API, but in order to
use some of our nice document features you need to implement a special
API for us (perhaps even get the community to help with a skeleton API
for developers to use.

[Ross Boucher]

I don't think it's a huge leap to suggest that the document architecture needs a lot of work. It's true that it doesn't really do anything out of the box, though I think requiring a certain base level of effort isn't necessarily a bad thing.

More importantly, it's actually one of the pieces of Cocoa that may have aged the worse. On the desktop, it has largely been replaced by CoreData and custom formats. And server apps have shown us that most things are much more likely to be broken down into small pieces of data stored in a database, versus a single blob that can be loaded and saved once.

So one approach might be to simply move forward with a CoreData like system either as part of or in place of any work on CPDocument. I think Saikat probably has a lot to add on this front as well since they're doing interesting things with documents.

On Dec 8, 2010, at 9:14 AM, Francisco Tolmasky wrote:

[Tom Robinson]

There's definitely a huge spectrum of data persistence styles, from document "blobs" to highly structured data. The document architecture is aimed at the former, but the latter is probably more common in web apps. I doubt there's one good solution for the whole spectrum.

[Randy Luecke]

I would imagine the real solution to structured data is to provide
CoreData which could simultaneously make calls to the server and we're
just going to have to define the server API for that if they want to
use it. The key to this is documenting it well, and make sure people
understand they have options.

In general applications have users and users have data. The connection
between the server and client is largely boilerplate code that we are
left to rewrite in every app. Since Cappuccino aims to eleminate as
much boilerplate code as possible, I think it is worth our time to
explore options. If we're not able to deliver by 1.0 I don't think we
should stress over it. However a more complete document system should
be something we definitely add to the todo list.

I haven't looked into core data however Nick has told me that the API
is massive, which is why i question it's feasibility in 1.0. On the
other hand a simple data store that makes simple rest calls to the
server for setting and polling key-value data might also be useful.

On Dec 8, 6:00 pm, Tom Robinson <tlrobin...@gmail.com> wrote:
> There's definitely a huge spectrum of data persistence styles, from document "blobs" to highly structured data. The document architecture is aimed at the former, but the latter is probably more common in web apps. I doubt there's one good solution for the whole spectrum.
>
> On Dec 8, 2010, at 2:22 PM, Ross Boucher wrote:
>
>
>
> > I don't think it's a huge leap to suggest that the document architecture needs a lot of work. It's true that it doesn't really do anything out of the box, though I think requiring a certain base level of effort isn't necessarily a bad thing.
>
> > More importantly, it's actually one of the pieces of Cocoa that may have aged the worse. On the desktop, it has largely been replaced by CoreData and custom formats. And server apps have shown us that most things are much more likely to be broken down into small pieces of data stored in a database, versus a single blob that can be loaded and saved once.
>
> > So one approach might be to simply move forward with a CoreData like system either as part of or in place of any work on CPDocument. I think Saikat probably has a lot to add on this front as well since they're doing interesting things with documents.
>
> > On Dec 8, 2010, at 9:14 AM, Francisco Tolmasky wrote:
>
> >> So, the idea behind the current CPDocument open/save structure, which is incomplete no doubt, is that it should work out of the box give the server is RESTful. To give you an idea of how I internally implemented HTML5 local store, I came up with "local" URLs:
>
> >> cappuccino-local://path/to/file
>
> >> Our XMLHTTPRequests know internally that URLs of this nature should be put in the local store (or gotten from the local store) and return RESTful response codes. That way, we don't start having incredibly different logic for each different form of persistence. Additionally, this means that these documents can be opened in other parts of the app, again without worrying about the implementation details of how these things are *actually* stored. For example, say an app may open a document through the normal document process, OR open it with an XMLHTTPRequest and operate on the text directly. Additionally, this app has some documents that exist remotely and others that exist locally. In a world where URLs define this entirely, the code can treat all these documents identically, instead of having 2-4 code paths.
>
> >> --
> >> You received this message because you are subscribed to the Google Groups "Cappuccino & Objective-J Development List" group.
> >> To post to this group, send email to objecti...@googlegroups.com.
> >> To unsubscribe from this group, send email to objectivej-de...@googlegroups.com.

> >> For more options, visit this group athttp://groups.google.com/group/objectivej-dev?hl=en.

[Saikat Chakrabarti]

In Mockingbird, we have overridden a large part of CPDocument so I
agree with the general sentiment that the CPDocument API needs work.
I agree with Daniel that the steps to getting working with a
document-based app are not clear, but this is largely a documentation
issue. Also, just to clarify, there are two parts to CPDocument
really - CPDocument as a way to manage and represent your document's
data in the app, and then the CPDocument API for persisting your app.
I think the former isn't too bad and it's largely the latter that
needs to be changed or changed into something else entirely (I think
others are saying the same).

The issue is, I think, that CPDocument makes a lot of assumptions
about how you persist your data (also a slight nit -- i don't think it
is entirely restful -- the overwrite request and and CPSaveOperations
should be PUT requests, for example (might not even need the HTTP
header)). I thought about talking a bit about the unusual things we
do and which of CPDocument's assumptions are the ones that have caused
us to rewrite a parts of it, but none of the things we do really seem
that useful in the general use case, and I think that's entirely the
issue - many people will have a non-generalizable use case here. I
think we I agree with Tom/Ross that there should be some default
persistence solution (I don't know a lot about CoreData), but I think
we just need to be really good about making it clean and easy to
switch this part out as many people will need to (and currently,
overriding the server-interacting behavior in CPDocument is a bit
messy).

I also like the idea of having offline support via local storage
automatically in Cappuccino apps.

If you guys want, I can go through the parts of CPDocument that we've
overridden and why, but not sure how useful that will be if we want to
move to a different model entirely.

[Daniel Stolzenberg]

Saikat has pretty much hit the point that I targeted when starting
this discussion - the distinction between two aspects of CPDocument is
very helpful:

1) Application oriented features
2) Persistence oriented features

I also agree with Saikat, that the first aspect is fine and only the
second part needs some work. To be more precise one could subdivide
this second aspect based on commonality and variability:

2a) Persistence Management
2b) Persistence Execution (including Panels)

The aim of a major overhaul of CPDocument would be to clearly separate
these two subaspects which is currently not the case. This would make
it a lot easier getting the document architecture to work initially as
well as adapting an existing document based application to different
server technologies.

For those interested in details: Have a look at the implementation of
my proposal: https://gist.github.com/735057
Daniel

> >> For more options, visit this group athttp://groups.google.com/group/objectivej-dev?hl=en.

[MD]

Hi Daniel,

I am interested in your proposal, and actually the design looks good!

So could you share some of your updates if you have made improvements on this?

Thanks,
Takenori

[Sato Takenori]

An update from the latest document. 

This is an iOS doc that discusses some of design issues brought up here. 

(No one for Mac similar to this available)

http://developer.apple.com/library/ios/#releasenotes/DataManagement/RN-iCloudCoreData/_index.html

2012年5月26日土曜日 13時15分18秒 UTC+9 Sato Takenori: