SimpleWebKit (was GNUstep Web browser (was Re: WebKit Bounty))

[h...@computer.org]

Although threre has been a lot of scepticism if the approach of
reimplementing the WebKit API in pure Objective-C is reasonable and a
lot of discussion where to start at all, I have rendered the first
pages as you can see here:

http://www.quantum-step.com/images/web.png

On the left side is the NSLog() of the DOM tree when I link against
Apple Foundation and AppKit (which makes the development cycle very
fast).

I will try to track the current status here:
http://www.quantum-step.com/wiki.php?page=SimpleWebKit

Currently implemented is:
* loading from the network by using NSURLConnection wrapped into
WebDataSource
* HTML parsing (by using an extended NSXMLParser from Foundation)
* HTML to DOM tree translation
* DOM tree to NSAttributedString translation
* display in an NSTextView
* WebFrame, WebFrameView
* WebView

Missing is
* CSS
* Embedding and displaying images, forms etc. in an attributed string
* formatting of tables
* several formatting rules are missing
* JavaScript
* Plugins
* Anchors
* Frames
* Cookies

Fortunately, Riccardo Mottola will help to make it compile and run
with GNUstep base and gui (which shouldn't be too diffcult).

Nikolaus Schaller

[Gregory John Casamento]

There hasn't been skepticism, just the idea that if we do it from scratch we will be duplicating the efforts of the KHTML and Apple teams working on WebKit.

I believe that both approaches have merit. If we can get something working with a from scratch solution, then I believe it's a good thing. But, such an implementation will always have issues with bad webpages that many of the other browsers can handle.

That being said, it looks like you're doing an excellent job.

Later, GJC
--
Gregory Casamento

http://www.quantum-step.com/images/web.png

Nikolaus Schaller

_______________________________________________
Discuss-gnustep mailing list
Discuss...@gnu.org
http://lists.gnu.org/mailman/listinfo/discuss-gnustep

[Nicola Pero]

Personally, I would rather see a basic xhtml TextConverter bundle for gnustep-gui so that
we can finally write our local help files in xhtml rather than rtf. ;-)

Thanks

[Dr. H. Nikolaus Schaller]

Nicola,
there is no reason why you can't do that with SimpleWebKit (so it is
not a "rather").

I estimate that implementing NSAttributedString -initWithHTML: around
the WebKit API is just 20-30 LOC.

Nikolaus

[Dr. H. Nikolaus Schaller]

Am 06.03.2007 um 15:54 schrieb Gregory John Casamento:

> There hasn't been skepticism, just the idea that if we do it from
> scratch we will be duplicating the efforts of the KHTML and Apple
> teams working on WebKit.
>
> I believe that both approaches have merit. If we can get something
> working with a from scratch solution, then I believe it's a good
> thing. But, such an implementation will always have issues with
> bad webpages that many of the other browsers can handle.

Yes, I agree that this is the most crucial aspect for real life
useage. I have started to load other pages: www.apple.de fails
because it wants JavaScript. www.apple.com fails somewhere in my HTML
scanner. And so, bugs come to the surface.

Many other browsers also have issues with pages that others can
handle. A common situation is that although WebKit is one of the best
implementations, Web designers have a tendency to optimize for the
second or third best and break operation for Safari/WebKit. Safari
still has the "Submit error to Apple..." menu function. So, coming
to a more or less appropriately working html engine is more a
community process than engineering or coding.

And, all these projects have started being worse than existing
browsers. But they managed to catch up by feedback. And now, HTML 4
is quite stable, so I think we can use the superior implementation
speed of Objective-C to catch up much faster. Let's see how far we
come...

Nikolaus

[Yen-Ju Chen]

I agree with GJC that it is a big duplicate to have our own web browser.
But I also agree with Nicola that we need some basic xhtml view for
text conversion,
GNUMail and Grr have to render xhtml in many cases.
And if I remembered it correctly, they both have a simple
implementation inside
to deal with HTML content.

Yen-Ju

[h...@computer.org]

>
> I agree with GJC that it is a big duplicate to have our own web browser.

Ok, that brings up a different aspect - maintaining an own web browser
is a duplicate of work. And that is a completey different topic/
decision.

1. SimpleWebKit as part of mySTEP is here and shows that it works.
2. And I will continue to work on it since I have reasons to do so.
3. The offer to the GNUstep community to adopt it for a GNUstep web
browser it is here.
4. But the GNUstep community must decide if they want to follow this
path and integrate it or not.

> But I also agree with Nicola that we need some basic xhtml view for
> text conversion,
> GNUMail and Grr have to render xhtml in many cases.

Yes they receive HTML formatted mails.

The question remains to me why we should write a (new?) basic xhtml
view when we can have a full (x)html view - to avoid duplication.

Nikolaus

[Richard Frith-Macdonald]

I don't really see what the problem is with having both SimpleWebKit
and a port of WebKit ... then developers can choose.

As for the xhtml text converter bundle Nicola suggests for use by the
gui library ... the obvious thing IMO would be to write it to use
SimpleWebKit, and perhaps provide an alternative bundle using WebKit
later.

[h...@computer.org]

> As for the xhtml text converter bundle Nicola suggests for use by the
> gui library ... the obvious thing IMO would be to write it to use
> SimpleWebKit, and perhaps provide an alternative bundle using WebKit
> later.

The wrapper can indeed be made 100% interchangeable since both use the
WebKit API. So, users can finally install one or the other.

Here is a code snippet (this time from mySTEP AppKit and not really
tested):

- (id) initWithHTML:(NSData *)data documentAttributes:(NSDictionary
**)dict;
{
return [self initWithHTML:data options:nil documentAttributes:dict];
}

- (id) initWithHTML:(NSData *)data baseURL:(NSURL *)url
documentAttributes:(NSDictionary **)dict;
{
return [self initWithHTML:data options:[NSDictionary
dictionaryWithObject:url forKey:NSBaseURLDocumentOption]
documentAttributes:dict];
}

// reuse (this is not at all thread safe!)
static BOOL didLoadWebKit;
static WebView *webView;
static BOOL done;

- (void) webView:(WebView *)sender didFinishLoadForFrame:(WebFrame*)
frame
{
NSLog(@"did finish loading main frame");
if(frame == [sender mainFrame])
done=YES;
}

- (id) initWithHTML:(NSData *)data options:(NSDictionary *)options
documentAttributes:(NSDictionary **)dict;
{
// check for HTML like header before trying
if(!didLoadWebKit)
[[NSBundle bundleWithPath:@"/System/Library/Frameworks/
WebKit.framework"] load], didLoadWebKit=YES; // dynamically load
if(!webView)
{
webView=[[NSClassFromString(@"WebView") alloc]
initWithFrame:NSMakeRect(0.0, 0.0, 100.0, 100.0)];
if(!webView)
return nil; // can't initialize
}
[[webView mainFrame] loadData:data MIMEType:@"text/html"
textEncodingName:@"utf-8" baseURL:[options
objectForKey:NSBaseURLDocumentOption]];
[self setFrameLoadDelegate:self];
done=NO;
while(!done)
{ // loading is not yet done
[NSApp nextEventMatchingMask:NSAnyEventMask untilDate:[NSDate
dateWithTimeIntervalSinceNow:0.2] inMode:NSDefaultRunLoopMode
dequeue:NO]; // process events
}
if(dict)
*dict=[NSDictionary dictionaryWithKeysAndAttributes:
NSHTMLTextDocumentType, NSDocumentTypeDocumentAttribute,
[[[webView mainFrame] dataSource] textEncodingName],
NSCharacterEncodingDocumentAttribute, // may be nil?
[[[webView mainFrame] dataSource] pageTitle],
NSTitleDocumentAttribute, // title may be nil!!!
nil];
return [self initWithAttributedString:[[[[webView mainFrame]
frameView] documentView] attributedString]];
}

-- hns

[Yen-Ju Chen]

On 6 Mar 2007 12:05:39 -0800, h...@computer.org <h...@computer.org> wrote:
> > As for the xhtml text converter bundle Nicola suggests for use by the
> > gui library ... the obvious thing IMO would be to write it to use
> > SimpleWebKit, and perhaps provide an alternative bundle using WebKit
> > later.
>
> The wrapper can indeed be made 100% interchangeable since both use the
> WebKit API. So, users can finally install one or the other.
>

So I think SimpleWebKit can do the basic HTML rendering.
If it can be a stand-alone library or framework (or in GNUstep as addition)
and GNUMail and Grr are willing to use it,
then at least we have a basic HTML engine for now.
As for a complete web browser, well, it is not coming anytime soon.

Yen-Ju

[snip]

[Peter Cooper]

[resend]

Hello

The current progress in SimpleWebKit is very gratifying, I did a test
build of the code from a few days ago and the code compiled on
GNUstep (with only one slight change to an #ifdef), and a simple
palette seemed quite buildable (although in that version of the code
there was no content rendering).

> Ok, that brings up a different aspect - maintaining an own web browser
> is a duplicate of work. And that is a completey different topic/
> decision.

I agree, web browser application development is a bit out of scope
here. I'm not sure it fits into the GNUstep-as-library, or
GNUstep-as-dev-env philosophy at all, but it probably works for the
GNUstep-as-desktop-env people. I'd imagine Etoile is more interested
in a disarticulated, embeddable-everywhere object ;-)

> 1. SimpleWebKit as part of mySTEP is here and shows that it works.
> 2. And I will continue to work on it since I have reasons to do so.
> 3. The offer to the GNUstep community to adopt it for a GNUstep web
> browser it is here.
> 4. But the GNUstep community must decide if they want to follow this
> path and integrate it or not.

I would very much like SimpleWebKit to be included as a framework into
the GNUstep-gui or dev-libs collections. As a number of people have
mentioned, even a purely standards-compliant(!) (X)HTML renderer has
significant practical uses for many existing and future tools and
applications, especially where the application has some control over
it's source HTML.

As Nikolaus says, it *is* possible to have a broadly-working framework
for virtually all web content if enough effort is applied. The amount
of effort to get there is - unfortunately - likely to be very high.
For that reason, agreeing with Richard here, it'd be prudent to ensure
that SimpleWebKit could be easily swapped-out and replaced at runtime
by a future port of WebKit (with all those tens of people at Apple,
webkit.org and the KHTML crowd chasing compatibility with awful markup
for us), and although this can be left for a later date it'd be smart
to think about *how* to do this easily fairly soon.

> The question remains to me why we should write a (new?) basic xhtml
> view when we can have a full (x)html view - to avoid duplication.

At this rate, we'll have running code (thanks for the screenshot by
the way!) available for integration soon.

The previously-attempted port of WebKit has stalled and any new port
is (as far as I know, currently) vapourware. Choosing between
something available and vapourware is easy I think: there is no actual
duplication right now!

The benefits of the future port of WebKit (all-of-web functionality,
extremely bullet-proof and widely tested code, devteam numbering in
the tens of people) are not significantly diminished by having
SimpleWebKit in the tree. I still need the all-of-web functionality,
for example, as do others! Are any more people into helping with the
bounty?

Integrating SimpleWebKit into our tree is quite beneficial because it
makes some functionality available that we need (source code and Nib
compatibility with OSX, basic (X)HTML functionality), it is available
(Nikolaus is building it for his project), it is suitable for
limited-resource platforms, and there are other added social bonuses
in that we can add a little more development effort by enhancing and
testing SimpleWebKit which can be shared back to QuantumStep if
Nikolaus wishes.

In summary, I'd like SimpleWebKit added, and when it's in svn, perhaps
more people can help out.

I'd be interested in helping with the CSS components once I understand
a little more Nikolaus' approach to rendering.

Peter

[Dr. H. Nikolaus Schaller]

Am 07.03.2007 um 01:42 schrieb Peter Cooper:

> Hello
>
> The current progress in SimpleWebKit is very gratifying, I did a
> test build of the code from a few days ago and the code compiled
> on GNUstep (with only one slight change to an #ifdef), and a simple
> palette seemed quite buildable (although in that version of the
> code there was no content rendering).

Great! Maybe, you can share the GNUmakefile?

>
>> Ok, that brings up a different aspect - maintaining an own web
>> browser
>> is a duplicate of work. And that is a completey different topic/
>> decision.
>
> I agree, web browser application development is a bit out of scope
> here. I'm not sure it fits into the GNUstep-as-library, or GNUstep-
> as-dev-env philosophy at all, but it probably works for the GNUstep-
> as-desktop-env people.

Well, WebKit is more a library - allowing to easily build a browser...

There have been suggestions to rename it from SimpleWebKit - why not
simply call it GNUstep-htmlengine? There is GNUstep-base, GNUstep-gui
which is pretty descriptive of what the role of the library is. It is
a base library, a GUI library and this one is a htmlengine.

>
> I'd be interested in helping with the CSS components once I
> understand a little more Nikolaus' approach to rendering.

Basically the approach is to parse HTML into a DOM tree (DOMHTML.h) -
exactly as KHTML/WebKit does. Then, rendering is done top down from
the WebHTMLView which has a single child, an NSTextView and the root
of the DOM tree. The NSTextView finally renders an attributedString
(well, an NSTextStorage), that is built from the DOM Tree. I.e.
building this attributedString does half of the work and the other
one is NSTextView.

This allows the trick for NSAttributedString(Additions) to fetch the
attributedString out of the WebView to e.g. convert to RTF.

Now, adding CSS would be a similar approach. Either, the CSS
description must load from a separate file first, or it is included
in the DOM tree as <style> tags. So, the first step would be to parse
the CSS into an DOMCSS tree. As soon as this is available, the DOM to
attributedString conversion can use it to return a well formatted
string. Finished loading of an external CSS must trigger the
rendering to be repeated since layout and style may have changed.
This is a bahaviour that you can see sometimes with Safari on a slow
connection - firstly, you get a page with raw text and then the
background, tables, images and formats come in.

Nikolaus

[Peter Cooper]

Hi

> Great! Maybe, you can share the GNUmakefile?

Here's one that compiles and installs SimpleWebKit as a library. A
framework is just as easy. I've also attached a small diff -c to one of
the files.

[snipped stuff about web browser being a bit out of scope]

> Well, WebKit is more a library - allowing to easily build a browser...

Yes. Agreed.

> There have been suggestions to rename it from SimpleWebKit - why not
> simply call it GNUstep-htmlengine? There is GNUstep-base, GNUstep-gui
> which is pretty descriptive of what the role of the library is. It is
> a base library, a GUI library and this one is a htmlengine.

I don't have strong opinions about this, but I would very much like the
compatibility of headers etc to exist. It'd be nice for Mac OS X
developers to be able to guess that the functionality exists in GNUstep,
which is a small plus point in retaining the name.

> Basically the approach is to parse HTML into a DOM tree (DOMHTML.h) -
> exactly as KHTML/WebKit does. Then, rendering is done top down from
> the WebHTMLView which has a single child, an NSTextView and the root
> of the DOM tree. The NSTextView finally renders an attributedString
> (well, an NSTextStorage), that is built from the DOM Tree. I.e.
> building this attributedString does half of the work and the other
> one is NSTextView.

Ok. And the rendering activity may create additional subviews? I can see
this being useful for optimisations to make fast fluid-layout redrawing
with the CSS box model possible. It'd be nice not to have to re-render the
entire document everytime the view is resized ;-)

> This allows the trick for NSAttributedString(Additions) to fetch the
> attributedString out of the WebView to e.g. convert to RTF.

Yup.

> Now, adding CSS would be a similar approach. Either, the CSS
> description must load from a separate file first, or it is included
> in the DOM tree as <style> tags. So, the first step would be to parse
> the CSS into an DOMCSS tree. As soon as this is available, the DOM to
> attributedString conversion can use it to return a well formatted
> string. Finished loading of an external CSS must trigger the
> rendering to be repeated since layout and style may have changed.
> This is a bahaviour that you can see sometimes with Safari on a slow
> connection - firstly, you get a page with raw text and then the
> background, tables, images and formats come in.

Cool. Do you want me to do translation of the DOM level 2 IDL files into
objc headers using the same approach (compatible names) as WebKit? Should
be achievable before the weekend.

Regards

Peter

[Gregory John Casamento]

> I don't really see what the problem is with having both SimpleWebKit
> and a port of WebKit ... then developers can choose.
>

> As for the xhtml text converter bundle Nicola suggests for use by the
> gui library ... the obvious thing IMO would be to write it to use
> SimpleWebKit, and perhaps provide an alternative bundle using WebKit
> later.

I have no problem with having both either. I am not trying to discourage development of one or the other. I was pointing out that the SimpleWebKit implementation might not handle everything that WebKit does because it will be less mature and have fewer people maintaining it. The advantage of SimpleWebKit is that we can have it in the GNUstep repo and that it will be light-weight.

Later, GJC

[Nicola Pero]

Ehi, great stuff! :-)

Just a small comment: your GNUmakefile contains the line

GNUSTEP_INSTALLATION_DIR = $(GNUSTEP_USER_ROOT)

Please remove it ... GNUSTEP_INSTALLATION_DIR is deprecated
as it won't work with FHS.

I suggest just leaving the installation domain unspecified in the GNUmakefile. ;-)

Thanks

PS: If you really want to specify the installation domain, you can add

GNUSTEP_INSTALLATION_DOMAIN = USER

before including common.make. ;-)

-----Original Message-----
From: Peter Cooper <com...@obverse.com.au>
Sent: Wed, March 7, 2007 2:55 pm
To: Dr. H. Nikolaus Schaller <h...@computer.org>
Cc: Discuss GNUstep <discuss...@gnu.org>, Riccardo <mul...@ngi.it>
Subject: Re: SimpleWebKit (was GNUstep Web browser (was Re: WebKit Bounty))

Hi

Yes. Agreed.

Yup.

Regards

Peter_______________________________________________

[Peter Cooper]

> Ehi, great stuff! :-)
>
> Just a small comment: your GNUmakefile contains the line
>
> GNUSTEP_INSTALLATION_DIR = $(GNUSTEP_USER_ROOT)

Ooops (should have cleaned it up before sending it :-)

> PS: If you really want to specify the installation domain, you can add
>
> GNUSTEP_INSTALLATION_DOMAIN = USER
>
> before including common.make. ;-)

Thanks for your work on GNUstep-make, Nicola, I'm sure you don't get
told nearly often enough what an awesomely fast and simple build
environment it is. Especially considering it's built on the crufty old
horror that is make!

Peter

[Dr. H. Nikolaus Schaller]

Peter,

Am 07.03.2007 um 14:55 schrieb Peter Cooper:

>> Basically the approach is to parse HTML into a DOM tree (DOMHTML.h) -
>> exactly as KHTML/WebKit does. Then, rendering is done top down from
>> the WebHTMLView which has a single child, an NSTextView and the root
>> of the DOM tree. The NSTextView finally renders an attributedString
>> (well, an NSTextStorage), that is built from the DOM Tree. I.e.
>> building this attributedString does half of the work and the other
>> one is NSTextView.
>
> Ok. And the rendering activity may create additional subviews? I
> can see
> this being useful for optimisations to make fast fluid-layout
> redrawing
> with the CSS box model possible. It'd be nice not to have to re-
> render the
> entire document everytime the view is resized ;-)

It is pretty fast on a Mac. Apparently needs just some milliseconds
to do
all that work.

View resizing does not need to re-render anything. Rendering geometry
is currently
done in the NSTextView. But I don't know if that changes by the CSS
box model.

>> This allows the trick for NSAttributedString(Additions) to fetch the
>> attributedString out of the WebView to e.g. convert to RTF.
>
> Yup.
>
>> Now, adding CSS would be a similar approach. Either, the CSS
>> description must load from a separate file first, or it is included
>> in the DOM tree as <style> tags. So, the first step would be to parse
>> the CSS into an DOMCSS tree. As soon as this is available, the DOM to
>> attributedString conversion can use it to return a well formatted
>> string. Finished loading of an external CSS must trigger the
>> rendering to be repeated since layout and style may have changed.
>> This is a bahaviour that you can see sometimes with Safari on a slow
>> connection - firstly, you get a page with raw text and then the
>> background, tables, images and formats come in.
>
> Cool. Do you want me to do translation of the DOM level 2 IDL files
> into
> objc headers using the same approach (compatible names) as WebKit?
> Should
> be achievable before the weekend.

That would be nice!

You can base the DOMCSSxxx.h files and classes by including DOMCore.h
And, you need to create a WebCSSDocumentRepresentation class that
handles getting the file in NSData fragments when the CSS is loaded
from an external source.
It should try to parse it as good as possible since the file may be
incomplete. You can take WebHTMLDocumentRepresentation as a sample.

The basic process for loading from external resources works as follows:
* a <link> element tells the HTML engine that it has to load an
external CSS
* it creates a new WebDataSource for it
* that one opens an NSURLConnection
* as soon as it is clear that it is a CSS resource, a new
WebCSSDocumentRepresentation is opened
* that one receives the new data from the NSURLConnection through -
(void) receivedData:(NSData *) data withDataSource:(WebDataSource *)
source;

What the HTML parser also needs to have is some -initWithString: to
directly handle the style attribute and <style> tag with inline CSS.
Well, it could also create a WebDataSource with a given NSData source
and go the standard way as described above but a shortcut would be
nice IMHO.

Finally, there must be a query method to find the style attributes
for a given tuple of (tag, tagpath, id, class).

Hope this works as you have intended...

Nikolaus

[Dr. H. Nikolaus Schaller]

Peter, Riccardo, Group,
just to keep you informed, I am still trying to find out why my GNA
account has no svn+ssh access...

In the meantime I am continuing to fix bugs and rework the
architectue. Here is a "status list" for some web pages

http://www.quantum-step.com loads main page but does not redirect
http://www.gnustep.org loads main page with formatting issues; does
not show images
http://www.fsf.org shows document title; hangs
http://www.google.com loads but does not show content
http://www.apple.com shows document title; hangs
http://www.apple.de loads but shows content only after reload
http://www.yahoo.de shows document title; hangs
frameset.html hangs with exception
headerbody.html hangs
headerbody.html hangs
robustness.html renders text more or less ok

Where I have no clear solution is how subresources (e.g. an embedded
<img>) correctly notify the visual elemens (which is a subclass of
NSImageCell embedded as an NSTextAttachment in an NSTextView) that
their content has changed.

Another unsolved issue is how the loading mechanism should enable the
CSS parser to handle <style type="text/css">@import "URL"</style>
which needs to resume parsing onlyy after the subresource data
becomes available. Well, parsing could trigger loading of the
subresource unless it has already been loaded and stop. Completion of
the subresource could restart the full CSS parse process.

Best regards,
Nikolaus

[Peter Cooper]

Hi Nikolaus

> Where I have no clear solution is how subresources (e.g. an embedded
<img>) correctly notify the visual elemens (which is a subclass of
NSImageCell embedded as an NSTextAttachment in an NSTextView) that their
content has changed.

My suggestion is to put a placeholder NSTextAttachment (perhaps a "broken
image" icon as per other browsers) into the NSTextView and defer putting
the real resource in until the resource has loaded. The image or object
tag exists in the DOM tree, and your queued loading activity can backfill
any references and the correct NSTextAttachment when complete.

> Another unsolved issue is how the loading mechanism should enable the
CSS parser to handle <style type="text/css">@import "URL"</style> which
needs to resume parsing onlyy after the subresource data
> becomes available. Well, parsing could trigger loading of the
> subresource unless it has already been loaded and stop. Completion of
the subresource could restart the full CSS parse process.

I'd agree with this. A key user experience optimisation would be creating
an idea of a "settling time" for the document - mark the document dirty
and wait for either a small time (0.5 seconds) with no further successful
resource loads or for the completion (successful or otherwise) of all
resource loads before triggering a full CSS parse and therefore
re-rendering.

Regards

Peter

[Dr. H. Nikolaus Schaller]

Am 15.03.2007 um 12:49 schrieb Peter Cooper:

> Hi Nikolaus
>
>> Where I have no clear solution is how subresources (e.g. an embedded
> <img>) correctly notify the visual elemens (which is a subclass of
> NSImageCell embedded as an NSTextAttachment in an NSTextView) that
> their
> content has changed.
>
> My suggestion is to put a placeholder NSTextAttachment (perhaps a
> "broken
> image" icon as per other browsers) into the NSTextView and defer
> putting

It is implemented that way and the placeholder already shows...

> the real resource in until the resource has loaded. The image or
> object
> tag exists in the DOM tree, and your queued loading activity can
> backfill

This is subissue #1: how is that properly notified through the
WebDataResource,
WebRepresentation and Web*View protocols and classes to an NSImageCell.

> any references and the correct NSTextAttachment when complete.

Subissue #2 is to really find the *correct* attachment and tell the
NSTextView to redraw just that one...
Finding the attachment could be easily solved by adding a pointer to
the DOM element for the <img> tag,
but knowing in which range of the NSTextStorage that attachment
resides might end up
in sequentially searching attribute runs... Well, not a nice/fast
solution but might be working.

>> Another unsolved issue is how the loading mechanism should enable the
> CSS parser to handle <style type="text/css">@import "URL"</style>
> which
> needs to resume parsing onlyy after the subresource data
>> becomes available. Well, parsing could trigger loading of the
>> subresource unless it has already been loaded and stop. Completion of
> the subresource could restart the full CSS parse process.
>
> I'd agree with this. A key user experience optimisation would be
> creating
> an idea of a "settling time" for the document - mark the document
> dirty
> and wait for either a small time (0.5 seconds) with no further
> successful
> resource loads or for the completion (successful or otherwise) of all
> resource loads before triggering a full CSS parse and therefore
> re-rendering.

This brings me to a different approach than the one I currently have
- let all
that dirty flags be handled within the NSView hierarchy. Then,
arriving new data
and updates of the DOM elements just set setNeedsDisplay:YES and
setNeedsLayout:YES
(there is a setNeedsLayout call in the WebDocumentView protocol for
that purpose).

And the drawRect: method is then called - as soon as the system has time
to do so. If it needs layout, the DOM trees are processed as good as
possible
to derive the best view that is currently available. If new data
comes in, the
dirty flags are set again and reparsing/redrawing occurs again.

The difference to my current approach is that in the new one, all
parsing is
only done as a response to a drawRect request (which might then become
quite time-consuming). This could constrain parsing to only the really
visible parts (*** although we must also parse the invisible parts
for e.g. JavaScript ***)

Currently, all parsing is done as a repsonse to new data having arrived.
Well not all - the handling of the <frame> and <frameset> tags is
triggered by the
View hierarchy. Maybe this is the reason why some pages don't display
anything - it appears that they do not get a drawRect: call.

Both approaches have their pros and cons - perhaps we should think about
a mixture?

[Mark Rowe]

Hi there,

I feel compelled to mention the incredibly large amount of work that
will be required to bring SimpleWebKit up to a reasonable standard of
web compatibility. HTML, XHTML, and CSS are not simple specifications.
Supporting them to a sufficient degree to have a usable web browser is
quite an ask. Add in the requirement for handling technologies like
JavaScript, the HTML and CSS DOMs, and browser plugins and you
significantly increase the complexity and man-hours required.

A common misunderstanding is that the only complex part in creating a
web browser is supporting "broken" web pages. It is true that this can
complicate matters, but the fact that no major web browsers have
*complete* support for the majority of web standards should give some
idea as to how much work is involved in implementing them. Internet
Explorer and Mozilla have been developed over the last 10 years,
primarily by full-time software engineers.

When the original GNUstep port[1] of WebKit was attempted the codebase
wasn't structured in a fashion conducive to ports. There was no easily
usable platform abstractions. Many portions of the code were in the
Objective-C WebKit layer and were dependent on AppKit implementation
details. We have remedied many of these issues, and now have one very
active port to the Qt toolkit, with further semi-active ports to Gdk
and Windows. Adobe has also ported WebKit to their Flash-based
rendering system. A lot of work has been put in to assisting porting
efforts with the aim of increasing portability and helping WebKit work
on as many platforms as is practical. I would strongly encourage any
interested developers to consider porting WebKit directly rather than
wasting a large amount of time in reimplementation.

Please get in touch[2] with us if anyone is interested in getting
involved with porting WebKit to GNUstep. It will almost certainly be a
quicker means of having a fully-featured web browser on GNUstep than
attempting to start from scratch.

Kind regards,

Mark Rowe
WebKit developer
<http://bdash.net.nz/>

[1] <https://gna.org/projects/gswebkit/>.
[2] <http://webkit.org/contact.html> has a number of types of contact
info, with IRC usually being one of the more active means.

[h...@computer.org]

On 19 Mrz., 01:07, Mark Rowe <b...@webkit.org> wrote:
> Hi there,
>
> I feel compelled to mention the incredibly large amount of work that
> will be required to bring SimpleWebKit up to a reasonable standard of

originally I wasn't either.

But doing it has shown that it is not at all that complicated as
everybody initially thinks.

Especially emphasizing the main benefits of Objective-C,
implementations are achieved very fast, with lower effort than
anything in C++.

I myself now start to wonder why WebKit was ever based on Objective-C+
+. This meant a lot of additional complexity for the Apple gcc
compiler tream by mangling two languages that are not easily
compatible. Take this effort into account, and the picture looks quite
different.

And the main reason for me why I simple can't use WebKit is that I am
tied for historical reasons on some systems of gcc-2.95.3 which does
NEVER support Objective-C++.

> web compatibility. HTML, XHTML, and CSS are not simple specifications.

No, they aren't. But Objective-C-only makes them still simple to
implement.

> Supporting them to a sufficient degree to have a usable web browser is

> quite an task. Add in the requirement for handling technologies like

> JavaScript, the HTML and CSS DOMs, and browser plugins and you

I have worked the last days on JavaScript and fortunately it is fully
specified here including all the semantics:
http://www.ecma-international.org/publications/standards/Ecma-262.htm

It is quite trivial to follow these specifications and code them into
Objective-C. E.g. if it says "Evaluate expression", you write [expr
_evaluate]. So, I already have implemented the expression parsing and
evaluation stuff. Missing is the statement stuff, but that is pure
typing work - having the spec in one window and the code in the other.

> significantly increase the complexity and man-hours required.

Approx. 20-30 man hours so far for the JavaScript engine. Here is some
snapshot code to look at:

http://www.quantum-step.com/download/sources/mySTEP/SimpleWebKit/WebScriptObject.m

Well, coding is one side of the coin and debugging the other.
Experience shows that this takes approx. 5 times as much...

> A common misunderstanding is that the only complex part in creating a
> web browser is supporting "broken" web pages. It is true that this can

I agree, that is not really complex compared to other aspects.

> complicate matters, but the fact that no major web browsers have
> *complete* support for the majority of web standards should give some
> idea as to how much work is involved in implementing them. Internet
> Explorer and Mozilla have been developed over the last 10 years,
> primarily by full-time software engineers.
>
> When the original GNUstep port[1] of WebKit was attempted the codebase
> wasn't structured in a fashion conducive to ports. There was no easily
> usable platform abstractions. Many portions of the code were in the
> Objective-C WebKit layer and were dependent on AppKit implementation
> details. We have remedied many of these issues, and now have one very
> active port to the Qt toolkit, with further semi-active ports to Gdk
> and Windows. Adobe has also ported WebKit to their Flash-based
> rendering system. A lot of work has been put in to assisting porting
> efforts with the aim of increasing portability and helping WebKit work
> on as many platforms as is practical. I would strongly encourage any
> interested developers to consider porting WebKit directly rather than
> wasting a large amount of time in reimplementation.

Well, that is a completely different project goal.

The target of the SimpleWebKit re-implementation is *not* to work on
as many platforms as possible, just to work with GNUstep-GUI (mySTEP-
GUI and Apple AppKit for debugging) and nothing else. This keeps the
project much more concise. And, before porting to other GUI systems
like Qt, it is IMHO better to port full GNUstep to other platforms
since that is what it is designed for: a platform independent OpenSTEP
implementation.

So, SimpleWebKit is kept simple by *not* even trying to satisfy
everybody and to be portable to completely different GUI toolkits.

> Please get in touch[2] with us if anyone is interested in getting
> involved with porting WebKit to GNUstep. It will almost certainly be a
> quicker means of having a fully-featured web browser on GNUstep than
> attempting to start from scratch.

Given the quite different goals of the projects, I see enough space
and need for both of them.

Please get in touch with the SimpleWebKit project if you favour the
direct, Objective-C-only approach and just want to have a browser for
GNUstep-GUI.

Kind regards,
Nikolaus Schaller

[Chris B. Vetter]

On 19 Mar 2007 01:59:48 -0700, h...@computer.org <h...@computer.org> wrote:
[...]

> I myself now start to wonder why WebKit was ever based on Objective-C++

[...]

The fact that WebKit is based on KHTML may have had to do with that decision.

--
Chris

[Rogelio Serrano]

Maybe apple thought that an objective-c only solution is not worth it?
Apple obviously believes that being able to talk directly to all the
c++ code out their is worth it.

--
the thing i like with my linux pc is that i can sum up my complaints in 5 items

[Riccardo]

Hi,

since I have seen that you commented on my blog and that your comment
generated other comments...

On 2007-03-19 01:07:17 +0100 Mark Rowe <bd...@webkit.org> wrote:

> Hi there,

> A common misunderstanding is that the only complex part in creating a
> web
> browser is supporting "broken" web pages. It is true that this can

> complicate matters, but the fact that no major web browsers have
> *complete*
> support for the majority of web standards should give some idea as to
> how
> much work is involved in implementing them. Internet Explorer and
> Mozilla
> have been developed over the last 10 years, primarily by full-time
> software
> engineers.

I think that nobody underestimates the time needed. But I think that
the big time spent developing mozilla is also due by many different
goals. I do believe that a compact, clean web engine can be written
with much less effort if some conditions are relaxed. I don't know
that the goals of Dr. Schaller are, but I myself would call me
satisfied if we get an engine with supports most of the used standards
in HTML, XHTML and CSS if they are written correctly, if it renders
acceptably broken pages and if a usable JavaScript is there. So if I
get 90% of gecko functionality with less bloat, less resources less
code I'd call myself satisfied.
If you look at how decently iCab did work (I haven't checked its
maintainance status lately), which is the effort of 2 people...

Although people will say Obj-C++ was needed because of the usage of
KHTML, I think that Apple underestimated the task of getting KHTML to
work cleanly and if those resources were put in better use a cleaner
and better WebKit woul dhave existed. And no need of ugly languages,
less crashes, better memory managment. And about speed - I don't know.

> Please get in touch[2] with us if anyone is interested in getting
> involved
> with porting WebKit to GNUstep. It will almost certainly be a
> quicker means
> of having a fully-featured web browser on GNUstep than attempting to
> start
> from scratch.

I think there is space for both engines - since they are going to be
API compatible. Both may exist on both APple and GNUstep platforms. I
see them in the short-mid-term as very different beasts which will
have differences in performance and ompatibility with different bugs.
At the moment I thought that devoting time to help Nikolaus in the
port is well invested time (given that there is other work needed to
better sync GNUstep and myStep... it is great time), since porting
WebKit will be a great task too, will probably need latest compilers
and will bind us at Apple.
What if in the future WebKit will need Obj-C 2.0 features - let us be
saved. Or it will ocnstantly need some ugly Core* stuff? Or some other
decisions whcih might hurt us or which we don't want to share?

both WebKit and SimpleWebKit have hurdles on the road and promise
different results. For now, I see substance in SimpleWebKit, if
someone else starts a GNUstep port of WebKit nothing prevents me to
help him too.

Nice Work,
Riccardo

[Yen-Ju Chen]

To me, there are two purposes to have a web engine.
One is to be a full browser, and WebKit is the way to go.
The other is a more simple one for email, rss reader or help viewer.
In that case, both SimpleWebKit and WebKit will do.

Yen-Ju

[Mark Rowe]

Hi there,

I'm not interested in getting into a debate about which direction
GNUstep should take for developing a web rendering engine. As a
complete outsider to the community anything I say has little influence,
and I have no place in trying to exert any. I am merely trying to
share some of the knowledge and experience that I have acquired over
the few years that I have been involved with WebKit. If the GNUstep
community decides to move towards a different solution, whether it be
SimpleWebKit or some other appropach, I wish you the best of luck.

I'll only address the points relevant to WebKit below:

On 03/20/07, Riccardo <mul...@ngi.it> said:

> Although people will say Obj-C++ was needed because of the usage of
> KHTML, I think that Apple underestimated the task of getting KHTML to
> work cleanly and if those resources were put in better use a cleaner
> and better WebKit woul dhave existed. And no need of ugly languages,
> less crashes, better memory managment. And about speed - I don't know.

Speed and memory footprint are *huge* factors in the choice of language
in a situation such as this. WebKit in Safari 2.0.4, the version
currently shipping with Mac OS X 10.4, is one of the fastest browsers
available on any platform. The current development version has had
substantial performance improvements in many areas. This is achieved
through a combination of smart algorithms and a programming language
that can take advantage of the computing resources available. While
Objective-C is a lovely language to develop in, it's dynamic nature
severly limits the possibility for performance optimisations by the
compiler.

If you believe Objective-C is the One True Programming Language then by
all means use it to develop a web platform. WebKit is in many ways
about using the right tool for the job. The performance-criticial
portions in WebCore and JavaScriptCore are written in C++. The
easy-to-use API that most developers see on a day-to-day basis is
Objective-C in the WebKit layer. Objective-C++ allows high
performance without sacrificing a developer-friendly API.

> [...] porting WebKit will be a great task too, will probably need

> latest compilers
> and will bind us at Apple.
> What if in the future WebKit will need Obj-C 2.0 features - let us be
> saved. Or it will ocnstantly need some ugly Core* stuff? Or some other
> decisions whcih might hurt us or which we don't want to share?

The core functionality in WebKit is in cross-platform C++ code. Both
JavaScriptCore and WebCore are buildable without any dependencies on
Apple-specific technologies, even on Mac OS X. The implementation of
the WebKit API layer, and it's hooks into JavaScript via
WebScriptObject etc, currently has some dependencies on either
CoreFoundation, Mac OS X private APIs, or Cocoa implementation details.
A port to GNUstep would involve factoring these platform-specifics out
of the main code.

WebKit currently makes use of Objective-C 2.0 features if you're
building it on a prerelease version of Mac OS X 10.5. This is purely
an API nicety, it is no way used in the implementation of WebKit
itself. As far as I know there a no plans to use any Objective-C 2.0
language features in WebKit in the near future. If GNUstep developers
are involved with WebKit there would be a good incentive to keep the
current state of play.

Kind regards,

Mark Rowe
<http://bdash.net.nz/>

[Yen-Ju Chen]

I think this quotation from Mark's comment on blog
summarizes what work needed to be done:

"JavaScriptCore and WebCore are quite portable,
and have already been ported to Qt, Gdk and Windows.
Work would need to go in at the WebCore layer to support
GNUsteps rendering primitives (the Mac OS X version use
CoreGraphics so isn't directly portable). Work would also need
to go in at the WebKit level to have it function on GNUstep
without its Mac OS X dependencies."

Yen-Ju

>
> Kind regards,
>
> Mark Rowe
> <http://bdash.net.nz/>
>
>
>
>

[Daniel J Farrell]

Hi folks,

What can GNUStepWeb be used to do? Are there any web application
examples I could have a look at online? Are there any tutorial on how
to get started?

Cheers,

Dan.

[David Wetzel]

Hi,

expect a merge of my branch with trunk in not so long.
GSW can do everything WO can do :-)

have a look at http://www.ambassadorrv.de/GSWeb/Camper.woa to see a real life RV rental application.

> _______________________________________________
> Discuss-gnustep mailing list
> Discuss...@gnu.org
> http://lists.gnu.org/mailman/listinfo/discuss-gnustep
>

--
_ _
_(_)(_)_ David Wetzel, Turbocat's Development,
(_) __ (_) Buchhorster Strasse 23, D-16567 Muehlenbeck/Berlin, FRG,
_/ \_ Fax +49 33056 82835 Phone +49 33056 82834
(______) http://www.turbocat.de/

[Daniel J Farrell]

Hi,

I get a feel for what it can do now, thanks.

Dan

[YL]

Any comments about the relationship among WO451, GNUStepWeb and SOPE?
Thanks a lot.

YL

On 21 Mar 2007, at 17:38, David Wetzel wrote:

> Hi,
>
> expect a merge of my branch with trunk in not so long.
> GSW can do everything WO can do :-)
>
> have a look at http://www.ambassadorrv.de/GSWeb/Camper.woa to see a
> real life RV rental application.
>
>> Hi folks,
>>
>> What can GNUStepWeb be used to do? Are there any web application
>> examples I could have a look at online? Are there any tutorial on how
>> to get started?
>>
>> Cheers,
>>
>> Dan.
>> _______________________________________________
>> Discuss-gnustep mailing list
>> Discuss...@gnu.org
>> http://lists.gnu.org/mailman/listinfo/discuss-gnustep
>>

[Helge Hess]

On Mar 21, 2007, at 21:00, YL wrote:
> Any comments about the relationship among WO451, GNUStepWeb and SOPE?

Well, SOPE was developed at MDlink since about 1996. It evolved from
the WebObjects compatible library libNGObjWeb to the rather large set
of frameworks it is today (it also includes stuff like IMAP4 protocol
support, iCalendar/vCard parsing, XML processing, LDAP clients, etc
etc).
The initial motivation was that we needed an application server for
our groupware software, that we loved WO (2.0? at the time) but that
it was impossible to use WO for the application due to its price (~
$50.000 at the time). So we had to write an own one ...

SOPE was proprietary software until mid-2003. It got released as LGPL
when the OpenGroupware.org project was formed.

I think Manuel started to work on gnustep-web ~2000 when SOPE was
still closed source, so there was no other (ObjC) web framework. In
fact he included a few things I could make free software at the time.

Well, and WebObjects ;-) I think gnustep-web still targets being a
1:1 WO 4.5 clone, but they can probably tell you more exactly.
SOPE doesn't, we don't aim for full WO compatibility anymore but
rather enhanced WO concepts in significant ways. Eg we introduced a
lot of concepts from Zope. Of course it still almost completely WO
API compatible.

Greets,
Helge
--
Helge Hess
http://www.helgehess.eu/

[jhcl...@charter.net]

On 2007-03-18 20:07:17 -0400 Mark Rowe <bd...@webkit.org> wrote:

> I feel compelled to mention the incredibly large amount of work that will be

> required to bring SimpleWebKit up to a reasonable standard of web
> compatibility. HTML, XHTML, and CSS are not simple specifications. Supporting
> them to a sufficient degree to have a usable web browser is quite an ask. Add

> in the requirement for handling technologies like JavaScript, the HTML and

> CSS DOMs, and browser plugins and you significantly increase the complexity
> and man-hours required.
>

> A common misunderstanding is that the only complex part in creating a web
> browser is supporting "broken" web pages. It is true that this can
> complicate matters, but the fact that no major web browsers have *complete*
> support for the majority of web standards should give some idea as to how
> much work is involved in implementing them. Internet Explorer and Mozilla
> have been developed over the last 10 years, primarily by full-time software
> engineers.

Not only so, but there's the fact that a lot of websites are tested in one, MAYBE two browsers. If you're not bug-for-bug compatible with Gecko, IE, or WebKit, you're wasting your time. Period. Using an existing engine is a force multiplier. Not using an existing engine is insanity, unless you have a very restricted set of goals. If you want a web browser, your goals are not so limited. Anybody who thinks "I'll just implement those handy open specs on w3c.org!" is on a fool's errand.

Go ahead and do it if you want, but it's not suitable for a web browser, especially with a small set of developers.

[Rogelio Serrano]

I also believed that 5 years ago...

Thats why i never "wasted" any time on it. But then i realized that if
i just steadily worked on in it since then i would something more
usable today.

Lets see what we have 5 years from now...

I would have condemned Edison for trying to invent the light bulb you
know and criticize his 1% success rate...

[h...@computer.org]

> Not only so, but there's the fact that a lot of websites are tested in one, MAYBE two browsers. If you're not bug-for-bug compatible with Gecko, IE, or WebKit, you're wasting your time. Period. Using an existing engine is a force multiplier. Not using an existing
engine is insanity, unless you have a very restricted set of goals.
If you want a web browser, your goals are not so limited. Anybody
who thinks "I'll just implement those handy open specs on w3c.org!" is
on a fool's errand.

Quite offending talk, don't you also think so?

And: where would Linux or GNUstep or Gecko or IE or WebKit be now if
they had followed your recommendations? We would still all use BSD and
Netscape 1.0. Period.

> Go ahead and do it if you want, but it's not suitable for a web browser, especially with a small set of developers.

How do you know before you have done it? Can you predict the future?
And has the pure number of developers ever been an indicator for
project success?

-- hns

[Mark Rowe]

On 03/23/07, "h...@computer.org" <h...@computer.org> said:

> And: where would Linux or GNUstep or Gecko or IE or WebKit be now if
> they had followed your recommendations? We would still all use BSD and
> Netscape 1.0. Period.

Gecko, IE, and WebKit all have commercial backing. KHTML, prior to
Apple's involvement, had a very insubstantial market share and suffered
from one of the problems I alluded to earlier: great standards support,
but poor real-world compatibility. This is still true to a small
extent with Webkit and has been a big area of focus. GNUstep is still
an incredibly niche platform, and it could be argued that Linux has
only headed in the direction of the mainstream when commercial backing
became common.

> How do you know before you have done it? Can you predict the future?
> And has the pure number of developers ever been an indicator for
> project success?

You don't know, but the web isn't going to sit still for the next two
or three years while a fledgling browser catches up. The Internet is
becoming a central part of the computer experience, and many new
technologies are being introduced to meet needs as the arise. Keeping
up with the start of the art takes work.

- Mark

[Stefan Bidigaray]

I haven't chimed in this conversation so far just because I really don't know anything.  But what I would like to think is that the goals of SimpleWebKit are not the same as that of WebKit.  Since it's announcement of this list, I got the impression that SimpleWebKit was to serve as a WebKit for browsers in applications where resources are scarce, after all it did come from the myStep project, which is a GNUstep fork with PDAs in mind.  In my mind, SimpleWebKit is not being developed to compete with Gecko/WebKit/IE, but to serve as an embeddable framework where WebKit is simply overkill, and there are many applications that need such properties.

 

If what you are saying is true, that means there would really be no market for browsers such as Links, Lynx, Dillo, etc yet these browsers are fairly popular for the market they serve.  I've used all 3 of the ones I listed at one point or another, and I understand that for what I wanted to do at the time, anything else would be too much.

 

Stefan

[Helge Hess]

On Mar 23, 2007, at 15:31, Mark Rowe wrote:
> Gecko, IE, and WebKit all have commercial backing. KHTML, prior to
> Apple's involvement, had a very insubstantial market share and
> suffered from one of the problems I alluded to earlier: great
> standards support, but poor real-world compatibility.

I suppose nobody questions that, its just that the goals are
different. Apparently Nikolaus just wants to build a browser with
great standards support (maybe thats why its named *Simple* WebKit?).
So why bother him? ;-)

The only thing I'm a bit concerned about is how the two
implementations would live side by side if SimpleWebKit directly
implements WebKit classes instead of using a separate class hierarchy
which is then somehow mapped to the WebKit classes.
To bad we don't have namespaces in ObjC ;-)

>> How do you know before you have done it? Can you predict the future?
>> And has the pure number of developers ever been an indicator for
>> project success?

> You don't know, but the web isn't going to sit still for the next
> two or three years while a fledgling browser catches up. The
> Internet is becoming a central part of the computer experience, and
> many new technologies are being introduced to meet needs as the
> arise. Keeping up with the start of the art takes work.

As a matter of fact a cleanroom implementation of those "many new
technologies" is quite often much faster to implement than tweaking
10..15 year old, aging C++ code which needs to be super-cautions wrt
changes.

Well, thats a rather stupid discussion, no? Why not stop it? ;-) I'm
quite interested in what Nikolaus comes up with, though I also doubt
it will be useful for *me* personally.
Well, even if just a small JavaScript ObjC interpreter comes out of
this, it would be a very cool and useful thing! :-)

[h...@computer.org]

On 23 Mrz., 16:40, Helge Hess <helge.h...@opengroupware.org> wrote:
>
> I suppose nobody questions that, its just that the goals are
> different. Apparently Nikolaus just wants to build a browser with
> great standards support (maybe thats why its named *Simple* WebKit?).

Exactly - we just want implement the standard and simple things so
that they work. Not to please everybody, no portability to different
platforms, not to handle workarounds on the server side for rare bugs
of other browsers etc.

> The only thing I'm a bit concerned about is how the two
> implementations would live side by side if SimpleWebKit directly
> implements WebKit classes instead of using a separate class hierarchy
> which is then somehow mapped to the WebKit classes.
> To bad we don't have namespaces in ObjC ;-)

There is no problem to expect with that. Both implement the documented
WebView and WebFrame API in two different frameworks. You can link to
either one in your applications (there is IMHO no reason to have both
in a single application). So, from API point of view they are
interchangeable. Maybe, not from functionality.

> Well, even if just a small JavaScript ObjC interpreter comes out of
> this, it would be a very cool and useful thing! :-)

Yes... it is already working well for simple expressions like
(3+5)*8+" km". Parsing of the whole ECMAScript syntax is at approx.
90%. Evaluation lacks handling of variables and control structures.
And, the builtin Script-Prototypes like Array, Boolean plus their
methods wait for implementation.

Nikolaus Schaller

[Nicolas Roard]

On 3/23/07, Helge Hess <helge...@opengroupware.org> wrote:
> On Mar 23, 2007, at 15:31, Mark Rowe wrote:

> The only thing I'm a bit concerned about is how the two
> implementations would live side by side if SimpleWebKit directly
> implements WebKit classes instead of using a separate class hierarchy
> which is then somehow mapped to the WebKit classes.
> To bad we don't have namespaces in ObjC ;-)

There's no problem for having two HTML engines coexisting in WebKit
from what I understood -- so I hope that SimpleWebKit is written
taking that into account :)

> >> How do you know before you have done it? Can you predict the future?
> >> And has the pure number of developers ever been an indicator for
> >> project success?

> > You don't know, but the web isn't going to sit still for the next
> > two or three years while a fledgling browser catches up. The
> > Internet is becoming a central part of the computer experience, and
> > many new technologies are being introduced to meet needs as the
> > arise. Keeping up with the start of the art takes work.
>
> As a matter of fact a cleanroom implementation of those "many new
> technologies" is quite often much faster to implement than tweaking
> 10..15 year old, aging C++ code which needs to be super-cautions wrt
> changes.

yes... Well I doubt that SimpleWebKit will provide anytime soon a
"useful" browser, as in, good enough to browse most of the existing
websites. But providing a browser implementing standard (X)HTML ? it's
not that hard imho. And such a "simple" browser would be already quite
neat (documentation, etc), even if perhaps you won't be able to use it
to connect to your bank account...

> Well, thats a rather stupid discussion, no? Why not stop it? ;-) I'm
> quite interested in what Nikolaus comes up with, though I also doubt
> it will be useful for *me* personally.

I think it might be very useful for documentation or things like RSS
posts at the very least!

> Well, even if just a small JavaScript ObjC interpreter comes out of
> this, it would be a very cool and useful thing! :-)

Speaking of that... aren't you supposed to release that shiny
WebScript implementation ? ;-)

--
Nicolas Roard
"La perfection, ce n'est pas quand il n'y a plus rien à ajouter, c'est
quand il n'y a plus rien à retrancher." -- Antoine de St-Exupéry

[Helge Hess]

On Mar 23, 2007, at 17:57, Nicolas Roard wrote:
> But providing a browser implementing standard (X)HTML ? it's
> not that hard imho.

Well, its probably not hard from a functional point of view. But
getting it sufficiently fast (especially CSS) might be not as trivial.
But thats just pure guesswork, we'll see what Nikolaus comes up with.

> And such a "simple" browser would be already quite
> neat (documentation, etc), even if perhaps you won't be able to use it
> to connect to your bank account...

Definitely.

>> Well, thats a rather stupid discussion, no? Why not stop it? ;-) I'm
>> quite interested in what Nikolaus comes up with, though I also doubt
>> it will be useful for *me* personally.
> I think it might be very useful for documentation or things like RSS
> posts at the very least!

Yes.

>> Well, even if just a small JavaScript ObjC interpreter comes out of
>> this, it would be a very cool and useful thing! :-)
> Speaking of that... aren't you supposed to release that shiny
> WebScript implementation ? ;-)

Hehe, I'm trying to find the time to grab the sources and send it to
you. As mentioned its "somewhat working", but quite incomplete.
Possibly you manage to bring it into shape? ;-)

[Rogelio Serrano]

On 3/24/07, Helge Hess <helge...@opengroupware.org> wrote:
> On Mar 23, 2007, at 17:57, Nicolas Roard wrote:
> > But providing a browser implementing standard (X)HTML ? it's
> > not that hard imho.
>
> Well, its probably not hard from a functional point of view. But
> getting it sufficiently fast (especially CSS) might be not as trivial.
> But thats just pure guesswork, we'll see what Nikolaus comes up with.
>
> > And such a "simple" browser would be already quite
> > neat (documentation, etc), even if perhaps you won't be able to use it
> > to connect to your bank account...
>
> Definitely.
>

unless it was an atom protocol based website and uses xhtml only. even
without javascript and cookies you can do a lot.

[jhcl...@charter.net]

<<Quite offending talk, don't you also think so?>>

It wasn't intended to be. I just felt the need to amplify what Mark had said about the task of web compatibility and the massive task that building a resilient rendering engine is. It wasn't directed at you or your efforts, which are perfectly fine. (I was tired when I wrote the email and probably didn't communicate effectively.)

<<And: where would Linux or GNUstep or Gecko or IE or WebKit be now if they had followed your recommendations? We would still all use BSD and Netscape 1.0. Period.>>

Those projects went through many years of testing web applications, complicated CSS layouts, buggy JavaScript code, crazy user input and server output, etc. The sheer amount of stuff they've learned to account for over the years is staggering. It's not easy to create an end-user web browsing renderer that accounts for these things in a short amount of time.

<<How do you know before you have done it? Can you predict the future? And has the pure number of developers ever been an indicator for project success?>>

Because I've observed the chaotic nature of web development. Even people who are fanatical about standards cannot agree on what they really mean and what the best practices are. Just take a look at some of the flame-fests over XHTML vs. HTML and semantic markup vs. preserving deprecated-but-not-necessarily-deprecated tags like B and I. And then there are all the subtle interractions between CSS rules. And the people who use FrontPage because they don't know better. And the pages written in Notepad in 1994. And on and on and on. That's why I say it's a lot more complicated than just following the specs if you want a web browser (the original topic of this thread) for end-users.

But, as I say, your design goals are not a web browser, so this doesn't really apply to your valiant efforts. (And if, by some miracle, you produce the next great browser, I'll be first in line to shake your hand!)

[Andreas Wagner]

Hello Nicolas, hello list,

* Nicolas Roard wrote on Mär/23/2007:
> I think it might be very useful for documentation or things like RSS posts at the very least!

Finally that posting tickled my memory and reminded me of something that Nick Bradbury once wrote in his blog. I
looked it up and here it is -- just for the archives.

http://nick.typepad.com/blog/2006/09/fixing_funky_fe_1.html

...the same problem of catering for malformed content seems to surface in RSS feeds, too. (which is still an issue
different from displaying documentation) There's nothing in particular I want to convey by pointing out that posting,
but I think maybe it helps to just gather as much info about different approaches as possible. Oh well, at least the
bandwidth isn't stressed too much by a mail such as this one.

Cheers,
Andreas

--
Komme gleich wieder - Godot

--
My Public PGP Keys:
1024 Bit DH/DSS: 0x869F81BA
768 Bit RSA: 0x1AD97BA5

[h...@computer.org]

> >> Well, even if just a small JavaScript ObjC interpreter comes out of
> >> this, it would be a very cool and useful thing! :-)
> > Speaking of that... aren't you supposed to release that shiny
> > WebScript implementation ? ;-)
>
> Hehe, I'm trying to find the time to grab the sources and send it to
> you. As mentioned its "somewhat working", but quite incomplete.
> Possibly you manage to bring it into shape? ;-)

I would be happy to upload to the GNUstep repository but I am still
fighting to make my gna svn+ssh-access work. I uploaded different keys
several times... And I even copied one from the server of a working
installation. No success...

A trouble ticket wiht gna admins has already been issued, so it can't
be far away. BTW: making this work appears more complex than writing a
Web browser :-) The reason is simple: for a web browser you have full
control and you can modify your client to make it work. For ssh/svn
you can't.

Nikolaus

[Riccardo]

Heym

On 2007-03-24 10:20:33 +0100 h...@computer.org <h...@computer.org> wrote:

>>
>> Hehe, I'm trying to find the time to grab the sources and send it to
>> you.
>> As mentioned its "somewhat working", but quite incomplete. Possibly
>> you
>> manage to bring it into shape? ;-)
>
> I would be happy to upload to the GNUstep repository but I am still
> fighting to make my gna svn+ssh-access work. I uploaded different keys
> several times... And I even copied one from the server of a working
> installation. No success...

Since Nikolaus has problem with access, the svn merge is completely on
my shoulders, I have merged in a working subsed of SimpleWebKit and
moivng on from that. Javascript is not yet in though. Be patient.

-Riccardo

[Helge Hess]

On Mar 23, 2007, at 17:50, h...@computer.org wrote:
> There is no problem to expect with that. Both implement the documented
> WebView and WebFrame API in two different frameworks. You can link to
> either one in your applications (there is IMHO no reason to have both
> in a single application).

Well, I disagree with that and its exactly what I'm concerned about.
Since the API is *not* the same (the implemented HTML is obviously
part of the API in this case!), the code should be able to select
which implementation it needs.

And since an application can be composed from different frameworks
(say an RSS framework and a browser framework), its also extremely
useful to have both in one application.
In fact I expect that a lot of frameworks and apps will use HTML
plugins instead of native GUI elements over time (sometime
SimpleWebKit might be perfect for). Just check NewsFire or Adium.

If you only want to build two versions of a web browser, it doesn't
matter. But if you want to integrate an HTML engine in an app and use
it as a core component, it matters a lot.

[Helge Hess]

On Mar 23, 2007, at 20:49, Andreas Wagner wrote:
> ...the same problem of catering for malformed content seems to
> surface in RSS feeds, too.

The little issues discussed in the link is myriads away from the
compat issues in real world HTML/CSS/JS!!! :-)

[h...@computer.org]

On 25 Mrz., 12:18, Helge Hess <helge.h...@opengroupware.org> wrote:
> On Mar 23, 2007, at 17:50, h...@computer.org wrote:
>
> > There is no problem to expect with that. Both implement the documented
> > WebView and WebFrame API in two different frameworks. You can link to
> > either one in your applications (there is IMHO no reason to have both
> > in a single application).
>
> Well, I disagree with that and its exactly what I'm concerned about.
> Since the API is *not* the same (the implemented HTML is obviously
> part of the API in this case!), the code should be able to select

I would not say that it is API. It is different capabilities - some of
them are mandatory and others optional. Would you also see NSImage's
set of supported formats as part of the API?

> which implementation it needs.

Hm. Since everybody appears to agree that full WebKit is more powerful
and robust and SimpleWebKit can only be a subset, you should in that
case simply use the ported WebKit and not SimpleWebKit.

I have no idea how to really solve that otherwise - besides renaming
all classes to avoid name conflicts. But that hurts those >80%
applications which just need either one.

> And since an application can be composed from different frameworks
> (say an RSS framework and a browser framework), its also extremely
> useful to have both in one application.

Yes. But neither WebKit nor SimpleWebKit supports RSS (directly). So,
you can link either one with the RSS framework. But both have a
compatible plugin architecture for adding new MIME content types since
it is part of WebView.

> In fact I expect that a lot of frameworks and apps will use HTML
> plugins instead of native GUI elements over time (sometime
> SimpleWebKit might be perfect for). Just check NewsFire or Adium.

There are IMHO exactly two sets of interfaces which applications
should use to remain portable:
Web* classes and NSAttributedString's -initWithHTML:

> If you only want to build two versions of a web browser, it doesn't
> matter. But if you want to integrate an HTML engine in an app and use
> it as a core component, it matters a lot.

I think -[NSAttributedString initWithHTML:] and -[WebView
stringByEvaluatingJavaScriptFromString:] should be the standard means
of integrating the HTML engine from a software architectural point of
view with the KISS-principle in mind. You would have to do it that way
as well if you port full WebKit or run on Cocoa. Or do you think of a
different task?

Nikolaus

[Andreas Wagner]

Hello Helge, hello list,

* Helge Hess wrote on Mär/25/2007:

> On Mar 23, 2007, at 20:49, Andreas Wagner wrote:
>> ...the same problem of catering for malformed content seems to
>> surface in RSS feeds, too.
>
> The little issues discussed in the link is myriads away from the
> compat issues in real world HTML/CSS/JS!!! :-)

I see. Sorry then.

Regards,
Andreas

--
Beware of low-flying butterflies.

[Helge Hess]

On Mar 25, 2007, at 12:49, h...@computer.org wrote:
> Hm. Since everybody appears to agree that full WebKit is more powerful
> and robust and SimpleWebKit can only be a subset, you should in that
> case simply use the ported WebKit and not SimpleWebKit.

I don't think so. Embedding full Gecko or KHTML probably requires
much more resources than SimpleWebKit. As mentioned several times by
others there are more uses to HTML than just web browsing :-)

> I have no idea how to really solve that otherwise - besides renaming
> all classes to avoid name conflicts. But that hurts those >80%
> applications which just need either one.

Right now we have no applications using either one, not sure where
you get the 80% number :-).
And how it will be used also remains to be seen :-) I'd say that
potentially 80+ *will* use both (because SimpleWebKit could be used a
lot for widgets, not for regular HTML) and still will need to branch
into the public web.

You just need to do the API as a frontend, exactly like its done for
KHTML. Don't know how this will hurt any application. It makes the
backend slightly more complex (because of the WebKit layer).

> Yes. But neither WebKit nor SimpleWebKit supports RSS (directly). So,
> you can link either one with the RSS framework.

Of course, but I don't want to load a 10MB browser on startup when I
just need to render a few tags :-)

> But both have a compatible plugin architecture for adding new MIME
> content types since it is part of WebView.

Yes, thats totally unrelated.

>> In fact I expect that a lot of frameworks and apps will use HTML
>> plugins instead of native GUI elements over time (sometime
>> SimpleWebKit might be perfect for). Just check NewsFire or Adium.
>
> There are IMHO exactly two sets of interfaces which applications
> should use to remain portable:
> Web* classes and NSAttributedString's -initWithHTML:
>
>> If you only want to build two versions of a web browser, it doesn't
>> matter. But if you want to integrate an HTML engine in an app and use
>> it as a core component, it matters a lot.
> I think -[NSAttributedString initWithHTML:] and -[WebView
> stringByEvaluatingJavaScriptFromString:] should be the standard means
> of integrating the HTML engine from a software architectural point of
> view with the KISS-principle in mind. You would have to do it that way
> as well if you port full WebKit or run on Cocoa. Or do you think of a
> different task?

Just take a look at how the applications I mentioned use WebKit.
Using HTML for regular UI stuff is getting more and more common.

Anyways, you probably want to concentrate on coding instead of
discussions ;-) Actually I'm *most* interested in a GNUstep/xyzStep
for the N800. And this involves a browser which is sufficiently small
to run on that device.

[h...@computer.org]

> I don't think so. Embedding full Gecko or KHTML probably requires
> much more resources than SimpleWebKit. As mentioned several times by

Agreed.

> others there are more uses to HTML than just web browsing :-)

Yes. Sure. There is e.g. handling HTML formatted mails. Or viewing
Help pages etc.

> > I have no idea how to really solve that otherwise - besides renaming
> > all classes to avoid name conflicts. But that hurts those >80%
> > applications which just need either one.
>
> Right now we have no applications using either one, not sure where
> you get the 80% number :-).

The <20% is on those applications where I can imagine that you need a
simple
and a full web rendering engine in parallel. A Web browser just needs
one.
A HTML-Mailer just needs one. A help viewer just needs one.

Well, you may run a browser and a help viewer in parallel in separate
processes
and both can use a differently powerful set of WebKit.frameworks.

But in my understanding this is a different point - or I still do not
understand the problem.

> And how it will be used also remains to be seen :-) I'd say that
> potentially 80+ *will* use both (because SimpleWebKit could be used a
> lot for widgets, not for regular HTML) and still will need to branch
> into the public web.

But why would you want to duplicate things by adding SimpleWebKit if
you have a
full WebKit available?

> You just need to do the API as a frontend, exactly like its done for
> KHTML. Don't know how this will hurt any application. It makes the
> backend slightly more complex (because of the WebKit layer).

Hm. I think the architecture is quite different from what you assume.

Simple webkit has no backend besides AppKit NSView and NSCell.

It implements all the @interfaces of WebView, WebFrame, DOMHTML
etc. based on Foundation and AppKit. And most generally, it is a
single
subclass of NSView that can be integrated into an application:
WebView.

It provides a method to take an URL and view its contents. Much
similar
to NSImageView which gets an NSImage and displays it.

Well, and a Browser is a GUI application that uses this WebView in
each
document window and provides some bells and whistles. Not more.

> > Yes. But neither WebKit nor SimpleWebKit supports RSS (directly). So,
> > you can link either one with the RSS framework.
>
> Of course, but I don't want to load a 10MB browser on startup when I
> just need to render a few tags :-)

Well, with full WebKit, you load 4 frameworks: WebKit, WebCore,
JavaScriptCore, JavaScriptGlue.
With SimpleWebKit, you load just one: (Simple)WebKit.framework

Currently, the code size (incl. symbols) for an Intel Mac is 1.6 MByte
and for the ARM processor (libWebKit.so) 1.2 MByte.

> >> In fact I expect that a lot of frameworks and apps will use HTML
> >> plugins instead of native GUI elements over time (sometime
> >> SimpleWebKit might be perfect for). Just check NewsFire or Adium.

> Using HTML for regular UI stuff is getting more and more common.

Yes, I understand this approach - but it makes things much slower
instead of using the builtin widgets (well Moore's law will help).

And, I have still the impression that you don't have to include two
rendering engines in those applications. Just choose one and send it
the appropriate HTML and JavaScript strings.

> Anyways, you probably want to concentrate on coding instead of
> discussions ;-) Actually I'm *most* interested in a GNUstep/xyzStep
> for the N800. And this involves a browser which is sufficiently small

Yes... Me too :-)

> to run on that device.

Since it runs on the Sharp Zaurus, it should also work on the N800 (as
soon as I have a working EABI cross compiler on MacOS X - that costs
me more headaches).

Now, let's put up the sleeves again and fix bugs...

Nikolaus