Intent to Implement: Web Application Manifest

[Mounir Lamouri]

Primary eng/PM emails:
mlam...@chromium.org

Spec:
https://w3c.github.io/manifest/

Summary:
"This specification defines a JSON-based manifest, which provides
developers with a centralized place to put metadata associated with a
web application. This includes, but is not limited to, the web
application's name, links to icons, as well as the preferred URL to open
when the user launches the web application. The manifest also allows
developers to declare a default orientation for their web application,
as well as how the application is to be displayed by the user agent
(e.g. in fullscreen)."

Motivation:
The motivation is to have a centralized place to put metadata for web
applications, as the summary describes it. More specifically, we want to
have a standard and improved way to "bookmark to homescreen" which is
currently a feature available in most mobile browsers but is non optimal
and proprietary (reading all sort of <meta> elements). A manifest could
improve that and be a base for future improvements.

Compatibility Risk:
* Firefox: Shipped a proprietary API, working on implementing the new
standard specification
* Internet Explorer: Public support
* Safari: No public signals
* Web developers: No signals

Describe the degree of compatibility risk you believe this change poses:
Low, as is. However, additions to the manifest might include
compatibility risks and we should make sure that they follow a process
including API owners.

Ongoing technical constraints:
None.

Will this feature be supported on all five Blink platforms (Windows,
Mac, Linux,
Chrome OS, and Android)? Yes
Every platform is going to be able to detect, parse and read the
manifest. It will be then up to the embedder to use it or not. As far as
Chrome is concerned, I do not think there are plans to use it outside of
Chrome Android but we could definitely use it elsewhere. For example,
the streamlined hosted apps experiment could use this manifest.

OWP launch tracking bug:
https://code.google.com/p/chromium/issues/detail?id=366145

Entry in Chromium Dashboard:
http://www.chromestatus.com/admin/features/edit/6338383617982464

Requesting approval to ship?
No.

-- Mounir

[Elliott Sprehn]

I still have a concern that this is JSON. One of the most annoying things with chrome apps manifests is you can't add comments and you can't comment out lines. This makes it difficult to know why lines are there, or remove a line temporarily when debugging, or leave examples in the file.

Even the examples in the spec use comments, which makes them invalid.

Can we please add support for comments?

[Victor Costan]

As a developer, I would like to see more support for json5. I think it
would fit well here.
http://json5.org/

Asides from allowing comments, I really like that it allows trailing
commas in arrays and objects.
Victor

> To unsubscribe from this group and stop receiving emails from it, send an
> email to blink-dev+...@chromium.org.

[Jonas Sicking]

I agree that the lack of comments in JSON is a huge headache.

However one of the reasons that we made the manifest use JSON rather
than a custom format was to enable tooling both on the consuming side
and on the producing side, and to enable future expansions of the
format. Another obvious advantage with using JSON is that it's a
format familiar to developers and so it's easy for humans to interpret
and write.

I would expect the manifest format to grow fairly complex over time.
I.e. the set of properties that we might want to stick in there is
likely to get pretty complex. Examples of what we have ended up adding
in FirefoxOS are:

https://developer.mozilla.org/en-US/docs/WebAPI/Web_Activities#App_manifest_%28a.k.a._declaration_registration%29
https://developer.mozilla.org/en-US/Apps/Build/Manifest#permissions
https://developer.mozilla.org/en-US/Apps/Build/Manifest#locales

I'm not saying that these exact features should get added to
manifests, but I think we'll see equally complex features added over
time.

At one point I tried extending the appcache manifest to turn it into
something that could reasonably work and eventually gave up. It simply
got too messy. I fear that if we create our own custom data format for
the manifest we'll end up in a similar situation.

That said, I'd love to see json5 happen. It would solve the comment
issue and many more. But I'm reluctant to start that movement with
manifests as there's a risk that it'll end up as an odd-man-out.

I'd rather that we try to make json5 happen orthogonally, and once
that seems successful we can define that manifests are json5 rather
than json.

In the meantime we could make the manifest spec explicitly recommend
using properties named '//' to indicate comments. I.e. you'd write
something like:

{
"//": "The name for the icon once installed",
"name": "TheAwesomeGame",

"//": "Lots of icons here",
"icons": { ... }
}

The spec could even normatively call out "//" properties as something
that must always be ignored.

/ Jonas

[Nils Barth]

​

Jonas Sicking

​

:

​​

​​

​​

​​

I agree that the lack of comments in JSON is a huge headache.

However one of the reasons that we made the manifest use JSON rather
than a custom format was to enable tooling both on the consuming side
and on the producing side, and to enable future expansions of the
format. Another obvious advantage with using JSON is that it's a
format familiar to developers and so it's easy for humans to interpret
and write.

​​

​

​​

If you'd like comments in a JSON-compatible standard format, how about YAML?

Technically JSON+comments (with unique keys) is valid YAML (see YAML: Relation to JSON), admittedly in the inline format which one generally doesn't use.

Admittedly a YAML parser is more complex than a JSON parser, and perhaps overkill just for comments, but for producing you can produce JSON (sans comments) and it's valid YAML.

Other techniques of course are naming conventions (// or prefix with _), or the hideous hack of duplicating keys, using previous values of a given key as comments.

[Victor Costan]

On Wed, Apr 23, 2014 at 11:23 PM, Jonas Sicking <jo...@sicking.cc> wrote:
> I'd rather that we try to make json5 happen orthogonally, and once
> that seems successful we can define that manifests are json5 rather
> than json.

I agree that json5 is completely orthogonal to app manifests.
Upgrading to it should not block the app manifest work. I wanted to
point it out because I found it to be well thought-out, and I wanted
to make the folks in this thread aware of it. It seems a bit better
than JSON+comments, and I'd really like to be able to use it some day.

Victor

[Kenneth Rohde Christiansen]

I am of course biased, but LGTM :-) Would it be possible to allow embedders to extend the manifest parser via the content/ API?

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

--
Kenneth Rohde Christiansen
Web Platform Architect, Intel Corporation.
Phone  +45 4294 9458 ﹆﹆﹆

[Adam Barth]

On Wed Apr 23 2014 at 11:05:09 PM, Kenneth Rohde Christiansen <kenneth.ch...@gmail.com> wrote:

I am of course biased, but LGTM :-) Would it be possible to allow embedders to extend the manifest parser via the content/ API?

I'm worried that manifest will become a dumping ground for random features.  Instead of letting embedders extend the manifest via the content API, I'd rather that additional manifest semantics went through the usual web platform launch process.

Adam

[Kenneth Rohde Christiansen]

Hi Adam,

I generally agree with you, but the reason that I am asking is that we are using content/ API for our runtime which supports packaged apps.

For this reason we do need a few other keys for now that have not yet been standardized and it would be great to avoid having to patch our downstream branches of Chromium/Blink unnecessarily.

I suppose we can add the hooks ourselves (downstream) unless the design explicitly makes this hard. 

Kenneth

[Adam Barth]

I don't think we should support that use case in trunk.  Obviously the code is open source and you're welcome to modify it in any way you desire locally.

Adam

[Ian Hickson]

On Thu, 24 Apr 2014, Mounir Lamouri wrote:
>
> https://w3c.github.io/manifest/

As far as I can tell, this is almost entirely redundant with existing
features in HTML.

For examples, the "icons" feature is equivalent to <link rel=icon>, the
"display" member is equivalent to <meta name=application-name>, etc.

> More specifically, we want to have a standard and improved way to
> "bookmark to homescreen" which is currently a feature available in most
> mobile browsers but is non optimal and proprietary (reading all sort of
> <meta> elements). A manifest could improve that and be a base for future
> improvements.

I don't really understand why we can't just standardise on <meta> names,
since that's what people are using already, and it avoids requiring yet
another round-trip to get the data. It also avoids having yet another
redundant source of data that could lead to inconsistencies and
difficulties maintaining code -- e.g. if the manifest has different icons
than the page.

--
Ian Hickson U+1047E )\._.,--....,'``. fL
http://ln.hixie.ch/ U+263A /, _.. \ _\ ;`._ ,.
Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'

[Jake Archibald]

On Tue, Apr 29, 2014 at 5:31 AM, Ian Hickson <i...@hixie.ch> wrote:

I don't really understand why we can't just standardise on <meta> names,

since that's what people are using already, and it avoids requiring yet
another round-trip to get the data.

The round-trip isn't really a problem, this can be a low-priority download and highly cacheable. That's better than having it inline in the head.

 

It also avoids having yet another
redundant source of data that could lead to inconsistencies and
difficulties maintaining code -- e.g. if the manifest has different icons
than the page.

The meta tags have given us inconsistency. It means that each page of an 'app' can have different icons, start points & application titles.

[PhistucK]

Plus, including the same thing in all of the pages (for SEO reasons, basically, which this manifest might also accommodate?) is also a sizable chunk that should be cached but cannot be (on dynamic pages that are not cacheable). It could get to a kilobyte or more (meta tags for SEO, Open Graph information and more).

Even if you use GZIP, this stuff persists in the DOM. A simple waste of memory.

☆PhistucK

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Ian Hickson]

On Tue, 29 Apr 2014, Jake Archibald wrote:
>
> The round-trip isn't really a problem, this can be a low-priority
> download and highly cacheable. That's better than having it inline in
> the head.

We're not talking about much markup here.

> > It also avoids having yet another redundant source of data that could
> > lead to inconsistencies and difficulties maintaining code -- e.g. if
> > the manifest has different icons than the page.
>
> The meta tags have given us inconsistency. It means that each page of an
> 'app' can have different icons, start points & application titles.

Well, sure, just like they can point to a different manifest.

On Tue, 29 Apr 2014, PhistucK wrote:
>
> Plus, including the same thing in all of the pages (for SEO reasons,
> basically, which this manifest might also accommodate?) is also a
> sizable chunk that should be cached but cannot be (on dynamic pages that
> are not cacheable). It could get to a kilobyte or more (meta tags for
> SEO, Open Graph information and more). Even if you use GZIP, this stuff
> persists in the DOM. A simple waste of memory.

Well the stuff is going to be in RAM anyway.

The data we're talking about is de minimis. A few URLs, a few strings. The
HTTP headers you'd send to fetch the manifest or that you'd get back
describing the manifest's caching rules would be bigger than the data
we're talking about here.

The costs of having multiple ways to do these things is IMHO much higher
than any cost from having this inside the application's page itself.

[Alex Russell]

On Mon, Apr 28, 2014 at 9:31 PM, Ian Hickson <i...@hixie.ch> wrote:

On Thu, 24 Apr 2014, Mounir Lamouri wrote:
>
> https://w3c.github.io/manifest/

As far as I can tell, this is almost entirely redundant with existing
features in HTML.

The other way to phrase this is that HTML has invented many features which are so redundant in each-and-every page that a manifest is now an attractive way to become DRY [1].

 

For examples, the "icons" feature is equivalent to <link rel=icon>, the
"display" member is equivalent to <meta name=application-name>, etc.

> More specifically, we want to have a standard and improved way to
> "bookmark to homescreen" which is currently a feature available in most
> mobile browsers but is non optimal and proprietary (reading all sort of
> <meta> elements). A manifest could improve that and be a base for future
> improvements.

I don't really understand why we can't just standardise on <meta> names,
since that's what people are using already, and it avoids requiring yet
another round-trip to get the data.

This misreads the intent (as Jake notes below). The round-trip is incurred only for features that aren't time sensitive, unlike the meta-tag-soup we currently force into the most precious bytes in the document. 

 

It also avoids having yet another
redundant source of data that could lead to inconsistencies and
difficulties maintaining code -- e.g. if the manifest has different icons
than the page.

[1] http://en.wikipedia.org/wiki/Don't_repeat_yourself 

[Ian Hickson]

On Tue, 29 Apr 2014, Alex Russell wrote:
> On Mon, Apr 28, 2014 at 9:31 PM, Ian Hickson <i...@hixie.ch> wrote:
> > On Thu, 24 Apr 2014, Mounir Lamouri wrote:
> > >
> > > https://w3c.github.io/manifest/
> >
> > As far as I can tell, this is almost entirely redundant with existing
> > features in HTML.
>
> The other way to phrase this is that HTML has invented many features
> which are so redundant in each-and-every page that a manifest is now an
> attractive way to become DRY [1].

If the goal is to remove redundant data from HTML, then we should do that.
The manifest doesn't do that.

HTML Imports might be a better way to do that. That way we would get all
the benfits of not reinventing the wheel, with all the caching and DRY
benefits of the external file.

> This misreads the intent (as Jake notes below). The round-trip is
> incurred only for features that aren't time sensitive, unlike the
> meta-tag-soup we currently force into the most precious bytes in the
> document.

I think this overstates the case. The number of bytes we're talking about
here is de minimis.

[Alex Russell]

On Tue, Apr 29, 2014 at 11:02 AM, Ian Hickson <i...@hixie.ch> wrote:

On Tue, 29 Apr 2014, Alex Russell wrote:
> On Mon, Apr 28, 2014 at 9:31 PM, Ian Hickson <i...@hixie.ch> wrote:
> > On Thu, 24 Apr 2014, Mounir Lamouri wrote:
> > >
> > > https://w3c.github.io/manifest/
> >
> > As far as I can tell, this is almost entirely redundant with existing
> > features in HTML.
>
> The other way to phrase this is that HTML has invented many features
> which are so redundant in each-and-every page that a manifest is now an
> attractive way to become DRY [1].

If the goal is to remove redundant data from HTML, then we should do that.
The manifest doesn't do that.

HTML Imports might be a better way to do that.

Interesting!

 

That way we would get all
the benfits of not reinventing the wheel, with all the caching and DRY
benefits of the external file.

> This misreads the intent (as Jake notes below). The round-trip is
> incurred only for features that aren't time sensitive, unlike the
> meta-tag-soup we currently force into the most precious bytes in the
> document.

I think this overstates the case. The number of bytes we're talking about
here is de minimis.

It's not the amount, it's their location in the document. Content in the header is under massive contention for inclusion in the first packet which has a massive impact on our ability to get other operations under way quickly (e.g., DNS warm up, content pre-fetching, etc.).

[Joshua Bell]

Although it does not appear to be called out in the spec, I've heard the desire expressed to be able to consume this manifest data in non-browser services - e.g. "submit my web app to a web app catalog service".

Teaching a service to parse JSON is going to be easier than teaching it to parse HTML. That may be moot if we expect developers to submit http://app.example.com instead of http://app.example.com/manifest.json since you'd need to parse the HTML resource at the former to dig out the manifest link anyway. And perhaps parsing HTML on the server, at least enough to dig the links out of the head, is a solved problem.

[Ian Hickson]

On Tue, 29 Apr 2014, Alex Russell wrote:
> On Tue, Apr 29, 2014 at 11:02 AM, Ian Hickson <i...@hixie.ch> wrote:
> > On Tue, 29 Apr 2014, Alex Russell wrote:
> > > On Mon, Apr 28, 2014 at 9:31 PM, Ian Hickson <i...@hixie.ch> wrote:
> > > > On Thu, 24 Apr 2014, Mounir Lamouri wrote:
> > > > >
> > > > > https://w3c.github.io/manifest/
> > > >
> > > > As far as I can tell, this is almost entirely redundant with
> > > > existing features in HTML.
> > >
> > > The other way to phrase this is that HTML has invented many features
> > > which are so redundant in each-and-every page that a manifest is now
> > > an attractive way to become DRY [1].
> >
> > If the goal is to remove redundant data from HTML, then we should do
> > that. The manifest doesn't do that.
> >
> > HTML Imports might be a better way to do that.
>
> Interesting!

> > > This misreads the intent (as Jake notes below). The round-trip is
> > > incurred only for features that aren't time sensitive, unlike the
> > > meta-tag-soup we currently force into the most precious bytes in the
> > > document.
> >
> > I think this overstates the case. The number of bytes we're talking
> > about here is de minimis.
>
> It's not the amount, it's their location in the document. Content in the
> header is under massive contention for inclusion in the first packet
> which has a massive impact on our ability to get other operations under
> way quickly (e.g., DNS warm up, content pre-fetching, etc.).

Yeah, that's probably reasonable.

Doing this with HTML Imports would address that too.

[Jonas Sicking]

On Tue, Apr 29, 2014 at 11:02 AM, Ian Hickson <i...@hixie.ch> wrote:
> On Tue, 29 Apr 2014, Alex Russell wrote:
>> On Mon, Apr 28, 2014 at 9:31 PM, Ian Hickson <i...@hixie.ch> wrote:
>> > On Thu, 24 Apr 2014, Mounir Lamouri wrote:
>> > >
>> > > https://w3c.github.io/manifest/
>> >
>> > As far as I can tell, this is almost entirely redundant with existing
>> > features in HTML.
>>
>> The other way to phrase this is that HTML has invented many features
>> which are so redundant in each-and-every page that a manifest is now an
>> attractive way to become DRY [1].
>
> If the goal is to remove redundant data from HTML, then we should do that.
> The manifest doesn't do that.
>
> HTML Imports might be a better way to do that. That way we would get all
> the benfits of not reinventing the wheel, with all the caching and DRY
> benefits of the external file.

Using HTML Imports here would have two big downsides:

1. It would force the metadata information to be downloaded when it's
not needed. A plain <link rel=import> is going to cause blocking
behavior whenever <script> is used (i.e. quite often). This is totally
unneeded for the metadata that we're wanting to express here. Even if
we ask people to use <link rel=import async> it still means that the
manifest is going to compete with other resources that are also
loading async (which we're hoping eventually will be a lot).

2. The current format used to express metadata is not very elegant to
extend. See the email I sent previously in this thread for some
examples of the level of complexity of metadata that we're likely to
eventually will need to express. What would those examples look like
if we stick to <meta> elements?

The short of it is that the current <meta> elements is a fairly poorly
designed feature. The data that we're talking about here isn't really
data about the page. It's data about a group of pages that make up a
conceptual "app". Expressing that as metadata for a single page will
create awkwardness, no matter if we allow it to be syntactically be
broken out into a separate file or not.

Another way to look at it is this. Say that we didn't have <script
src=...> and <link rel=stylesheet> and instead just had
<script>...</script> and <style>...</style>. Would we have designed
external stylesheets and external scripts by using <link rel=import>
pointing at HTML files containing single <script> or <style> elements?

Would that be an improvement or worse for developers?

/ Jonas

[Jon Rimmer]

On the other hand, HTML does at least support comments, whereas JSON does not. Would we have designed scripting and stylesheet languages that didn't support comments?

If there's a requirement to base the manifest format on an existing language syntax, wouldn't CSS be a better choice? It's widely understood and reasonably tool-able, nicer for humans to write and parse due to the lack of constant quoting, and supports comments. Tab Atkins proposed the use of CSS-like syntax as a general mechanism for setting properties in HTML a couple of years ago [1], and it seems like it would be a good fit here.

[1] http://lists.w3.org/Archives/Public/public-webapps/2012JulSep/0508.html

Jon

 

[Tobie Langel]

Might be worth asking developers what format they would rather use. I doubt either metadata tag soup (with its inevitable custom grammars) or a new CSS-inspired syntax _for data_ would cut it.

JSON, despite its flaws, is hugely popular among Web developers. And it is already used by the Chrome store, Mozilla Market Place and node.js package manager (npm).

--tobie

[Dimitri Glazkov]

I heard "HTML Imports", so had to jump in :)

It would've been exciting to re-imagine Manifest functionality as an
HTML Imports thing. I can already see myself inventing custom element
vocabularies for manifest properties with maniacal glee. However..

There are smart folks who worked on this for a while. Mozilla has
shipping code that's gotten mileage and tons of learned experience.
The spec is not terrible, and mostly relies on plumbing that we
already have in Web platform. We don't have time to sit and ponder new
wonderful things that are years away.

I say we keep moving forward with this.

:DG<

[Eric Seidel]

It does seem like Manifests are solving some of the very same problems
as our current <meta> tag soup does. But I guess <meta> tags as
manifests only work well for single-page applications?

Maybe multi-page applications should just link back to their main/root
page as a manifest? I'm not convinced from reading the specification
why we need a new JSON format for this.

index.html
<meta soup>

other_page.html
<meta type="manifest" contents="/index.html">

As far as I (naively) can tell, that still solves the stated goals of
the spec of "provides developers with a centralized place to put

metadata associated with a web application. This includes, but is not
limited to, the web application's name, links to icons, as well as the
preferred URL to open when the user launches the web application. The
manifest also allows developers to declare a default orientation for
their web application, as well as how the application is to be

displayed by the user agent".

Without spending tens of pages to specify a new comment-hostile,
out-of-band format. :)

Maybe the goal is that manifests will eventually be so large that we
wouldn't want to load them as part of the main document? Then maybe
making them async <meta type="manifeset" contents="/manifest.html">
even in the main document could make sense?

I'm just not sold on the complexity of out-of-band JSON. meta tag
soup seems to work pretty well AFAICT.

I guess what I'm asking for is to have the spec updated to better
justify its choices. I trust that smarter people than I have already
looked at this, but I'm not sold by the current spec's arguments.

[Ojan Vafai]

+1 to Dimitri's we should just move forward with this. As per the original post, Firefox is already shipping a variant of this API. The benefits of a slightly better colored bikeshed don't seem worth the cost forcing them to change course, or, even worse, of ending up with two different APIs that web developers need to support. I don't see benefit in making the spec editors spend more of their limited time trying to better explain their decisions. This API is good enough.

I agree with Jonas that we should pursue extending JSON and also that doing so should be decoupled from moving forward with manifests. We'd likely support such a project in Blink (although, maybe this would actually live in V8). Also, just inferring from Jonas's comment, it sounds like Firefox might support extending JSON as well. JSON5 or some variant thereof seems reasonable to me. It'd be nice if we also addressed JSON supporting different newline characters than JavaScript while we're adding a new JSON version.

[Eric Seidel]

Adam explained to me offline that the reason why Manifest is json
instead of other existing formats is to make it trivial to be parsed
by something other than a renderer process (like the network stack or
the OS launcher, etc.)

That seems reasonable that a launcher may need to parse some file to
know which URL to open for your "saved to homescreen" web app.

In that case I can see the need for this and agree with Ojan that we
should just match whatever FF has shipped as I'm sure there are a many
shades of paint here.

I could anticipate that json may be too weak a format for manifests
long-term, for the reasons discussed above, but we can always have a
manifests v2.

[Jonas Sicking]

On Mon, May 5, 2014 at 9:53 AM, Eric Seidel <ese...@chromium.org> wrote:
> In that case I can see the need for this and agree with Ojan that we
> should just match whatever FF has shipped as I'm sure there are a many
> shades of paint here.

To be clear, the manifests that Firefox is shipping is only through a
prefixed API (navigator.mozApps) which was added before we have the
stricter ship policy than Blink and Firefox now uses.

And what we're shipping now is not compatible with what's in the
specification draft. Both because what we're currently shipping is
*only* driven through an API call which displays security dialog, and
because the actual property names in the JSON file are different.

So I don't think compatibility with what Firefox is shipping is any
stronger of an argument than compatibility with what Chrome Apps are
shipping. I.e. the spec is equally (in)compatible with both.

That said, I do obviously support using JSON here. And the fact that
both Firefox and Chrome both created JSON formats independently speaks
to that it has some merit.

/ Jonas