comments in package.json

12,443 views
Skip to first unread message

Alex Kocharin

unread,
Jan 5, 2013, 1:22:06 PM1/5/13
to nod...@googlegroups.com
Hello, everybody.


TL;DR: I think that JSON is not a suitable config file format, and I want npm to be able to read configs stored in some other way by default. It might be just javascript, or yaml, I don't really care as long as it better for configuration files than json.


So, there is a dependency list in package.json, and it would be a good practice to have a comment for every line describing why we require that package, why we require that version of that package, what known problems we have and so on.

But there's a small issue. JSON format doesn't allow comments in any way.

Right now there are a couple of different ways around it of course:

1. Non-standard JSON entries like "@comment": "blablabla". Unfortunately, javascript editors doesn't highlight it as a comment, and it's just plain ugly. Also this violates strict javascript mode, so God knows what trouble it'll cause in the future.
2. Keep a commented dependency list in a separate file. This violates DRY principle, so we could update one file and forget to update another. The same goes for /**package **/ hack I believe.
3. Use some kind of build system. Just for damn comments in one file?

Also, there's another wrong thing with JSON, it's too strict. You can't omit double quotes from keys, you can't leave a trailing comma, etc. JSON is human-readable, but it's just not damn human-writable.

Well... I went for 3rd option for a very long time. We used package.js file and a Makefile that compile js to json. Yes, that's three damn files instead of one. That's an example of our package.js file. https://gist.github.com/4462764 . But a number of supported packages grew, and compiling this slowly became a major pain in the ass. I recently got an issue when I updated package.js, but forgot to compile it, and debugging this one was a quite interesting experience. So, I'm now in a mood of forking things and making all my public packages incompatible with mainstream npm...


So, there's a couple of alternatives. For example, Travis use YAML, and there is CSON (it's coffeescript version with blackjack and hookers).

And I think there was a couple of discussions about it. So, did anybody come up with more or less sane idea how to deal with this? What happened to package.json.js?


Happy New Year!

Ilya Dmitrichenko

unread,
Jan 5, 2013, 3:35:21 PM1/5/13
to nod...@googlegroups.com
Why cannot you add a section on dependencies in the README file? There you can explain in plain-english whatever you wanna say about those dependencies!

Cheers,
-- 
Ilya 



--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Rick Waldron

unread,
Jan 5, 2013, 3:44:29 PM1/5/13
to nod...@googlegroups.com
I'd be more concerned with having configuration options that were obtuse enough to require in-line comments. 


Rick

Eric Mill

unread,
Jan 5, 2013, 4:54:35 PM1/5/13
to nod...@googlegroups.com
It can often be a good idea to add comments for yourself and others around your dependencies, especially on a large project. It doesn't mean you made bad choices about your dependencies. When stuff gets large, it helps to group things, label them, etc.

As package.json's get used for more and more things (for example, my deploys to my app host involve setting custom fields in package.json that they use to govern DNS and stuff), it'll become handier to have the ability to comment things. 

Also, to comment things in and out at will, during development. We all do that with things.

Plus, yes, being able to drop the quotes around keys is nice too. 

This is why when I make config files for myself, I make them .js files instead of .json. Preface the object with a "module.exports = ", and you can say "var config = require("./config")" very easily. It's a lot more convenient.

-- Eric

Dick Hardt

unread,
Jan 5, 2013, 5:21:15 PM1/5/13
to nod...@googlegroups.com
Or you could add a comment property to the JSON

, comment: "Here is a comment"

npm will ignore it, or may use it in the future for showing comments in some way

Dick Hardt

unread,
Jan 5, 2013, 5:22:50 PM1/5/13
to nod...@googlegroups.com
I of course meant:

, "comment": "Here is a comment"

Rick Waldron

unread,
Jan 5, 2013, 5:22:45 PM1/5/13
to nod...@googlegroups.com
On Sat, Jan 5, 2013 at 4:54 PM, Eric Mill <konk...@gmail.com> wrote:
It can often be a good idea to add comments for yourself and others around your dependencies, especially on a large project.

What more does a configuration file need to convey about dependencies, beyond "This is a dependency; this is the required version."? Anything more then that belongs in actual documentation and not doing so reeks of negligence and sloppy practices.

Rick

Luke Arduini

unread,
Jan 5, 2013, 5:55:41 PM1/5/13
to nod...@googlegroups.com
This sounds like a nightmare.

Alex Kocharin

unread,
Jan 5, 2013, 8:27:19 PM1/5/13
to nod...@googlegroups.com
Hi Ilya,
 
What all comments in source code are used for?
 
We can just strip *.js out of comments and place well commented *.js.txt file nearby. Why don't anybody do that? :)
 
--
Regards,
Alex
 
 
06.01.2013, 00:35, "Ilya Dmitrichenko" <errorde...@gmail.com>:

Alex Kocharin

unread,
Jan 5, 2013, 8:36:40 PM1/5/13
to nod...@googlegroups.com
Hi Dick,
 
 
I explained that in an initial message. Well... it won't hurt to repeat though.
 
1. This is not highlighted by editors properly
2. This is too long, dozen characters instead of two or three
3. No multiline comments
4. Isn't allowed by strict javascript (so, we might expect some trouble in the far future)
5. Looks ugly
 
--
Regards,
Alex
 
 
06.01.2013, 02:21, "Dick Hardt" <dick....@gmail.com>:

Dick Hardt

unread,
Jan 5, 2013, 9:13:12 PM1/5/13
to nod...@googlegroups.com

On Jan 5, 2013, at 5:36 PM, Alex Kocharin <al...@kocharin.ru> wrote:

> Hi Dick,
>
>
> I explained that in an initial message. Well... it won't hurt to repeat though.
>
> 1. This is not highlighted by editors properly
> 2. This is too long, dozen characters instead of two or three
> 3. No multiline comments
> 4. Isn't allowed by strict javascript (so, we might expect some trouble in the far future)
> 5. Looks ugly

Thanks for repeating.

1) life is not perfect
2) not really

,"comment":"
is all that is in addition to comment

3) you can make the comment string as long as you want
4) uh -- not sure what you are talking about here, it is a JSON file -- it all depends on what the program reading it cares about, which in this case is NPM -- most programs ignore unknown JSON
5) well, that is an opinion

This solves your problem without requiring 1,000s of people to change their package configuration, and most people don't have a need for comments

Given there has been no support for your proposal to date, and there is a work around, perhaps spend your time on something else?

… just trying to help (and cut down on list noise)

-- Dick

Martin Cooper

unread,
Jan 5, 2013, 9:23:38 PM1/5/13
to nod...@googlegroups.com
On Sat, Jan 5, 2013 at 5:36 PM, Alex Kocharin <al...@kocharin.ru> wrote:
Hi Dick,
 
 
I explained that in an initial message. Well... it won't hurt to repeat though.
 
1. This is not highlighted by editors properly
2. This is too long, dozen characters instead of two or three

    "//": "shorter"
 
3. No multiline comments

    "comment": [
        "This",
        "is",
        "a",
        "multiline",
        "comment"
    ]

--
Martin Cooper

 

Trevor Norris

unread,
Jan 6, 2013, 1:27:46 AM1/6/13
to nod...@googlegroups.com, al...@kocharin.ru
On Saturday, January 5, 2013 10:22:06 AM UTC-8, Alex Kocharin wrote:
Hello, everybody.


TL;DR: I think that JSON is not a suitable config file format, and I want npm to be able to read configs stored in some other way by default. It might be just javascript, or yaml, I don't really care as long as it better for configuration files than json.

 Whether you think this or not, this has nothing directly to do with node. npm uses gyp, which is used by Google to build v8. And because node is built on top of v8, it makes sense node would use the same build mechanism. Also there is a standing rule that no changes will be made to external dependencies specifically for node. So if you wish to suggest that gyp accept another file format along with JSON I'd suggest posting it on the gyp mailing list (https://groups.google.com/group/gyp-developer).

Trevor Norris

unread,
Jan 6, 2013, 1:32:50 AM1/6/13
to nod...@googlegroups.com, al...@kocharin.ru
On Saturday, January 5, 2013 10:22:06 AM UTC-8, Alex Kocharin wrote:
1. Non-standard JSON entries like "@comment": "blablabla". Unfortunately, javascript editors doesn't highlight it as a comment, and it's just plain ugly. Also this violates strict javascript mode, so God knows what trouble it'll cause in the future.

Oh, and if you had bothered to read the specification you would have found this:

"All input is expected to comply with JSON, with two exceptions: the # character (not inside strings) begins a comment that lasts until the end of the line, and trailing commas are permitted at the end of list and dict contents." 

Alex Kocharin

unread,
Jan 6, 2013, 2:34:25 AM1/6/13
to nod...@googlegroups.com
 
Hi Trevor,
 
 
I'm talking about package configuration, not build configuration. gyp has absolutely nothing to do with it.
 
 
--
// alex
 
 
06.01.2013, 10:27, "Trevor Norris" <trev....@gmail.com>:

Arunoda Susiripala

unread,
Jan 6, 2013, 2:45:15 AM1/6/13
to nod...@googlegroups.com

I think commenting in package.json is not required. But nice to have. 
And there are lot of module that allow to write json with comments. 
I think its better to create a pull request to
Npm with a implementation and see issacs comment on this. 

I hope he wont mind to do a pull. :-)
--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en


--
Arunoda Susiripala


Eric Mill

unread,
Jan 6, 2013, 2:53:15 AM1/6/13
to nod...@googlegroups.com
Some of you are being extremely derisive to a reasonable suggestion. You may not think it is a suggestion worth adopting, and it may not be, but it's being made seriously and has stated reasons behind it. Explain why you don't see the problem raised as enough of a problem to merit the change, but you don't need to couch your reaction in language designed to make the suggester feel small.

Developers' responses to specific questions/suggestions with "Why do you want to do that? It sounds like you're making bad decisions." is a cliche that still happens all the time, and annoys the heck out of me.


--

Trevor Norris

unread,
Jan 6, 2013, 3:00:49 AM1/6/13
to nod...@googlegroups.com, al...@kocharin.ru
On Saturday, January 5, 2013 11:34:25 PM UTC-8, Alex Kocharin wrote:
 
Hi Trevor,
 
 
I'm talking about package configuration, not build configuration. gyp has absolutely nothing to do with it.


Please excuse me while I remove my foot. Well, imho it would make sense that the package.json file would follow the same standards as the gyp files. The consistency makes sense to me. You can always create a pull request with changes: https://github.com/isaacs/npm

Isaac Schlueter

unread,
Jan 6, 2013, 11:57:19 AM1/6/13
to nodejs, al...@kocharin.ru
As long as I have a say in the matter, package.json file will always
be json. If anything, future versions of npm and node will *remove*
features rather than add them. (I'm looking at you, `scripts.install`
;)

That being said, I completely understand the desire to put additional
data into your JSON configuration files. That's why the "//" key will
never be used by npm for any purpose, and is reserved for comments.

If you want to use a multiple line comment, you can use either an
array, or multiple "//" keys.

{ "//": "this is the first line of a comment",
"//": "this is the second line of the comment" }

{ "//": [
"first line",
"second line" ] }

If your editor doesn't display this to your satisfaction, then fix
your editor. Vim, emacs, sublime text, textmate, and most other
editors support custom extensions and color schemes. Some of those
are even open source, so you can really go nuts if you want to.

You're not a bad person if you use comments in your configs. I would
tend to read that as a code smell, but the world is a complicated
place, and we all sometimes do things we're not proud of. If you need
to add comments to your JSON config file, consider if perhaps the keys
are not understandable enough, or if there is some more semantic way
to express what you're trying to do.

(The point was made about comments in code. I have the same opinion
there: comments are a sign of an overly complicated or poorly
understood design. But often it's the lesser evil.)

Any pull requests to Node, npm, or read-package-json which add
pre-formatting, cson, package.js, #-comments, etc, will be rejected.
Patches unwelcome.

Rick Waldron

unread,
Jan 6, 2013, 8:06:10 PM1/6/13
to nod...@googlegroups.com


On Sunday, January 6, 2013, Eric Mill wrote:
.

Developers' responses to specific questions/suggestions with "Why do you want to do that? It sounds like you're making bad decisions." is a cliche that still happens all the time, and annoys the heck out of me.

Really? Im facepalming so hard right now that I probably have black eyes. It is _every_  _software_ _developers_ _obligation_ to always question the approaches and solutions of their peers and their own. Feelings?? bits don't have feelings. Bit operations aren't emotional. Software doesn't need a hug. 

If I'm "doing it wrong", I expect—no, demand—that my peers inform me and show me the best way, to the best of their understanding of the problem space. If it turns out they're doing it wrong too, then we'll learn together. If constructive criticism annoys you, you are the problem. 

-Rick

Eric Mill

unread,
Jan 6, 2013, 10:58:29 PM1/6/13
to nod...@googlegroups.com
It completely depends how you do it. You can assume the asker does not know what they're doing and try to snuff out their question as quickly as possible. Or, you can assume that they are also smart people who may have their own reasons, politely answer their question, while calmly pointing out that this is usually the wrong approach and suggest a better one. I see excellent posts on lists and on StackOverflow all the time that do this.

Bits don't have feelings, and bits don't ask questions of their peers - people do. The amount of sneering that a newcomer encounters when asking questions of a community online has a huge impact on how likely they are to continue participating, or to use that technology at all. If it's a young person or someone just figuring out whether or not a career or hobby in technology is for them, it can literally kill their motivation for the craft.

I'm not trying to put words in the OP's mouth, nor am I at risk of being discouraged (I've been a professional web dev for 7 years now), but I am continually mystified by how tonedeaf some people in the tech community can be. The service of truth doesn't remove our basic obligation to be polite and welcoming.


--

Jacob Groundwater

unread,
Jan 6, 2013, 11:26:43 PM1/6/13
to nod...@googlegroups.com, nodejs, al...@kocharin.ru

> As long as I have a say in the matter, package.json file will always
> be json. If anything, future versions of npm and node will *remove*
> features rather than add them. (I'm looking at you, `scripts.install`
> ;)

Now I am curious about the discussion around removing scripts.install.

Isaac Schlueter

unread,
Jan 7, 2013, 11:29:29 AM1/7/13
to nodejs
On Sun, Jan 6, 2013 at 8:26 PM, Jacob Groundwater <ja...@nodefly.com> wrote:
> Now I am curious about the discussion around removing scripts.install.

https://twitter.com/npmjs/status/285085545602904064
https://github.com/isaacs/npm/commit/7e6f0b8077b9c892391ed894a7bbe88f45fde152

Not much of a "discussion" per se, but more like something that most
eventually come to realize, but only *after* doing it the wrong way
first.

It'd be better to only support node-gyp for building packages, and get
rid of scripts.install altogether.

Jacob Groundwater

unread,
Jan 7, 2013, 12:51:40 PM1/7/13
to nod...@googlegroups.com
Not much of a "discussion" per se, but more like something that most
eventually come to realize, but only *after* doing it the wrong way
first.

It'd be better to only support node-gyp for building packages, and get
rid of scripts.install altogether.
 
Thanks. This makes sense for compiling, however I had been using the install key for compiling coffee-script and minimizing resources when my application deployed to Heroku. 

I realize it is not a great solution, but the default node.js buildpack calls `npm install` and then just starts.

Dick Hardt

unread,
Jan 7, 2013, 1:13:19 PM1/7/13
to nod...@googlegroups.com
I also use npm to package up my app for deployment and use an install script to do key generation and DB setup. Is there a preferred pattern for those tasks? Is npm not intended to be used for deploying apps?

Forrest L Norvell

unread,
Jan 7, 2013, 2:06:35 PM1/7/13
to nod...@googlegroups.com
On Mon, Jan 7, 2013 at 9:51 AM, Jacob Groundwater <ja...@nodefly.com> wrote:
Thanks. This makes sense for compiling, however I had been using the install key for compiling coffee-script and minimizing resources when my application deployed to Heroku. 

I realize it is not a great solution, but the default node.js buildpack calls `npm install` and then just starts.

Transpiling and minification are better handled by a prepublish script anyway: https://npmjs.org/doc/scripts.html That way, the module doesn't need to depend on CoffeeScript just to install and deployment will happen that much faster.

F

Jacob Groundwater

unread,
Jan 7, 2013, 2:14:36 PM1/7/13
to nod...@googlegroups.com

Transpiling and minification are better handled by a prepublish script anyway: https://npmjs.org/doc/scripts.html That way, the module doesn't need to depend on CoffeeScript just to install and deployment will happen that much faster.

Unless I am mistaken, `prepublish` is only called when doing `npm publish`, which you would never do for a web application.

Most PaaS providers accept git bundles of your project, and then deploy from there with `npm install`. The `prestart` key also is not a good fit if you are using a Procfile.

Perhaps `scripts.install` needs to be replaced with a `scripts.predeploy` or something specific for preparing an application for production.

Forrest L Norvell

unread,
Jan 7, 2013, 2:22:05 PM1/7/13
to nod...@googlegroups.com
Not speaking for Isaac here, but npm doesn't seem like a really great tool for deployment. I guess when I've handled this in the past, the way I've done it is to make my apps self-bootstrapping or done it through scripts outside the deployment tool.

But I do apologize for not noticing the distinction between app deployment and module publishing.

F

--

José F. Romaniello

unread,
Jan 7, 2013, 2:33:02 PM1/7/13
to nod...@googlegroups.com
prepublish is called for a local npm install too, but i think run before not after.. This is inconvenient if you have the coffeescript compiler as a local dependency

2013/1/7 Jacob Groundwater <ja...@nodefly.com>

José F. Romaniello

unread,
Jan 7, 2013, 2:40:06 PM1/7/13
to nod...@googlegroups.com
mmh just tried and it doesnt work for me.. but i have proof i am not crazy



2013/1/7 José F. Romaniello <jfroma...@gmail.com>

Dick Hardt

unread,
Jan 7, 2013, 2:42:38 PM1/7/13
to nod...@googlegroups.com
I have found it works great for a step in deployment and sharing of apps as it pulls in all the dependent modules, and then my install.js is run to setup the world.

Isaac Schlueter

unread,
Jan 7, 2013, 5:15:47 PM1/7/13
to nodejs
On Mon, Jan 7, 2013 at 9:51 AM, Jacob Groundwater <ja...@nodefly.com> wrote:
> I realize it is not a great solution, but the default node.js buildpack
> calls `npm install` and then just starts.

That's why prepublish scripts get run on `npm install` :)

Trust me, it's better this way.

Jacob Groundwater

unread,
Jan 7, 2013, 6:02:15 PM1/7/13
to nod...@googlegroups.com
That's why prepublish scripts get run on `npm install` :)

This does not appear to be the case for me; it would however be great if it did. 

Isaac Schlueter

unread,
Jan 7, 2013, 6:12:06 PM1/7/13
to nodejs
Jacob,

Indeed, that appears to be a bug. Fixing it now :)

Jacob Groundwater

unread,
Jan 7, 2013, 6:13:58 PM1/7/13
to nod...@googlegroups.com
Since you're fixing it, can it run after dev-dependencies are installed? =]

Tauren Mills

unread,
Jan 7, 2013, 6:24:23 PM1/7/13
to nod...@googlegroups.com

Isaac,

That's great, I use"//" in other places but always assumed they would get iterated in package.json dependencies. So we can safely do this?

{ "dependencies" :
  { "foo" : "1.0.0 - 2.9999.9999"
  , "//" : " comment about baz"
  , "baz" : ">1.0.2 <=2.3.4"
  , "//": " comment about boo"
  , "boo" : "2.0.1"
  }
}

jmar777

unread,
Jan 8, 2013, 9:15:54 AM1/8/13
to nod...@googlegroups.com, al...@kocharin.ru
Just found a relevant quote from Crockford on the subject:

I removed comments from JSON because I saw people were using them to hold parsing directives, a practice which would have destroyed interoperability. I know that the lack of comments makes some people sad, but it shouldn't. 

Suppose you are using JSON to keep configuration files, which you would like to annotate. Go ahead and insert all the comments you like. Then pipe it through JSMin before handing it to your JSON parser.

Source: https://plus.google.com/118095276221607585885/posts/RK8qyGVaGSr

I don't agree with his reasoning there at all, but there you have it.


On Saturday, January 5, 2013 1:22:06 PM UTC-5, Alex Kocharin wrote:
Hello, everybody.


TL;DR: I think that JSON is not a suitable config file format, and I want npm to be able to read configs stored in some other way by default. It might be just javascript, or yaml, I don't really care as long as it better for configuration files than json.


So, there is a dependency list in package.json, and it would be a good practice to have a comment for every line describing why we require that package, why we require that version of that package, what known problems we have and so on.

But there's a small issue. JSON format doesn't allow comments in any way.

Right now there are a couple of different ways around it of course:

1. Non-standard JSON entries like "@comment": "blablabla". Unfortunately, javascript editors doesn't highlight it as a comment, and it's just plain ugly. Also this violates strict javascript mode, so God knows what trouble it'll cause in the future.

nin jin

unread,
Jan 8, 2013, 12:11:11 PM1/8/13
to nod...@googlegroups.com, al...@kocharin.ru
I introduce new human and computer friendly and easy to extension format. It name is "Tree"
This is only experiment, not for production yet!

Features:
1. streaming (dislike XML and JSON than needed root node): parsed line-by-line
2. simple (XML, YAML, JSON is more complex): only struct- and value- nodes.
3. no escaping (no backslashes, no ampersand-encoding, no symbol repeating): may content binary data as set of lines
4. fast (because 2 and 3): js-implementation is 15x faster than yaml-js. but slower than native json implementation of course.
5. extendable (some changes of scheme in XML, JSON etc are not back compatible): 

As example, let us write package.tree:

=========================================
name    =jin
version =1.2.0

description =Harmoy node.js sugar full-async framework

author
name  =Nin Jin
email =nin...@ya.ru

keywords
=javascript
=framework
=modules

dependencies
fibers =*
npm =*

engine = node >= 0.4.1

readme
=This is long
=multiline text
=========================================

Then we need to extend scheme to allow comments for each dependencies:

=========================================
dependencies
fibers =*
//= for synchronization of async tasks
npm =*
//= for installation modules on demand
=========================================

Or:

=========================================
dependencies
fibers =*
reason =Synchronization of async tasks
npm =*
reason =Installation modules on demand
=========================================

And this do not break back-compatibility! In XML or JSON it will be broken :(

Let us create package.json from any of 3 given above package.tree

=========================================
require( 'jin' )( function( $ ){

var pack= $.jin.tree([ $.fs.readFileSync( 'package.tree' ) ]).parse()
var json=
{   name: pack.select(' name / ').toString()
,   version: pack.select(' version / ').toString()
,   description: pack.select(' description / ').toString()
,   homepage: pack.select(' homepage / ').toString()
,   author:
    {   name: pack.select(' author / name / ').toString()
    ,   email: pack.select(' author / email / ').toString()
    }
,   keywords: pack.select(' keywords / ').values()
,   bugs:
    {   email: pack.select(' bugs / email / ').toString()
    }
,   dependencies: new function( ){
        pack.select(' dependencies / ')
        .forEach( function( dep ){
            this[ dep.name ]= dep.toString()
        }.bind( this ) )
    }
,   engine: pack.select(' engine / ').toString()
,   readme: pack.select(' readme / ').toString()
}

$.fs.writeFileSync( 'package.json', JSON.stringify( json ) )

} )
=========================================


суббота, 5 января 2013 г., 22:22:06 UTC+4 пользователь Alex Kocharin написал:

Alex Kocharin

unread,
Jan 10, 2013, 2:09:50 AM1/10/13
to nod...@googlegroups.com
 
I did some googling before I asked this question here, so I saw this quote already.
 
npm doesn't do any minification, it simply does JSON.parse there, so I can't use this suggestion anyway. For my own config files I use javascript right now, and eval'ing it using vm.runInNewContext(), which I bet is much faster that jsmin.
 
So, his suggestion is completely useless.
 
--
// alex
 
 
08.01.2013, 18:16, "jmar777" <jma...@gmail.com>:
--

Dmitry Pashkevich

unread,
Oct 2, 2013, 4:02:08 AM10/2/13
to nod...@googlegroups.com, al...@kocharin.ru
Sorry for resurrecting an old thread, but have you considered using JSON5 (https://github.com/aseemk/json5)?
Author's rationale:

JSON isn't the friendliest to write. Keys need to be quoted, objects and arrays can't have trailing commas, and comments aren't allowed — even though none of these are the case with regular JavaScript today.

That was fine when JSON's goal was to be a great data format, but JSON's usage has expanded beyondmachines. JSON is now used for writing configsmanifests, even tests — all by humans.

There are other formats that are human-friendlier, like YAML, but changing from JSON to a completely different format is undesirable in many cases. JSON5’s aim is to remain close to JSON and JavaScript.


All this while:
JSON5 remains a strict subset of JavaScript, adds no new data types, and works with all existing JSON content.

Alex Kocharin

unread,
Oct 2, 2013, 5:20:29 PM10/2/13
to nod...@googlegroups.com

Sorry for resurrecting an old thread, but have you considered using JSON5 (https://github.com/aseemk/json5)?
Author's rationale:

Nothing to be sorry about, the issue still exists, and there is no widely accepted solution in the community.

Yes, json5 will solve this nicely if it was accepted as a possible package info format in npm packages. But it doesn't seem to be happening soon.

Personally, I solved it using yaml language (and package.yaml files in a project), and yapm module (https://github.com/rlidwka/yapm), it's a wrapper around npm to convert it to json format on the fly.

Luke Arduini

unread,
Oct 2, 2013, 5:25:36 PM10/2/13
to nod...@googlegroups.com
There are no plans to my knowledge to add "JSON5" support to npm. 


On Wednesday, October 2, 2013, Alex Kocharin wrote:

Sorry for resurrecting an old thread, but have you considered using JSON5 (https://github.com/aseemk/json5)?
Author's rationale:

Nothing to be sorry about, the issue still exists, and there is no widely accepted solution in the community.

Yes, json5 will solve this nicely if it was accepted as a possible package info format in npm packages. But it doesn't seem to be happening soon.

Personally, I solved it using yaml language (and package.yaml files in a project), and yapm module (https://github.com/rlidwka/yapm), it's a wrapper around npm to convert it to json format on the fly.


On Wednesday, October 2, 2013 12:02:08 PM UTC+4, Dmitry Pashkevich wrote:

JSON isn't the friendliest to write. Keys need to be quoted, objects and arrays can't have trailing commas, and comments aren't allowed — even though none of these are the case with regular JavaScript today.

That was fine when JSON's goal was to be a great data format, but JSON's usage has expanded beyondmachines. JSON is now used for writing configsmanifests, even tests — all by humans.

There are other formats that are human-friendlier, like YAML, but changing from JSON to a completely different format is undesirable in many cases. JSON5’s aim is to remain close to JSON and JavaScript.


All this while:
JSON5 remains a strict subset of JavaScript, adds no new data types, and works with all existing JSON content.


On Saturday, January 5, 2013 10:22:06 PM UTC+4, Alex Kocharin wrote:
Hello, everybody.


TL;DR: I think that JSON is not a suitable config file format, and I want npm to be able to read configs stored in some other way by default. It might be just javascript, or yaml, I don't really care as long as it better for configuration files than json.


So, there is a dependency list in package.json, and it would be a good practice to have a comment for every line describing why we require that package, why we require that version of that package, what known problems we have and so on.

But there's a small issue. JSON format doesn't allow comments in any way.

Right now there are a couple of different ways around it of course:

1. Non-standard JSON entries like "@comment": "blablabla". Unfortunately, javascript editors doesn't highlight it as a comment, and it's just plain ugly. Also this violates strict javascript mode, so God knows what trouble it'll cause in the future.
2. Keep a commented dependency list in a separate file. This violates DRY principle, so we could update one file and forget to update another. The same goes for /**package **/ hack I believe.
3. Use some kind of build system. Just for damn comments in one file?

Also, there's another wrong thing with JSON, it's too strict. You can't omit double quotes from keys, you can't leave a trailing comma, etc. JSON is human-readable, but it's just not damn human-writable.

Well... I went for 3rd option for a very long time. We used package.js file and a Makefile that compile js to json. Yes, that's three damn files instead of one. That's an example of our package.js file. https://gist.github.com/4462764 . But a number of supported packages grew, and compiling this slowly became a major pain in the ass. I recently got an issue when I updated package.js, but forgot to compile it, and debu

--
--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
 
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Gagle

unread,
Oct 2, 2013, 6:30:36 PM10/2/13
to nod...@googlegroups.com
El miércoles, 2 de octubre de 2013 23:20:29 UTC+2, Alex Kocharin escribió:
But it doesn't seem to be happening soon.


Never. Extremly slow compared with native json parser. I also agree with the OP. The json format is being misused massively by the people. JSON is a transport protocol, it was never intended to be used as a storage format. Double quotes, no comments, impossible to maintain by humans.

Alex Kocharin

unread,
Oct 2, 2013, 7:47:06 PM10/2/13
to nod...@googlegroups.com
 
03.10.2013, 02:30, "Gagle" <gagl...@gmail.com>:
El miércoles, 2 de octubre de 2013 23:20:29 UTC+2, Alex Kocharin escribió:
But it doesn't seem to be happening soon.


Never. Extremly slow compared with native json parser.
 
It doesn't make any sense to compare native json parser with userland parser. Of course last one will be slower. As for "extremely slow", I love it when people tell that something like that without actually running all the benchmarks.
 
Anyway, speed doesn't matter here. Usual size for package.json file is about 1kb, and that's how fast it is parsed:
 
JSON-native      6.9195075 µs   // v8 native stuff
JSON-js         57.7448887 µs   // Crockford's reference parser
JSON-js-state   76.5400478 µs   // Crockford's reference FSM
JS-YAML         62.3995833 µs   // js-yaml npm module
JSON5           61.0992242 µs   // current JSON5 module
 
100 packages will be parsed under 10 milliseconds, and I highly doubt any package you'll deal with will have more dependencies than that. But even if you will, node.js will spend far more time reading file from hdd than parsing it anyway. Why do you even think about performance here?
 
 
Out of the interest, I also tested 500kb json file because I think it shows real performance better.
 
JSON-native      5.7313025 ms    // v8 native stuff
JSON-js         32.7220672 ms    // Crockford's reference parser
JSON-js-state   28.9030499 ms    // Crockford's reference FSM
JS-YAML         22.2169591 ms    // js-yaml npm module
JSON5           44.5108157 ms    // current JSON5 module
Well, it is slow all right, and it's kinda expected from an experimental module. But it's not *extremely* slow, and it is just 2 times slower than it could be, hardly a big deal at all.
 
By the way, I'm very surprised about js-yaml outperforming json-js on a big data chunk, that's yaml parser that parses json works better than json parser. Good work, huh.
 
Test file used here: http://pastebin.com/yHewdcDg
 
 
I also agree with the OP. The json format is being misused massively by the people. JSON is a transport protocol, it was never intended to be used as a storage format. Double quotes, no comments, impossible to maintain by humans.

 

--

Dmitry Pashkevich

unread,
Oct 3, 2013, 4:11:37 AM10/3/13
to nod...@googlegroups.com
So was it a mistake to use JSON for npm package description as it's intended to be edited by humans too, not just programs?


--
--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
 
---
You received this message because you are subscribed to a topic in the Google Groups "nodejs" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nodejs/NmL7jdeuw0M/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nodejs+un...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.



--
Dmitry Pashkevich

Gagle

unread,
Oct 3, 2013, 5:56:02 AM10/3/13
to nod...@googlegroups.com, al...@kocharin.ru
I tested it. It is 6 times slower aprox.

El jueves, 3 de octubre de 2013 01:47:06 UTC+2, Alex Kocharin escribió:
 ...

Alex Kocharin

unread,
Oct 3, 2013, 7:03:43 AM10/3/13
to nod...@googlegroups.com

It's called "legacy", not a mistake. If npm would be implemented from scratch now, then yes it would be, but 4 years ago there wasn't any good replacement for it.

npm wasn't initially using strict package.json by the way, it was a javascript under the json extension (see https://github.com/isaacs/npm/issues/408), much more close to what we call json5 now. And I believe it was implemented this way exactly for the reasons mentioned above. But later it was decided that it's bad to use javascript with json extension, and we moved to strict json. I believe it was a wrong move there, but other people might argue with that.
Reply all
Reply to author
Forward
0 new messages