Draft PEP on RSON configuration file format
Patrick Maupin
Finding .ini configuration files too limiting, JSON and XML to hard to
manually edit, and YAML too complex to parse quickly, I have started
work on a new configuration file parser.
I call the new format RSON (for "Readable Serial Object Notation"),
and it is designed to be a superset of JSON.
I would love for it to be considered valuable enough to be a part of
the standard library, but even if that does not come to pass, I would
be very interested in feedback to help me polish the specification,
and then possibly help for implementation and testing.
The documentation is in rst PEP form, at:
http://rson.googlecode.com/svn/trunk/doc/draftpep.txt
Thanks and best regards,
Pat
Paul Rubin
> I have started work on a new configuration file parser....
> The documentation is in rst PEP form, at:...
Noooooooo.... not another... there are too many already. :-(
-1
Steven D'Aprano
> All:
>
> Finding .ini configuration files too limiting, JSON and XML to hard to
> manually edit, and YAML too complex to parse quickly, I have started
> work on a new configuration file parser.
>
> I call the new format RSON (for "Readable Serial Object Notation"), and
> it is designed to be a superset of JSON.
Wait a minute... if JSON is too hard to edit, and RSON is a *superset* of
JSON, that means by definition every JSON file is also a valid RSON file.
Since JSON is too hard to manually edit, so is RSON.
> I would love for it to be considered valuable enough to be a part of the
> standard library,
Come back when you actually have MANY users other than yourself using
this is real-world projects. Until then, it is too early to even consider
adding it the std library. Python comes with batteries included, but not
experimental batteries that aren't even built yet, let alone proven that
they work.
--
Steven
John Nagle
> All:
>
> Finding .ini configuration files too limiting, JSON and XML to hard to
> manually edit, and YAML too complex to parse quickly, I have started
> work on a new configuration file parser.
You're not supposed to edit those formats manually.
John Nagle
Chris Rebert
Huh? True for XML /perhaps/, but for the others I quite disagree.
Perhaps you could explain your reasoning?
Cheers,
Chris
--
http://blog.rebertia.com
Patrick Maupin
hard to edit, and RSON is a *superset* of
> JSON, that means by definition every JSON file is also a valid RSON file.
> Since JSON is too hard to manually edit, so is RSON.
Well, Python is essentially a superset of JSON, with string escape
handling being ever so slightly different, and using True instead of
true, False instead of false, and None instead of null. YMMV, but I
find it possible, even probable, to write Python that is far easier to
edit than JSON, and in fact, I have used Python for configuration
files that are only to be edited by programmers or other technical
types.
> Come back when you actually have MANY users other than yourself using
> this is real-world projects. Until then, it is too early to even consider
> adding it the std library. Python comes with batteries included, but not
> experimental batteries that aren't even built yet, let alone proven that
> they work.
I know that it is too early to be included in the library. But there
is a chicken-and-egg problem. PEP 1 says you should circulate a
design for discussion before writing code...
I fully expect this sort of negative feedback, fully expect that it
will be years (if ever) before the code makes it into the standard
library, and yet I offer the design for your perusal, just on the slim
chance that I might get a few constructive criticisms on the actual
design to go along with all the silly strawmen reductio ad absurdum
arguments by people who aren't even going to bother to read the thing.
Best regards,
Pat
Patrick Maupin
Opinions on that will vary, and as the saying goes, "everybody is
entitled to his own opinions, but not his own facts."
I submit as fact that there are many initialization files in each of
these formats, and I would be willing to place a wager that the
majority of the people on this list have used a text editor to modify
at least one file in this category at least once.
IOW, in my opinion (and it is only an opinion, not a fact), you are
actually helping to make the case that a better format is needed.
Regards,
Pat
Robert Kern
> On Feb 28, 9:18 pm, Steven D'Aprano
>> Come back when you actually have MANY users other than yourself using
>> this is real-world projects. Until then, it is too early to even consider
>> adding it the std library. Python comes with batteries included, but not
>> experimental batteries that aren't even built yet, let alone proven that
>> they work.
>
> I know that it is too early to be included in the library. But there
> is a chicken-and-egg problem. PEP 1 says you should circulate a
> design for discussion before writing code...
Ignore it. That comment really doesn't apply to this case. That's for things
that only make sense in the language or standard library, like context managers.
For libraries like this, Steven's summary is correct. It needs to have a useful
life as a third party package for a few years before you should propose it for
inclusion into the standard library. By all means, distribute a design document
for comment before you implement things; it's a reasonable practice. But don't
bother with a PEP yet.
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
Patrick Maupin
> Ignore it. That comment really doesn't apply to this case. That's for things
> that only make sense in the language or standard library, like context managers.
> For libraries like this, Steven's summary is correct. It needs to have a useful
> life as a third party package for a few years before you should propose it for
> inclusion into the standard library. By all means, distribute a design document
> for comment before you implement things; it's a reasonable practice. But don't
> bother with a PEP yet.
So, I went and re-read PEP 1, and in the fine print it directs me to
PEP 2. Mea culpa -- I though I understood the PEP.
Nonetheless, the fact that I created what I call a "PEP draft" seems
to have had more read into it than I meant. I will bear this in mind
when I start future projects. I have a couple of projects, and
contribute to another, which in no way should wind up in the standard
library. But I find the lack of a good, up-to-date, configuration
reader in the library to be an issue, which is why I had the temerity
to try to start a discussion on what a good standard configuration
file reader should be.
Yes, I would like to create something compelling enough to go into the
standard library. No, I don't expect it to wind up there for a very
long time, if ever. BUT, at the same time, following the PEP format
is very valuable for my project. It forces me to think more deeply
about the problem I am solving and it forces me to write down some
documentation. Really, it's no more burdensome than any other similar
useful template.
Or, to put it another way, it is my goal (which I may or may not be
smart enough to reach) to write a module that anybody would want to
use; that is good enough to put into the standard library. Whether
this happens or not, one useful tool to help the module down that path
(and produce a better module for me and other users even if it never
becomes part of the standard library) is to think about what it takes
to get into the standard library. Part of this involves writing the
PEP. Writing a PEP is a good exercise; the format itself helps me to
focus on important issues.
So I was writing the contents of the PEP anyway, just for me. The
real question, then, is when to release the PEP (vs. the code). I've
already got some coding done, but it is not yet good enough to share,
and if there is anybody else out there grappling with the same issues
at this time, I would prefer not to go into production with it until
they have had a chance to look over the design and offer insights and
criticisms.
Best regards,
Pat
Robert Kern
Certainly. The PEP format is a useful one. I've used it myself for some numpy
design documents. But can you see why people might get confused about your
intentions when you call it a draft PEP and post it to python-dev? If you stop
calling it a PEP and stop talking about putting it in the standard library,
people will stop being distracted by those issues.
Paul Rubin
> it is my goal (which I may or may not be smart enough to reach) to
> write a module that anybody would want to use;
But you are working on a solution in search of a problem. The really
smart thing to do would be pick something more useful to work on. We
don't need another configuration language. I can't even say "yet
another" because there's already a "yet another" called yaml.
Daniel Fetchinson
And in case you are new here let me assure you that Paul is saying
this with his full intention of being helpful to you. I also would
think that working on such a project might be fun and educational for
you but completely useless if you have users other than yourself in
mind. Again, I'm trying to be helpful here, so you can focus on a
project that is both fun/educational for you and also potentially
useful for others. This RSON business is not one of them.
Cheers,
Daniel
--
Psss, psss, put it down! - http://www.cafepress.com/putitdown
Patrick Maupin
> But you are working on a solution in search of a problem. The really
> smart thing to do would be pick something more useful to work on. We
> don't need another configuration language. I can't even say "yet
> another" because there's already a "yet another" called yaml.
The only "in search of" here is that, instead of working on a point
solution for my particular problem, I am "in search of" a solution
that is a bit more elegant and general, and that might help solve
other people's problems too. If you are not one of those that has
this sort of problem at this point in time, then feel free to kill-
file this thread.
Thanks,
Pat
Patrick Maupin
> Certainly. The PEP format is a useful one. I've used it myself for some numpy
> design documents. But can you see why people might get confused about your
> intentions when you call it a draft PEP and post it to python-dev? If you stop
> calling it a PEP and stop talking about putting it in the standard library,
> people will stop being distracted by those issues.
As I mentioned, I didn't see the fine print in PEP 1 about PEP 2 being
the document for library modules. As I mentioned, mea culpa. It is
painfully obvious that some don't like the way I have gone about
describing the project. They obviously view me announcing this as
premature, or presumptuous, or something, and they have some sort of
visceral reaction to that.
However, I do not believe that any people (other than me) were really
confused in the process. I made my intentions clear, and some people
reacted badly to that because I didn't follow the process (for which I
apologize again). But calling it a draft PEP is a distraction
(because of the visceral reaction), but is not really what I would
call confusing. My intention actually is to try to build something
that is worthy of the standard library, and to eventually try to get
it accepted, because I perceive a hole there, with a lot of point
solutions being done to solve a common problem, and I believe the
pushback is coming from people who fully understood that intention
from my posting.
I will try to say "hey -- here's a hole in the library and a proposal
for how to fix it" more diplomatically and in the correct forum in the
future, but it would be disingenuous for me to disown my goal of
getting a better configparser into the standard library.
Regards,
Pat
Patrick Maupin
wrote:
> > But you are working on a solution in search of a problem. The really
> > smart thing to do would be pick something more useful to work on. We
> > don't need another configuration language. I can't even say "yet
> > another" because there's already a "yet another" called yaml.
>
> And in case you are new here let me assure you that Paul is saying
> this with his full intention of being helpful to you. I also would
> think that working on such a project might be fun and educational for
> you but completely useless if you have users other than yourself in
> mind. Again, I'm trying to be helpful here, so you can focus on a
> project that is both fun/educational for you and also potentially
> useful for others. This RSON business is not one of them.
OK, but I am a bit unclear on what you and/or Paul are claiming. It
could be one of a number of things. For example:
- There is a preexisting file format suitable for my needs, so I
should not invent another one.
- If I invent a file format suitable for my needs, it couldn't
possibly be general enough for anybody else.
- Even if it was general enough for somebody else, there would only be
two of them.
I've been known to waste time (or be accused of wasting time) on
various endeavors, but I like to know exactly *why* it is perceived to
be a waste.
Regards,
Pat
Paul Rubin
> - There is a preexisting file format suitable for my needs, so I
> should not invent another one.
There are in fact quite a few--json, yaml, .ini, xml, Python literals
(http://code.activestate.com/recipes/364469-safe-eval/), s-expressions,
actual Python code that the application can import, and so forth.
The problem isn't that you're trying to invent a useless file format per
se, but rather that in trying to get other people to learn it and use
it, you're also trying to appropriate a chunk of the user community's
scarce and precious brain cells without good justification. Newbie
designers are often lured into that and they're unfortunately (i.e. to
the community's detrimtent) often more successful than they really
should be.
Your one complaint about yaml is that it's slow to parse. Why do you
care about the parsing speed of a config file? If the existing
implementation is too slow, why not write a faster one instead of
designing yayaml? Even yaml is excessive in my view. "Yet another" was
an ok plan when Steve Johnson started the trope by writing Yacc 30 years
ago. These days, don't do yet another without very convincing reasons
for rejecting what is already there.
Patrick Maupin
> There are in fact quite a few--json, yaml, .ini, xml, Python literals
> (http://code.activestate.com/recipes/364469-safe-eval/), s-expressions,
> actual Python code that the application can import, and so forth.
Yes, I know about those.
> The problem isn't that you're trying to invent a useless file format per
> se, but rather that in trying to get other people to learn it and use
> it, you're also trying to appropriate a chunk of the user community's
> scarce and precious brain cells without good justification. Newbie
> designers are often lured into that and they're unfortunately (i.e. to
> the community's detrimtent) often more successful than they really
> should be.
Can't disagree with the statement about newbie designers. On the flip
side, sometimes things wind up being defacto standards, simply because
nobody could be bothered to write a better one, and they have lots of
little niggling warts that also collectively take up a lot of time and
attention. There is obviously a good balance somewhere.
> Your one complaint about yaml is that it's slow to parse. Why do you
> care about the parsing speed of a config file? If the existing
> implementation is too slow, why not write a faster one instead of
> designing yayaml? Even yaml is excessive in my view. "Yet another" was
> an ok plan when Steve Johnson started the trope by writing Yacc 30 years
> ago. These days, don't do yet another without very convincing reasons
> for rejecting what is already there.
One of my complaints. If you had read the document you would have
seen others. I actually have several complaints about YAML, but I
tried to write a cogent summary.
Regards,
Pat
Paul Rubin
> One of my complaints. If you had read the document you would have
> seen others. I actually have several complaints about YAML, but I
> tried to write a cogent summary.
Yaml sucks, but seems to have gotten some traction regardless.
Therefore the Python principle of "there should be one and only one
obvious way to do it" says: don't try to replace the existing thing if
your new thing is only slightly better. Just deal with the existing
thing's imperfections or make improvements to it. If you can make a
really powerful case that your new thing is 1000x better than the old
thing, that's different, but I don't think we're seeing that here.
Also, XML is used for pretty much everything in the Java world. It
sucks too, but it is highly standardized, it observably gets the job
done, there are tons of structure editors for it, etc. Frankly
I'd rather have stayed with it than deal with Yaml.
There are too many of these damn formats. We should ban all but one of
them (I don't much care which one). And making even more of them is not
the answer.
Patrick Maupin
> Yaml sucks, but seems to have gotten some traction regardless.
Yes, that's actually one of the reasons I want to do this. I've heard
that some of the YAML people want that in the standard library, and
IMHO that would be a huge mistake.
> Therefore the Python principle of "there should be one and only one
> obvious way to do it" says: don't try to replace the existing thing if
> your new thing is only slightly better.
But for my use-case, YAML is irretrievably broken. Sure, it looks
reasonably nice, but it increases regression runtime unacceptably.
Also, despite their current protestations that Yaml AIN'T markup
language, the syntax still belies its markup roots. I'm looking for a
configuration language, not a markup language.
> Just deal with the existing thing's imperfections or make improvements to it.
Well, I've looked at the YAML parser and I can assure you that I will
not be contributing to that project.
> If you can make a really powerful case that your new thing is 1000x better than the old thing, that's different, but I don't think we're seeing that here.
Well, there's no way I'm hitting three orders of magnitude, but I
don't think the bar is really that high. We will see.
> Also, XML is used for pretty much everything in the Java world. It
> sucks too, but it is highly standardized, it observably gets the job
> done, there are tons of structure editors for it, etc. Frankly
> I'd rather have stayed with it than deal with Yaml.
XML can certainly be made readable by humans. It's hard to make it
writeable by (average) humans just using a regular text editor, and
even though there are tons of structure editors, requiring non-
technical people to find one of those and start using it is a non-
starter in many cases. But I can't strongly disagree with the opinion
that YAML doesn't offer all that much over it.
> There are too many of these damn formats. We should ban all but one of
> them (I don't much care which one). And making even more of them is not
> the answer.
Well, I think that XML and YAML both overreach by trying to be all
things to all applications. So I'm aiming a lot lower than that. I
just want to make really good configuration files that are easy to
modify in a text editor.
None of the existing formats are suitable. I may be the only one who
feels that way. OTOH, if we never invented new things, then I
wouldn't be enjoying restructured text, and I'd be missing out on a
lot there...
Regards,
Pat
Paul Rubin
> But for my use-case, YAML is irretrievably broken. Sure, it looks
> reasonably nice, but it increases regression runtime unacceptably.
How big are the files that you want to parse with it? Sheesh.
> Well, I've looked at the YAML parser and I can assure you that I will
> not be contributing to that project.
So write a new one that parses the same syntax, but cleaner and faster.
> XML can certainly be made readable by humans. It's hard to make it
> writeable by (average) humans just using a regular text editor,
I do it all the time; it's a bit dreary but not difficult. And there is
absolutely no way to get anything done in this field anymore without
dealing with XML from time to time. So given that we all have some
experience using it, it's sensible to stick with it.
> None of the existing formats are suitable. I may be the only one who
> feels that way. OTOH, if we never invented new things, then I
> wouldn't be enjoying restructured text, and I'd be missing out on a
> lot there...
ReST is another abomination that should never have gotten off the
ground. It is one of the reasons I react so negatively to your
config format proposal. It just sounds like more of the same.
Philip Semanchuk
I dunno, times change, needs change. We must invent new tools, be
those computer languages or data formats. Otherwise we'd still be
programming in COBOL and writing fixed-length records to 12 inch
floppies.*
If Mr. Maupin was a giant corporation trying to shove a proprietary
format down our collective throats, I might object to RSON. But he's
not. He appears willing for it live or die on its merits, so I say
good luck to him. I don't want or need it, but someone else might.
Cheers
Philip
* You had floppies? Bleddy luxury! We wrote our data on wood pulp we'd
chewed ourselves and dried into paper, using drops of our own blood to
represent 1s and 0s.
Patrick Maupin
> Patrick Maupin <pmau...@gmail.com> writes:
> > But for my use-case, YAML is irretrievably broken. Sure, it looks
> > reasonably nice, but it increases regression runtime unacceptably.
>
> How big are the files that you want to parse with it? Sheesh.
Tiny, but over and over. The rst2pdf testsuite can generate
approximately 160 PDFs, totalling around 2.5 MB, in around 22 seconds
on one of my machines. But if I replace the JSON parser with a YAML
parser, that goes up to 55 seconds. Wait, maybe it's because JSON is
optimized in C! Nope, using JSON but disabling the C scanner only
takes it to 22.3 seconds...
>
> > Well, I've looked at the YAML parser and I can assure you that I will
> > not be contributing to that project.
>
> So write a new one that parses the same syntax, but cleaner and faster.
But there are already several parsers for YAML, and none of them
agree! The syntax definition is a mess. The thing's been in
development for 10 years now, and there is no one true way to do it.
Seriously, YAML overreaches for what I want.
> > XML can certainly be made readable by humans. It's hard to make it
> > writeable by (average) humans just using a regular text editor,
>
> I do it all the time; it's a bit dreary but not difficult. And there is
> absolutely no way to get anything done in this field anymore without
> dealing with XML from time to time. So given that we all have some
> experience using it, it's sensible to stick with it.
But people "in this field" are not really my target audience. Well, I
mean people in this field are the target audience for the library, but
not for the writing of the actual text files.
> ReST is another abomination that should never have gotten off the
> ground. It is one of the reasons I react so negatively to your
> config format proposal. It just sounds like more of the same.
Well, that clarifies a lot. I guess we'll just have to agree to
disagree :-)
Regards,
Pat
Emile van Sebille
> * You had floppies? Bleddy luxury! We wrote our data on wood pulp we'd
> chewed ourselves and dried into paper, using drops of our own blood to
> represent 1s and 0s.
You had left-over blood?!!
Emile :)
Erik Max Francis
Agreed. Even YAML's acronym indicates that it is already a bridge too
far; we don't need more.
--
Erik Max Francis && m...@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 18 N 121 57 W && AIM/Y!M/Skype erikmaxfrancis
It's better to be quotable than to be honest.
-- Tom Stoppard
Erik Max Francis
Psst. That you're allowed to present the idea that you think is good
doesn't mean that other people aren't allowed to respond and point out
that in their opinion it's not such a good idea. You don't own this or
any other thread.
Erik Max Francis
> On Feb 28, 9:18 pm, Steven D'Aprano > Wait a minute... if JSON is too
> hard to edit, and RSON is a *superset* of
>> JSON, that means by definition every JSON file is also a valid RSON file.
>> Since JSON is too hard to manually edit, so is RSON.
>
> Well, Python is essentially a superset of JSON, with string escape
> handling being ever so slightly different, and using True instead of
> true, False instead of false, and None instead of null. YMMV, but I
> find it possible, even probable, to write Python that is far easier to
> edit than JSON, and in fact, I have used Python for configuration
> files that are only to be edited by programmers or other technical
> types.
This not only seriously stretching the meaning of the term "superset"
(as Python is most definitely not even remotely a superset of JSON), but
still doesn't address the question. Is RSON and _actual_ superset of
JSON, or are you just misusing the term there, as well? If it is, then
your rationale for not using JSON makes no sense if you're making a new
format that's merely a superset of it. Obviously JSON can't be that
unreadable if you're _extending_ it to make your own "more readable"
format. If JSON is unreadable, so must be RSON.
Patrick Maupin
> Psst. That you're allowed to present the idea that you think is good
> doesn't mean that other people aren't allowed to respond and point out
> that in their opinion it's not such a good idea. You don't own this or
> any other thread.
Absolutely, but I still do (and will always) express a clear
preference for opinions that have at least a modicum of reasoning
behind them.
Regards,
Pat
Patrick Maupin
> Patrick Maupin wrote:
> This not only seriously stretching the meaning of the term "superset"
> (as Python is most definitely not even remotely a superset of JSON), but
Well, you are entitled to that opinion, but seriously, if I take valid
JSON, replace unquoted true with True, unquoted false with False,
replace unquoted null with None, and take the quoted strings and
replace occurrences of \uXXXX with the appropriate unicode, then I do,
in fact, have valid Python. But don't take my word for it -- try it
out!
But if you really want to be pedantic about it, JavaScript (rather
than Python) is, in fact a superset of JSON, and, despite the
disparagement JavaScript receives, in my opinion, it is possible to
write much better looking JavaScript than JSON for many tasks.
YAML, also, is a superset of JSON, and IMO, it is possible to write
much better looking YAML than JSON.
> still doesn't address the question. Is RSON and _actual_ superset of
> JSON, or are you just misusing the term there, as well?
Yes, the RSON definition, in fact, a superset of JSON, just like the
YAML definition. But RSON is a much smaller grammar than YAML.
If it is, then
> your rationale for not using JSON makes no sense if you're making a new
> format that's merely a superset of it. Obviously JSON can't be that
> unreadable if you're _extending_ it to make your own "more readable"
> format. If JSON is unreadable, so must be RSON.
Well, we'll have to agree to disagree here. Bearing in mind that the
definition of "unreadable" depends on the target application and user,
obviously, it will be *possible* to write unreadable RSON, just as it
is *possible* to write unreadable JavaScript or Python or YAML, but it
will be *possible* to write better looking RSON than is possible to
achieve with JSON, just as it is *possible* to write better looking
JavaScript or YAML or Python than it is *possible* to achieve with
pure JSON.
Best regards,
Pat
Kirill Simonov
> Daniel Fetchinson wrote:
>>>> it is my goal (which I may or may not be smart enough to reach) to
>>>> write a module that anybody would want to use;
>>> But you are working on a solution in search of a problem. The really
>>> smart thing to do would be pick something more useful to work on. We
>>> don't need another configuration language. I can't even say "yet
>>> another" because there's already a "yet another" called yaml.
>>
>> And in case you are new here let me assure you that Paul is saying
>> this with his full intention of being helpful to you. I also would
>> think that working on such a project might be fun and educational for
>> you but completely useless if you have users other than yourself in
>> mind. Again, I'm trying to be helpful here, so you can focus on a
>> project that is both fun/educational for you and also potentially
>> useful for others. This RSON business is not one of them.
>
> Agreed. Even YAML's acronym indicates that it is already a bridge too
> far; we don't need more.
>
Note that YA in the acronym doesn't mean Yet Another, YAML = YAML Ain't
Markup Language.
Thanks,
Kirill
Kirill Simonov
> All:
>
> Finding .ini configuration files too limiting, JSON and XML to hard to
> manually edit, and YAML too complex to parse quickly, I have started
> work on a new configuration file parser.
I'd like to note that with the optional libyaml bindings, the PyYAML
parser is pretty fast.
> I call the new format RSON (for "Readable Serial Object Notation"),
> and it is designed to be a superset of JSON.
>
> I would love for it to be considered valuable enough to be a part of
> the standard library, but even if that does not come to pass, I would
> be very interested in feedback to help me polish the specification,
> and then possibly help for implementation and testing.
>
> The documentation is in rst PEP form, at:
>
> http://rson.googlecode.com/svn/trunk/doc/draftpep.txt
=== cut ===
Because YAML does allow for highly readable configuration files, it
is tempting to overlook its other flaws for the task. But a fully
(or almost) compliant parser has to understand the whole YAML
specification, and this is apparently expensive. Running the rst2pdf
testsuite, without sphinx or most of the other optional packages, in
"fast" mode (preloading all the modules, and then forking for every
test) generates 161 smallish PDF files, totaling around 2.5 MB. On
one test system this process takes 22 seconds. Disabling the _json C
scanner and reading the configuration files using the json pure Python
implementation adds about 0.3 seconds to the 22 seconds. But using
pyyaml v. 3.09 instead of json adds 33 seconds to the 22 second process!
It might seem that this is an edge case, but it makes it unacceptable to
use YAML for this sort of testing, and taking 200 ms to read in 1000
lines of simple JSON will be unacceptable in many other application
domains as well.
=== cut ===
I'd question your testing methodology. From your description, it looks
like the _json speedup never was enabled. Also PyYAML provides optional
bindings to libyaml, which makes parsing and emitting yaml much faster.
In my tests, it parses a 10Mb file in 3 sec.
=== cut ===
RSON semantics are based on JSON. Like JSON, an RSON document represents
either a single scalar object, or a DAG (Directed Acyclic Graph), which
may contain only a few simple data types.
=== cut ===
JSON doesn't represent a DAG, at least, not an arbitrary DAG since each
node in the document has no more than one parent. It would be more
accurate to say that that it represents a tree-like structure.
=== cut ===
The YAML syntax for supporting back-references was considered and deemed
unsatisfactory. A human user who wants to put identical information in a
"ship to" and "bill to" address is much more likely to use cut and paste
than he is to understand and use backreferences, so the additional overhead
of supporting more complex document structures is unwarranted.
The concept of a "merge" in YAML, where two sub-trees of data can be
merged together (similar to a recursive Python dictionary update)
is quite useful, though, and will be copied. This does not alter the
outcome that parsing a RSON file will result in a DAG, but does give
more flexibility in the syntax that can be used to achieve a particular
output DAG.
=== cut ===
This paragraph assumes the reader is familiar with intricate details of
the YAML grammar and semantics. I bet most of your audience are
completely lost here.
=== cut ===
Enhanced example::
key1/key2a
key3a = Some random string
key3b = 42
key1/key2a
key3c
1
2
{}
key4a = anything
key4b = something else
[]
a
b
c
3
4
key1/key2b = [1, 2, 3, 4]
key5 = ""
This is a multi-line string. It is
dedented to the farthest left
column that is indented from
the line containing "".
key6 = [""]
This is an array of strings, one per line.
Each string is dedented appropriately.
=== cut ===
Frankly, this is an example that only a mother could love. I'd suggest
you to add some real-world examples, make sure they look nice and put
them to the introductory part of the document. Examples is how the
format will be evaluated by the readers, and yours don't stand a chance.
Seriously, the only reason YAML enjoys its moderate popularity despite
its overcomplicated grammar, chronic lack of manpower and deficient
implementations is because it's so cute.
Disclaimer: I'm the author of PyYAML and libyaml.
Thanks,
Kirill
Patrick Maupin
Thank you for your constructive criticism. This is the gem that made
it worthwhile to post my document. I think all of your points are
spot-on, and I will be fixing the documentation.
I can well believe that the C implementation of YAML is much faster
than the Python one, but I am aiming for something that will be
reasonably quick in pure Python. I will double-check the JSON C test
results, but something I probably did not make clear is that the 22
seconds is not spent parsing -- that is for the entire test, which
involves reading restructured text and generating some 160 separate
PDF files.
Best regards,
Pat
Patrick Maupin
BTW, congratulations on slogging through the YAML grammar to generate
such a good working C library!
That must have been a tremendous effort.
Regards,
Pat
Kirill Simonov
> Kirill:
>
> Thank you for your constructive criticism. This is the gem that made
> it worthwhile to post my document. I think all of your points are
> spot-on, and I will be fixing the documentation.
You are welcome. Despite what others have been saying, I don't think
this area is closed to innovations.
> I can well believe that the C implementation of YAML is much faster
> than the Python one, but I am aiming for something that will be
> reasonably quick in pure Python. I will double-check the JSON C test
> results, but something I probably did not make clear is that the 22
> seconds is not spent parsing -- that is for the entire test, which
> involves reading restructured text and generating some 160 separate
> PDF files.
Yes, this makes more sense. It's quite possible that the pure-Python
PyYAML parser is much slower than the pure-Python JSON parser.
At the same time, semantically meaningful whitespaces will likely hinder
the pure-Python performance. To make it fast, you'll need to convert
the inner loops of the parser to regexps, and it is hard to support
variable-length indentation with static regular expressions.
Thanks,
Kirill
Kirill Simonov
The trick was to completely ignore the grammar described in the
specification. In fact, the syntax of YAML is pretty close the Python
syntax and, with some effort, the Python scanner and parser could be
adapted to parsing YAML. Once I realized it, I got a working parser in
a week or so.
Thanks,
Kirill
Terry Reedy
> On Mar 1, 5:57 pm, Erik Max Francis<m...@alcyone.com> wrote:
>> Patrick Maupin wrote:
>> This not only seriously stretching the meaning of the term "superset"
>> (as Python is most definitely not even remotely a superset of JSON), but
>
> Well, you are entitled to that opinion, but seriously, if I take valid
> JSON, replace unquoted true with True, unquoted false with False,
> replace unquoted null with None, and take the quoted strings and
> replace occurrences of \uXXXX with the appropriate unicode, then I do,
> in fact, have valid Python. But don't take my word for it -- try it
> out!
To me this is so strained that I do not see why why you are arguing the
point. So what? The resulting Python 'program' will be equivalent, I
believe, to 'pass'. Ie, construct objects and then discard them with no
computation or output. I suggest dropping this red-herring distraction.
> But if you really want to be pedantic about it, JavaScript (rather
> than Python) is, in fact a superset of JSON, and, despite the
> disparagement JavaScript receives, in my opinion, it is possible to
> write much better looking JavaScript than JSON for many tasks.
>
> YAML, also, is a superset of JSON, and IMO, it is possible to write
> much better looking YAML than JSON.
I agree that adding a bit of syntax to something can sometimes make it
easier to write readable text. This is hardly a new idea and should not
be controversial. That is why people developed 'macro assembley' as a
superset of 'assembly' languages and why, for instance, Python added the
'with' statement.
I read your proposal. I have not needed config files and have never
written json or yaml and so cannot really evaluate your proposal for
something 'in between'. It does seem plausible that it could be useful.
While using the PEP format is great, calling your currently vaperware
module proposal a 'standards track' PEP is somewhat off-putting and
confusing. If Guido rejected it, would you simply drop the idea? If not,
if you would continue it as a third-party module that would eventually
be released and announced on PyPI, I seriously suggest renaming it to
what it is.
Terry Jan Reedy
Arnaud Delobelle
> Patrick Maupin wrote:
>> On Feb 28, 9:18 pm, Steven D'Aprano > Wait a minute... if JSON is too
>> hard to edit, and RSON is a *superset* of
>>> JSON, that means by definition every JSON file is also a valid RSON file.
>>> Since JSON is too hard to manually edit, so is RSON.
>>
>> Well, Python is essentially a superset of JSON, with string escape
>> handling being ever so slightly different, and using True instead of
>> true, False instead of false, and None instead of null. YMMV, but I
>> find it possible, even probable, to write Python that is far easier to
>> edit than JSON, and in fact, I have used Python for configuration
>> files that are only to be edited by programmers or other technical
>> types.
>
> This not only seriously stretching the meaning of the term "superset"
> (as Python is most definitely not even remotely a superset of JSON),
> but still doesn't address the question. Is RSON and _actual_ superset
> of JSON, or are you just misusing the term there, as well? If it is,
> then your rationale for not using JSON makes no sense if you're making
> a new format that's merely a superset of it. Obviously JSON can't be
> that unreadable if you're _extending_ it to make your own "more
> readable" format. If JSON is unreadable, so must be RSON.
Your argument is utterly speculative as you are making clear you haven't
read the OP's proposal.
--
Arnaud
Daniel Fetchinson
>> > smart thing to do would be pick something more useful to work on. We
>> > don't need another configuration language. I can't even say "yet
>> > another" because there's already a "yet another" called yaml.
>>
>> And in case you are new here let me assure you that Paul is saying
>> this with his full intention of being helpful to you. I also would
>> think that working on such a project might be fun and educational for
>> you but completely useless if you have users other than yourself in
>> mind. Again, I'm trying to be helpful here, so you can focus on a
>> project that is both fun/educational for you and also potentially
>> useful for others. This RSON business is not one of them.
>
> OK, but I am a bit unclear on what you and/or Paul are claiming. It
> could be one of a number of things. For example:
>
> - There is a preexisting file format suitable for my needs, so I
> should not invent another one.
I suspect this to be true, if we mean the same thing by "configuration
file format". Especially if RSON will be a superset of JSON.
> - If I invent a file format suitable for my needs, it couldn't
> possibly be general enough for anybody else.
Quite possibly, the reason is that the already existing file formats
have an ecosystem around them that make them attractive. Your file
format will have to cope with this barrier to attract new users which
I think will be very difficult, given the wide range of already
existing formats, covering just about any use case.
> - Even if it was general enough for somebody else, there would only be
> two of them.
See above.
> I've been known to waste time (or be accused of wasting time) on
> various endeavors, but I like to know exactly *why* it is perceived to
> be a waste.
Don't get me wrong, I also waste lot of time on hobby/fun/educational
projects ("waste" in this case is only meant as "useless for others",
not "useless for me") because it's, well, hobby and fun and
educational :) It's just good to know if a given project is in this
category or outside.
Robert Kern
> On 3/1/2010 7:56 PM, Patrick Maupin wrote:
>>> Patrick Maupin wrote:
>>> This not only seriously stretching the meaning of the term "superset"
>>> (as Python is most definitely not even remotely a superset of JSON), but
>>
>> Well, you are entitled to that opinion, but seriously, if I take valid
>> JSON, replace unquoted true with True, unquoted false with False,
>> replace unquoted null with None, and take the quoted strings and
>> replace occurrences of \uXXXX with the appropriate unicode, then I do,
>> in fact, have valid Python. But don't take my word for it -- try it
>> out!
>
> point. So what? The resulting Python 'program' will be equivalent, I
> believe, to 'pass'. Ie, construct objects and then discard them with no
> computation or output.
Not if you eval() rather than exec(). It's reasonably well-accepted that JSON is
very close to being a subset of Python's expression syntax with just a few
modifications.
--
Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco
Terry Reedy
> On 2010-03-01 22:55 PM, Terry Reedy wrote:
>> On 3/1/2010 7:56 PM, Patrick Maupin wrote:
>>>> Patrick Maupin wrote:
>>>> This not only seriously stretching the meaning of the term "superset"
>>>> (as Python is most definitely not even remotely a superset of JSON),
>>>> but
>>>
>>> Well, you are entitled to that opinion, but seriously, if I take valid
>>> JSON, replace unquoted true with True, unquoted false with False,
>>> replace unquoted null with None, and take the quoted strings and
>>> replace occurrences of \uXXXX with the appropriate unicode, then I do,
>>> in fact, have valid Python. But don't take my word for it -- try it
>>> out!
>>
>> point. So what? The resulting Python 'program' will be equivalent, I
>> believe, to 'pass'. Ie, construct objects and then discard them with no
>> computation or output.
>
> Not if you eval() rather than exec().
>>> eval(1)
creates and objects and discards it, with a net result of 'pass'.
What do you think I am missing.
It's reasonably well-accepted that
> JSON is very close to being a subset of Python's expression syntax with
> just a few modifications.
It is specifically JavaScript Object Notation, which is very similar to
a subset of Python's object notation (number and string literals and
list and dict displays (but not set displays), and three named
constants). Without operators, it barely qualifies, to me, even as
'expression syntax'.
To me, comparing object notation with programming language is not
helpful to the OP's purpose. His main claim is that JSON can be usefully
extended but that YAML is too much, so that perhaps he, with help, can
find a 'sweet spot' in between.
Terry Jan Reedy
Robert Kern
> On 3/2/2010 11:34 AM, Robert Kern wrote:
>> On 2010-03-01 22:55 PM, Terry Reedy wrote:
>>> On 3/1/2010 7:56 PM, Patrick Maupin wrote:
>>>>> Patrick Maupin wrote:
>>>>> This not only seriously stretching the meaning of the term "superset"
>>>>> (as Python is most definitely not even remotely a superset of JSON),
>>>>> but
>>>>
>>>> Well, you are entitled to that opinion, but seriously, if I take valid
>>>> JSON, replace unquoted true with True, unquoted false with False,
>>>> replace unquoted null with None, and take the quoted strings and
>>>> replace occurrences of \uXXXX with the appropriate unicode, then I do,
>>>> in fact, have valid Python. But don't take my word for it -- try it
>>>> out!
>>>
>>> point. So what? The resulting Python 'program' will be equivalent, I
>>> believe, to 'pass'. Ie, construct objects and then discard them with no
>>> computation or output.
>>
>> Not if you eval() rather than exec().
>
> >>> eval(1)
>
> creates and objects and discards it, with a net result of 'pass'.
> What do you think I am missing.
x = eval('1')
> It's reasonably well-accepted that
>> JSON is very close to being a subset of Python's expression syntax with
>> just a few modifications.
>
> It is specifically JavaScript Object Notation, which is very similar to
> a subset of Python's object notation (number and string literals and
> list and dict displays (but not set displays), and three named
> constants). Without operators, it barely qualifies, to me, even as
> 'expression syntax'.
Literal expression syntax, then.
Patrick Maupin
> To me, comparing object notation with programming language is not
> helpful to the OP's purpose.
Yes, I agree, it was a distraction. I fell into the trap of
responding to the ludicrous claim that "if X is a superset of Y, then
X cannot possibly look better than Y" (a claim made by multiple people
all thinking it was clever) by showing that Y has other supersets that
do in fact look better than Y. In doing this, I made the mistake of
choosing a superset of an analogue to Y, rather than to Y itself.
When called out on it, I showed that, in fact, the actual X that IS a
superset of Y can be used in a way that looks better. However, you
are right that JSON is such a small subset of JS that it's really
pretty ridiculous to even compare them, but that still makes the point
that the original argument I was trying to refute is completely
specious. In retrospect, though, I should have chosen a better way to
make that point, because I let myself get caught up in making and then
defending a flippant statement that I don't really care about one way
or the other.
> His main claim is that JSON can be usefully
> extended but that YAML is too much, so that perhaps he, with help, can
> find a 'sweet spot' in between.
An excellent summary of my position.
Thanks,
Pat
Steven D'Aprano
> On Mar 2, 11:59 am, Terry Reedy <tjre...@udel.edu> wrote:
>
>> To me, comparing object notation with programming language is not
>> helpful to the OP's purpose.
>
> Yes, I agree, it was a distraction. I fell into the trap of responding
> to the ludicrous claim that "if X is a superset of Y, then X cannot
> possibly look better than Y" (a claim made by multiple people all
> thinking it was clever) by showing that Y has other supersets that do in
> fact look better than Y.
It's not ludicrous.
You claim that:
(1) JSON is too hard to edit;
(2) RSON is a superset of JSON (the PEP even explicitly says "All valid
UTF-8 encoded JSON files are also valid RSON files");
(3) which implies that all JSON files are valid RSON files.
If you reject the logical conclusion that RSON must therefore also be too
hard to edit, then perhaps JSON isn't too hard to edit either.
You seem to be taking the position that if you start with a config file
config.json, it is "too hard to edit", but then by renaming it to
config.rson it magically becomes easier to edit. That *is* ludicrous.
Perhaps what you mean to say is that JSON *can be* (not is) too hard to
edit, and RSON *can be* too hard to edit too, but RSON has additional
benefits, including being easier to edit *sometimes*.
So far you have done (in my opinion) a really poor job of explaining what
those benefits are. You've bad-mouthed existing config formats, then
tried to convince us that RSON is almost exactly the same as one of those
formats apart from a couple of trivial changes of spelling (True for
true, etc.).
In my opinion, if you're going to get any traction with RSON, you need to
demonstrate some examples of where JSON actually is hard to write, and
show how RSON makes it easier. It's not good enough showing badly written
JSON, it has to be examples that can't be written less badly given the
constraints of JSON.
--
Steven
Paul Rubin
> (3) which implies that all JSON files are valid RSON files.
>
> If you reject the logical conclusion that RSON must therefore also be too
> hard to edit, then perhaps JSON isn't too hard to edit either.
I would say that JSON is hard to edit because, among other things, it
has no comment syntax. It's quite difficult to maintain a hand-edited
JSON file, or figure out what to edit into it, if it can't have any
comments in it describing what it's doing. JSON is a serialization
format for machine to machine communication, not intended for hand
editing. Simply adding a comment syntax to JSON would go a long way
towards making it easier to edit.
Patrick Maupin
cybersource.com.au> wrote:
> You seem to be taking the position that if you start with a config file
> config.json, it is "too hard to edit", but then by renaming it to
> config.rson it magically becomes easier to edit. That *is* ludicrous.
No, but that seems to be the position you keep trying to ascribe to
me: "Wait a minute... if JSON is too hard to edit, and RSON is a
*superset* of JSON, that means by definition every JSON file is also a
valid RSON file. Since JSON is too hard to manually edit, so is
RSON."
> Perhaps what you mean to say is that JSON *can be* (not is) too hard to
> edit, and RSON *can be* too hard to edit too, but RSON has additional
> benefits, including being easier to edit *sometimes*.
I have said as much, in multiple prior postings on this thread, if you
care to look.
> So far you have done (in my opinion) a really poor job of explaining what
> those benefits are.
That's certainly a valid complaint, and I can certainly believe that's
true (and I'm going to be working on the doc some more), but instead
of making this complaint, you made the (still silly IMO) assertion
that a superset has to be as unreadable as the base set.
> You've bad-mouthed existing config formats, then
> tried to convince us that RSON is almost exactly the same as one of those formats apart from a couple of trivial changes of spelling (True for true, etc.).
This is conflating two different issues. In a lame attempt to show
the silliness of the argument that the superset must be as bad as the
subset, I was trying to explain that, if you make a very few changes,
Python is a superset of JSON. However, my lame attempt at this was
obviously as poorly thought out as some of the rest of my words,
because it started an additional discussion that wasn't really germane
to the real issues about configuration files.
> In my opinion, if you're going to get any traction with RSON, you need to
> demonstrate some examples of where JSON actually is hard to write, and
> show how RSON makes it easier. It's not good enough showing badly written
> JSON, it has to be examples that can't be written less badly given the
> constraints of JSON.
Agreed, and I will be working on that.
Best regards,
Pat
Alf P. Steinbach
> On Mar 2, 5:36 pm, Steven D'Aprano <st...@REMOVE-THIS-
> cybersource.com.au> wrote:
>> You seem to be taking the position that if you start with a config file
>> config.json, it is "too hard to edit", but then by renaming it to
>> config.rson it magically becomes easier to edit. That *is* ludicrous.
>
> No, but that seems to be the position you keep trying to ascribe to
> me: "Wait a minute... if JSON is too hard to edit, and RSON is a
> *superset* of JSON, that means by definition every JSON file is also a
> valid RSON file. Since JSON is too hard to manually edit, so is
> RSON."
Just a heads-up: IME it's next to impossible (maybe actually impossible) to
convince Steven that your position is not the one he ascribes to you. At least
not without an extremely lengthy debate veering off on multiple similar tangents
plus some that one would never dream of. So I'd give up on that.
Anyway, I agree with the sentiments expressed by many in this thread that what's
important is the established "eco-culture", not the editability.
Thus, even though XML isn't all that editable it's what I'd choose by default
for rich configuration information, regardless of whether some simpler to edit
alternative existed. Simply because there are so many XML tools out there, and
people know about XML. It's like MS-DOS once became popular in spite of being
extremely technically imperfect for its purpose, and then Windows... :-)
Cheers,
- Alf
Erik Max Francis
> On Mar 2, 5:36 pm, Steven D'Aprano <st...@REMOVE-THIS-
> cybersource.com.au> wrote:
>> You seem to be taking the position that if you start with a config file
>> config.json, it is "too hard to edit", but then by renaming it to
>> config.rson it magically becomes easier to edit. That *is* ludicrous.
>
> No, but that seems to be the position you keep trying to ascribe to
> me: "Wait a minute... if JSON is too hard to edit, and RSON is a
> *superset* of JSON, that means by definition every JSON file is also a
> valid RSON file. Since JSON is too hard to manually edit, so is
> RSON."
Huh? That's the argument being used against you, not the argument being
ascribed to you. You're getting confused about something, somewhere.
--
Erik Max Francis && m...@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 18 N 121 57 W && AIM/Y!M/Skype erikmaxfrancis
I wonder if heaven got a ghetto
-- Tupac Shakur
Patrick Maupin
> Patrick Maupin wrote:
> > On Mar 2, 5:36 pm, Steven D'Aprano <st...@REMOVE-THIS-
> > cybersource.com.au> wrote:
> >> You seem to be taking the position that if you start with a config file
> >> config.json, it is "too hard to edit", but then by renaming it to
> >> config.rson it magically becomes easier to edit. That *is* ludicrous.
>
> > No, but that seems to be the position you keep trying to ascribe to
> > me: "Wait a minute... if JSON is too hard to edit, and RSON is a
> > *superset* of JSON, that means by definition every JSON file is also a
> > valid RSON file. Since JSON is too hard to manually edit, so is
> > RSON."
>
> Huh? That's the argument being used against you, not the argument being
> ascribed to you. You're getting confused about something, somewhere.
>
> --
> San Jose, CA, USA && 37 18 N 121 57 W && AIM/Y!M/Skype erikmaxfrancis
> I wonder if heaven got a ghetto
> -- Tupac Shakur
Yes, it is very confusing. Steven used that purported argument
against me, and then since I disagreed with it, it apparently meant
that I was arguing that changing the file extension type (which I've
never even proposed or discussed, btw) from json to rson somehow
magically makes a preexisting file all better. I have a headache now,
and it will only get worse, so that's really all I have left to say
about this issue.
Best regards,
Pat
Steven D'Aprano
> Yes, it is very confusing. Steven used that purported argument against
> me, and then since I disagreed with it, it apparently meant that I was
> arguing that changing the file extension type (which I've never even
> proposed or discussed, btw) from json to rson
My argument is *figurative*, not literal.
> somehow magically makes a preexisting file all better.
You might not have intended that ludicrous conclusion, but it was implied
by some of your posts, particularly the early ones. Of course I don't
think that you *actually* believe the conclusion.
I believe that, whatever the real or imagined benefit of RSON over JSON,
in this thread you have attempted to over-sell RSON while giving
relatively little justification for it.
I wish you every success in your project, but think that it is off-topic
for a Python discussion group.
--
Steven
Erik Max Francis
> On Mar 2, 9:20 pm, Erik Max Francis <m...@alcyone.com> wrote:
>> Patrick Maupin wrote:
>>> On Mar 2, 5:36 pm, Steven D'Aprano <st...@REMOVE-THIS-
>>> cybersource.com.au> wrote:
>>>> You seem to be taking the position that if you start with a config file
>>>> config.json, it is "too hard to edit", but then by renaming it to
>>>> config.rson it magically becomes easier to edit. That *is* ludicrous.
>>> No, but that seems to be the position you keep trying to ascribe to
>>> me: "Wait a minute... if JSON is too hard to edit, and RSON is a
>>> *superset* of JSON, that means by definition every JSON file is also a
>>> valid RSON file. Since JSON is too hard to manually edit, so is
>>> RSON."
>> Huh? That's the argument being used against you, not the argument being
>> ascribed to you. You're getting confused about something, somewhere.
>
> against me, and then since I disagreed with it, it apparently meant
> that I was arguing that changing the file extension type (which I've
> never even proposed or discussed, btw) from json to rson somehow
> magically makes a preexisting file all better.
No, that was not the point he was making. You should reread more
carefully (and be aware of sarcasm).
--
Erik Max Francis && m...@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 18 N 121 57 W && AIM/Y!M/Skype erikmaxfrancis
Perfect situations must go wrong
-- Florence, _Chess_
mk
> Patrick Maupin <pma...@gmail.com> writes:
>> One of my complaints. If you had read the document you would have
>> seen others. I actually have several complaints about YAML, but I
>> tried to write a cogent summary.
> Yaml sucks, but seems to have gotten some traction regardless.
> Therefore the Python principle of "there should be one and only one
> obvious way to do it" says: don't try to replace the existing thing if
> your new thing is only slightly better.
With all due respect, Paul, and with thanks for all the help you've
given me, I have to disagree here: this is a really, really complicated
matter and I think there is a case even for making things slightly better.
I think this is a matter of "investment period", so to speak: is it
short or long? In short term, it absolutely makes no sense to produce
even slight improvement.
But in the long run it will almost certainly pay off to switch to smth
even somewhat better implementation (say, imaginary "20%" if you get my
drift): suppose we stay with sucky format for 10 years. Wouldn't it make
sense to implement a new one and be "in red" in terms of effort expended
versus saved for 3 years, but then be "in black" for the following 7 years?
> Just deal with the existing
> thing's imperfections or make improvements to it.
OK, but how? How would you make up e.g. for JSON's lack of comments?
Producing accompanying ".json-comment" format and writing libraries that
parse the comments and interleave them with JSON file for producing
human-readable commented output?
I think the effort required by all parties, both developers and users,
before they produced smth like this and learned to use this widely and
comprehensively, for this manner of improvement would be so high that it
would be actually cheaper to dump the thing and develop smth new that
has built-in support for comments.
If you mean some other method of improving existing things like formats,
well let's hear it; but I for one don't see any worth doing to
significant extent really, other than dumping the thing or producing
next, improved version at least.
Improvement: other than making basic tools like parsing libraries
editors, what improvements can you realistically make? And such
improvements in and of themselves are not very expensive: my GPL
Notepad++ has syntax highlighting for YAML (on top of gazillion other
languages), and there are parsing libraries for it. So where's this
terrible cost to it?
OTOH, if YAML produces net benefit for as few as, say, 200 people in
real world, the effort to make it has been well worth it.
> If you can make a
> really powerful case that your new thing is 1000x better than the old
> thing, that's different, but I don't think we're seeing that here.
Perhaps in ideal world we would be able to develop smth good or at least
decent without long series of abominations preceding it.
But I don't think we live in such world and I don't think it's possible
to produce a decent format (or language) without decades of having to
deal with abominations first. We learn as we go along, there's no way
but to produce whatever works best at the moment, learning from it,
dumping it and then doing smth better. I don't think e.g. Python could
be produced without C, COBOL and Fortran preceding it: it's important
not only to know how to do it, but also how (and why) not to do it, and
learning that can't be done without producing some sort of abomination.
I'd argue that abominations are inevitable price of progress and evolution.
> Also, XML is used for pretty much everything in the Java world. It
> sucks too, but it is highly standardized, it observably gets the job
> done, there are tons of structure editors for it, etc. Frankly
> I'd rather have stayed with it than deal with Yaml.
http://myarch.com/why-xml-is-bad-for-humans
http://www.ibm.com/developerworks/xml/library/x-sbxml.html
Such reasons alone are enough to consider dumping XML for smth better.
Today I had to hand-edit XML config files for two programs (no other
option). The files were rather large, complicated and doing it frankly
sucked.
I also have to maintain a few applications that internally use XML as
data format: while they are tolerable, they still leave smth to be
desired, as those applications are really slow for larger datasets,
their users systematically make errors (like forgetting to attach DTD
before editing), and working across various versions of Windows is still
not perfect.
If somebody out there invents smth that is better than XML "only" by
half, I'm all for it.
> There are too many of these damn formats. We should ban all but one of
> them (I don't much care which one). And making even more of them is not
> the answer.
I think this is a situation of "beware of what you wish for". Suppose
those alternative formats disappeared and you'd have no tools, however
imperfect, to use them: I think that in many, many contexts deficiencies
of "the" format would be so painful that most developers would just
write their own "private" ones, and everyone would be even worse off
than they are now.
I wouldn't worry too much about "stretching scarce resources thin"
either: abominations completely unfit to live waste little in the way of
resources, and we learn a deal off them too.
There are demonstrable benefits to this too: I for one am happy that
ReST is available for me and I don't have to learn a behemoth such as
DocBook to write documentation.
(imaginary dialog:
Paul: "eat your greens and learn your DocBook!"
Me: "but I don't like it and there's too much of it..."
;-)
OK me off the soapbox.
Regards,
mk
Steve Howell
> >> One of my complaints. If you had read the document you would have
> >> seen others. I actually have several complaints about YAML, but I
> >> tried to write a cogent summary.
> > Yaml sucks, but seems to have gotten some traction regardless.
> > Therefore the Python principle of "there should be one and only one
> > obvious way to do it" says: don't try to replace the existing thing if
> > your new thing is only slightly better.
>
> With all due respect, Paul, and with thanks for all the help you've
> given me, I have to disagree here: this is a really, really complicated
> matter and I think there is a case even for making things slightly better.
>
> I think this is a matter of "investment period", so to speak: is it
> short or long? In short term, it absolutely makes no sense to produce
> even slight improvement.
>
> But in the long run it will almost certainly pay off to switch to smth
> even somewhat better implementation (say, imaginary "20%" if you get my
> drift): suppose we stay with sucky format for 10 years. Wouldn't it make
> sense to implement a new one and be "in red" in terms of effort expended
> versus saved for 3 years, but then be "in black" for the following 7 years?
>
I think ten years is about the right horizon to be looking down.
Anything further than ten years is probably so speculative as to have
lots of diminishing returns, although I admire people who have really
BIG ideas and are starting on them now. (The particular context of
this thread doesn't lead to big ideas, unless I just lack
imagination.)
I think it's wrong, though, only to look five years ahead. If the
only goal was to make software development easier in 2015, then I'd
say, by all means, let's pick the current best-of-breed tools and
simply perfect them as much as we can. This is a worthwhile goal
regardless of your ultimate time horizon, and that effort tends to
happen anyway, since so many people rightly live in the here and
now.
Somewhere in the 2020s, though, I predict that a lot of technologies
are either going to finally die off, or at least be restricted to the
niches that they serve well. Take Java, for example. I think it will
be still be used, and people will still even be writing new programs
in it, but it will be rightly scorned in a lot of places where it is
now embraced. Some of this won't actually be due to technological
advances, but just changes in perception. For example, I predict lots
of programs that people now write in Java will be written in Python,
even if the core language of Python remains fairly stable.
Beyond just changing mindsets, though, I think evolution is
inevitable. Some subset of Python tools will almost certainly develop
features that are more friendly to the Java mindset, but work in
Python, and this will help move folks from Java to Python. I also
think that Java will be supplanted for lots of use cases by some
languages invented after 2000. Maybe Scala will become more
mainstream. Maybe Go will turn into more of an enterprise-y
platform. Who knows?
With regard to XML, I think at a bare minimum, folks will stop using
XML for use cases that YAML, JSON, and maybe even RSON serve better
today. I bet that at least one of YAML and JSON survives in some
form, and my money is on JSON, but I bet there will also be some new
competing formats. I also think that we'll still have multiple
formats that are only marginally better than each other for general
use cases, but which people will still choose for specific reasons.
Developers *love* good tools almost as much as they hate confusion--
there will always be tension between having too many choices and not
enough.
Going back to Paul's statement, I agree that "there should be one and
only one obvious way to do it" in Python, but I don't think the
philosophy applies to the greater ecosystem of software development.
In our generation I think we have the live with the confusion and
chaos that comes from a plethora of tools, and that's just part of
progress. Ironically, I think the tools that survive will be very
focused in their own right; it's just that we'll still have many to
choose from.
Going back to XML, I found myself using it last night for a completely
inappropriate use case. It so happens that it would have been about
100% better if it had simply been written in JSON, so there was no
compelling need for yet another alternative. But if the inventers of
JSON had been complacent about XML, we wouldn't even have that as an
option. And, of course, there is nothing radical at all about JSON--I
am pretty sure it was just a common sense realization about the
inadequacies of current technologies that led to its development, and
I'm sure early versions of it were pretty raw. Without having looked
into RSON, I am sure it's the same mindset that drives its invention--
current tools exist that can get the some done, but we can do better.
Whether RSON is really an improvement or not is an orthogonal issue to
whether we should strive for improvement.
mk
> Somewhere in the 2020s, though, I predict that a lot of technologies
> are either going to finally die off, or at least be restricted to the
> niches that they serve well. Take Java, for example. I think it will
> be still be used, and people will still even be writing new programs
> in it, but it will be rightly scorned in a lot of places where it is
> now embraced. Some of this won't actually be due to technological
> advances, but just changes in perception. For example, I predict lots
> of programs that people now write in Java will be written in Python,
> even if the core language of Python remains fairly stable.
A friend of mine, and a good Java programmer, says caustically: "Java is
COBOL of the future".
Where I work we develop a huge application in Websphere (IBM Java-based
application server). The problems with legacy code made project manager
joke "perhaps we should rewrite this in Python". Perhaps some day it
will not be a joke anymore?
Personally, I chose to stay away from Java, even though it would
temporarily help me: the amount of time & effort it takes to master the
necessary toolset is *huge*, and my scarce time is better spent
elsewhere, on more productive tools, and I really, really do not want
lots of my limited time to go down the drain in a few years.
Take EJB for example: even its creators realized they've overdone it
with EJB 2 and simplified somewhat EJB 3 and switched to annotations
instead of gazillion XML formats. But still I dread the thought of
having to spend so much time learning it before I can do a few lines of
productive work in it.
In a way it's horrible: all this gargantuan effort in a few years will
be completely wasted, down the drain. All those developer hours and
dollars wasted.. In a way, C wasn't as bad as Java has been: at least
many of C libs, with new bindings, still live on and do work.
> Going back to Paul's statement, I agree that "there should be one and
> only one obvious way to do it" in Python, but I don't think the
> philosophy applies to the greater ecosystem of software development.
+1
Note that when it comes to bigger tools or frameworks, even in the world
of Python things are not "one obvious way", e.g. Django for quick and
dirty and small apps, and Pylons for big and powerful apps. There may be
"one obvious way to do it" in a very, very narrow context, but when
contexts widen, like, say: "what is web framework I should choose?" the
answers diverge, because answer has to be variation of "it depends on
your situation".
> Whether RSON is really an improvement or not is an orthogonal issue to
> whether we should strive for improvement.
+1
Regards,
mk
Paul Rubin
> OK, but how? How would you make up e.g. for JSON's lack of comments?
Modify the JSON standard so that "JSON 2.0" allows comments.
> OTOH, if YAML produces net benefit for as few as, say, 200 people in
> real world, the effort to make it has been well worth it.
Not if 200,000 other people have to deal with it but don't receive the
benefit.
> http://myarch.com/why-xml-is-bad-for-humans
> http://www.ibm.com/developerworks/xml/library/x-sbxml.html
You might like this one too:
http://www.schnada.de/grapt/eriknaggum-xmlrant.html
>
> I also have to maintain a few applications that internally use XML as
> data format: while they are tolerable, they still leave smth to be
> desired, as those applications are really slow for larger datasets,
I thought we were talking about configuration files, not "larger datasets".
> There are demonstrable benefits to this too: I for one am happy that
> ReST is available for me and I don't have to learn a behemoth such as
> DocBook to write documentation.
DocBook is so far off my radar I'd have never thought of it. I just now
learned that it's not Windows-only. There is already POD, Pydoc,
Texinfo, a billion or so flavors of wiki markup, vanilla LaTeX, and most
straightforwardly of all, plain old ascii. ReST was another solution in
search of a problem.
Steve Howell
> mk <mrk...@gmail.com> writes:
> > OK, but how? How would you make up e.g. for JSON's lack of comments?
>
> Modify the JSON standard so that "JSON 2.0" allows comments.
If you don't control the JSON standard, providing a compelling
alternative to JSON might be the best way to force JSON to accomodate
a wider audience. It might just be that the people behind JSON
deliberately avoid comments, because it's not in the scope of the
problem they are trying to solve. Hence another need for
alternatives.
> > OTOH, if YAML produces net benefit for as few as, say, 200 people in
> > real world, the effort to make it has been well worth it.
>
> Not if 200,000 other people have to deal with it but don't receive the
> benefit.
>
How many hundreds of thousands of people have had to deal with XML
without receiving its benefits? Do well-established standards get an
exemption from the rule that software is not allowed to annoy non-
willing users of it?
Gregory Ewing
> ReST was another solution in search of a problem.
I think the basic idea behind ReST is quite good, i.e.
understanding as markup various typographical conventions
that make sense in plain text, such as underlined
headings, bullets, numbered paragraphs.
Unfortunately it went overboard with a slew of cryptic
codes for footnotes, hyperlinks, etc. that nobody would
naturally think to use in a plain text document.
--
Greg
Steve Howell
The same thing happened with YAML to a certain extent, from my
perspective. YAML was never meant to be an exact alternative to XML,
but its basic premise was sound--use indentation for more elegant
syntax, and model its semantics more toward how data actually gets
used internally by scripting languages in particular. But there is
also some featuritis with YAML that makes it hard to digest and
needlessly cumbersome to implement.
JSON is not perfect by any means, but I consider it to be a more
useful descendant of XML and YAML, even if it did not directly borrow
from either. (YAML and JSON are certainly similar, but that could be
a coincidental convergence.) Even if YAML itself has not been a
resounding success, it set the bar to a certain degree.
Paul Rubin
>> Modify the JSON standard so that "JSON 2.0" allows comments.
>
> If you don't control the JSON standard, providing a compelling
> alternative to JSON might be the best way to force JSON to accomodate
> a wider audience.
Ehh, either the JSON standardizers care about this issue or else they
don't. JSON (as currently defined) is a machine-to-machine
serialization format and just isn't that good a choice for handwritten
files. Adding a comment specification is a small perturbation that
might be accepted into the standard, but a big departure like RSON is a
whole nother creature.
> How many hundreds of thousands of people have had to deal with XML
> without receiving its benefits? Do well-established standards get an
> exemption from the rule that software is not allowed to annoy non-
> willing users of it?
We already have to deal with XML. So using XML for config files doesn't
require anyone to deal with any lousy formats that they didn't have to
deal with before. So the basic answer to your question about
well-established standards is yes: one annoying but standardized format
is better than multiple annoying unstandardized ones.
Steven D'Aprano
I use footnotes all the time[1] in plain text documents and emails. I
don't think there's anything bizarre about it at all.
[1] When I say "all the time", I actually mean occasionally.
--
Steven
John Bokma
> On Fri, 05 Mar 2010 18:36:06 +1300, Gregory Ewing wrote:
>
>> Paul Rubin wrote:
>>> ReST was another solution in search of a problem.
>>
>> I think the basic idea behind ReST is quite good, i.e. understanding as
>> markup various typographical conventions that make sense in plain text,
>> such as underlined headings, bullets, numbered paragraphs.
>>
>> Unfortunately it went overboard with a slew of cryptic codes for
>> footnotes, hyperlinks, etc. that nobody would naturally think to use in
>> a plain text document.
>
> I use footnotes all the time[1] in plain text documents and emails. I
> don't think there's anything bizarre about it at all.
http://docutils.sourceforge.net/docs/user/rst/quickref.html#footnotes [#]_.
.. [#] the keyword is ReST.
--
John Bokma j3b
Hacking & Hiking in Mexico - http://johnbokma.com/
http://castleamber.com/ - Perl & Python Development
Steve Howell
>
> Ehh, either the JSON standardizers care about this issue or else they
> don't. JSON (as currently defined) is a machine-to-machine
> serialization format and just isn't that good a choice for handwritten
> files. Adding a comment specification is a small perturbation that
> might be accepted into the standard, but a big departure like RSON is a
> whole nother creature.
>
> > How many hundreds of thousands of people have had to deal with XML
> > without receiving its benefits? Do well-established standards get an
> > exemption from the rule that software is not allowed to annoy non-
> > willing users of it?
>
> We already have to deal with XML. So using XML for config files doesn't
> require anyone to deal with any lousy formats that they didn't have to
> deal with before. So the basic answer to your question about
> well-established standards is yes: one annoying but standardized format
> is better than multiple annoying unstandardized ones.
<question type="rhetorical">
Does this mean we should stick with XML until the end of time?
</question>
Tim Chase
One word: COBOL
:-)
-tkc
Aahz
Paul Rubin <no.e...@nospam.invalid> wrote:
>
>ReST is another abomination that should never have gotten off the
>ground. It is one of the reasons I react so negatively to your
>config format proposal. It just sounds like more of the same.
Really? What should we use instead of reST?
--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/
"Many customs in this life persist because they ease friction and promote
productivity as a result of universal agreement, and whether they are
precisely the optimal choices is much less important." --Henry Spencer
Gregory Ewing
> I use footnotes all the time[1] in plain text documents and emails. I
> don't think there's anything bizarre about it at all.
It's not the use of footnotes I'm talking about, it's
the cryptic character sequences used to mark them up
in ReST. Nobody would come up with them on their own
without having read the fine print of an ReST manual.
--
Greg
Robert Kern
> On Fri, Mar 5, 2010 at 2:45 PM, Aahz <aa...@pythoncraft.com
> <mailto:aa...@pythoncraft.com>> wrote:
> > In article <7xwrxv4...@ruckus.brouhaha.com
> <mailto:7xwrxv4...@ruckus.brouhaha.com>>,
> > Paul Rubin <no.e...@nospam.invalid> wrote:
> >>
> >>ReST is another abomination that should never have gotten off the
> >>ground. It is one of the reasons I react so negatively to your
> >>config format proposal. It just sounds like more of the same.
> >
> > Really? What should we use instead of reST?
>
> Python as to be nearly Python-specific.
> It'd be nice not to have to learn yet another markup language without
> good reason.
I believe ReST predates all of those mentioned.
Paul Rubin
>> Markdown, Textile, or possibly Creole;...
> I believe ReST predates all of those mentioned.
Texinfo and POD are certainly older than ReST. I'm not sure about
MediaWiki markup, but it's gotten so much traction that Markdown
should probably be abandoned in its favor even if Markdown is older.
Patrick Maupin
the tokenizer saves the indentation information for the parser), but
the initial prototype parses all of JSON plus a lot of syntax
enhancements (comments, hex/binary/octal numbers, relaxed quoting
requirements for strings, trailing comments allowed, triple-quoted
strings, etc.)
I have also started the rson-discuss group on google for further
discussions on the actual syntax and initial implementation, if anyone
is interested, and have posted a message there with a bit more
information about the prototype status.
The project page is at http://code.google.com/p/rson/
Best regards,
Pat
Lawrence D'Oliveiro
> Patrick Maupin wrote:
>>
>> Finding .ini configuration files too limiting, JSON and XML to hard to
>> manually edit, and YAML too complex to parse quickly, I have started
>> work on a new configuration file parser.
>
> You're not supposed to edit those formats manually.
Strange. A previous client of mine was a totally non-IT-technical guy, yet I
was able to set him up with editing XML files (needed to drive some software
I wrote for him) just fine.
Of course he was using a purpose-built editor, namely KXMLEditor.