[racket] Json pretty-print

[Nick Shelley]

Is there a way to get formatted json output to a file? I couldn't figure out how to get pretty-print hooked in with write-json. I'm probably just missing something simple.

[Nick Shelley]

I ended up just doing a system call to Python as suggested here: http://stackoverflow.com/questions/352098/how-to-pretty-print-json-from-the-command-line.

When one of my coworkers (who I've been trying to convince that Racket is a real language) reviewed my code, he laughed at that line because he's a Python guy. I guess he wins this round...

[Nick Shelley]

In talking with an experienced Racketeer, I realized that pretty-print has a specific meaning in Racket that takes width into account. What I really want is json with properly placed newlines and tabs (or spaces).

I'm willing to take a stab at it if it doesn't exist. I just wanted to make sure I wasn't missing it somewhere (which happens often with me).

[Eli Barzilay]

Just now, Nick Shelley wrote:
> In talking with an experienced Racketeer, I realized that
> pretty-print has a specific meaning in Racket that takes width into
> account.

That's exactly what I thought you were talking about. Doing it from
scratch seems wrong, and I thought that the right way to do it is to
hook into the pretty-printer that we already have.

> What I really want is json with properly placed newlines and tabs
> (or spaces).

This sounds somewhere between very easy (if all you want is
newlines+indentation up to a specific level) to something more
complicated if you want to control other aspects like spaces around
colons, sort keys in tables, etc.

> I'm willing to take a stab at it if it doesn't exist. I just wanted
> to make sure I wasn't missing it somewhere (which happens often with
> me).

If you can summarize the python-equivalent features in a rackety way,
I can see if it's easy to add them to our printer.

IIUC, this interface means that `write-json' would get a bunch of
additional keywords, but I'm not sure about it, since in Racket
`write' usually refers to something closer to serializing a value in a
readable way rather than printing it in a readable way. So maybe the
right thing is to add a `print-json' function even if it duplicates
the functionality of `write-json'.

--
((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay:
http://barzilay.org/ Maze is Life!
____________________
Racket Users list:
http://lists.racket-lang.org/users

[Sean McBeth]

I've had good luck porting simple Python examples to Racket. Perhaps you could start with the Python code that works as you expect, port it to Racket, then refactor it to more Rackety ways:

http://hg.python.org/cpython/file/cc0e72082e52/Lib/json

[Nick Shelley]

I'm pretty sure what I want is the easy thing. We have data represented as json, and we want changes to it to be easy to see in a difftool. Everything on one line makes this hard. We just have a bunch of nested arrays and dictionaries that need to be tabbed and newlined with no regard to width.

As far as order goes, I think Python alphabetizes the keys, but for our purposes, order isn't important as long as it's consistent (so we don't get diffs that just change the order).

I'm willing to implement it, and I might enjoy learning the process of contributing with something as easy as this. But if someone else gets to it first because it's easier to just do it right the first time than rewrite mine, I wouldn't mind.

Thanks for the responses and sorry for my unclear initial question.

[Carl Eastlund]

Nick,

I've just built pretty-printing for JSON in a Gist you can see here:

  https://gist.github.com/carl-eastlund/5398440

It's built on top of my own "stylish printing" library, mischief/stylish, that you can install by running:

  raco pkg install mischief

I'll add this to the mischief package soon, for now you can copy it from the Gist site and modify as needed.  I don't know if this indents things the best possible way, but hopefully it's a good start.

Carl Eastlund

[Danny Yoo]

Similar forking going on. :)

Here's another version (based on the json library in the standard
library, but hacked up a bit):

https://gist.github.com/dyoo/5398549

I think the point is: it should be very easy to do this; it's just a
matter of knowing what behavior you want.

[Eli Barzilay]

40 minutes ago, Nick Shelley wrote:
> I'm pretty sure what I want is the easy thing. We have data
> represented as json, and we want changes to it to be easy to see in
> a difftool. Everything on one line makes this hard. We just have a
> bunch of nested arrays and dictionaries that need to be tabbed and
> newlined with no regard to width.

If the goal is just diffing, then it makes things a little different:
you'll want newlines, and you won't care about indentation. But a
better solution would be to find a diff tool that works on the line,
or even better a tree diff thing...

Alternatively, you could just pretty-print the Racket value and use
that output for the diff.

> As far as order goes, I think Python alphabetizes the keys, but for
> our purposes, order isn't important as long as it's consistent (so
> we don't get diffs that just change the order).

You definitely want the keys sorted -- Racket hash tables don't
guarantee some specific order, and it's likely to change from run to
run.

[Danny Yoo]

Racket provides a generic order predicate: see data/order:

http://docs.racket-lang.org/data/Orders_and_Ordered_Dictionaries.html

For example,

https://gist.github.com/dyoo/5398549#file-pretty-print-json-rkt-L5
https://gist.github.com/dyoo/5398549#file-pretty-print-json-rkt-L12
https://gist.github.com/dyoo/5398549#file-pretty-print-json-rkt-L12

[Eli Barzilay]

That's not really relevant in this case: there is no problem here
comparing the keys since they are all symbols.

10 minutes ago, Danny Yoo wrote:
> Racket provides a generic order predicate: see data/order:
>
> http://docs.racket-lang.org/data/Orders_and_Ordered_Dictionaries.html
>
> For example,
>
> https://gist.github.com/dyoo/5398549#file-pretty-print-json-rkt-L5
> https://gist.github.com/dyoo/5398549#file-pretty-print-json-rkt-L12
> https://gist.github.com/dyoo/5398549#file-pretty-print-json-rkt-L12

--
((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay:
http://barzilay.org/ Maze is Life!

[Nick Shelley]

First off, thanks Carl and Danny for the code. The thing is, the system call to python works for our needs and is easier than rolling my own. The reason I started this thread in the first place is because it seems like outputting human-readable json is common and often desired (see the upvotes in the SO article in the second message for proof). Because it seems so common, I just assumed I was missing something. When I found out it was missing from the Racket json module, I thought it would be more complete with this functionality. I'm now seeing that there are many different uses and desires and design choices that need to be made, so what functionality to provide isn't as clear as I thought (I'm not sure what development process the other libraries went through for the human-readable option).

I still would like to see this functionality in the Racket json library, but since the Python thing works for now, my reasons are mainly to save face with my coworker and to avoid repeating this process if I ever think I need human-readable json in the future and want to use Racket. I also thought it might be cool to make a contribution, but seeing the code provided above, I'm thinking I may not be as qualified as I thought.

My following responses should be taken in that context:

Eli Barzilay wrote:

> If the goal is just diffing, then it makes things a little different:
> you'll want newlines, and you won't care about indentation. 

The goal is to have human-readable json mainly for the purpose of making diffing easier. Our data is a nested hierarchy, and I would argue that indentation makes it more readable by humans. I agree that it's not completely necessary for diffing, but it does help visualize hierarchy depth, which may be part of what's different.

> But a better solution would be to find a diff tool that works on the line,
> or even better a tree diff thing...

More specifically, this json file is part of project in a git repository, and we use gerrit (https://code.google.com/p/gerrit/) for code reviews. Large one-line diffs are hard to make sense of, and gerrit only allows comments at the line level.

Danny Yoo wrote:

> I think the point is: it should be very easy to do this; it's just a
> matter of knowing what behavior you want.

We used to have an Objective-C project with all the data hard-coded and written to json when we wanted, but I couldn't stand editing that beast any more, so I made a  Racket command-line tool to read in the json and process commands to edit it in ways we need. I would have just switched to a database, but the json representation is a sanity check to make sure I didn't do anything stupid (and also for others to verify that in gerrit), hence the need to have it human readable.

I assume we'll eventually go to some neat UI or web interface or something, but that probably won't happen for a while. I give you that context because I think it makes the desired behavior clearer than me trying to describe it, and it also gives you the chance to tell me what I should want.

[Grant Rettke]

On Tue, Apr 16, 2013 at 4:23 PM, Nick Shelley <nickms...@gmail.com> wrote:
> The thing is, the system call to python works for our needs and is easier than rolling my own.

FWIW it is so pleasant in Python that at work the .NET guys use Python
to do the same thing! :)