files in synthetic file systems for status, e.g. /proc

[ron minnich]

files in Unix /proc have always been a mess. There are tables, tuples, tables and tuples, numbers, ... it's not consistent because it grew in pieces.

This also covers other synthetics, since in the plan 9 model each driver offers its own tree, which is very different from the centralized /sys tree in linux. In plan 9, however, things are similarly inconsistent. Sometimes you get one line of numbers. Sometimes you get a thing that's like a table. Sometimes it's key value pairs. It's a mess. And, once a given file in the synthetic is  changed, sometimes it's impossible to know how to read it.

For synthetics that offer status, e.g. /net/tcp/*/status and so on, I'd like to propose something we did on Linux for the supercomputers that worked really well, and which is even used by HP to this day on their cluster offerings, though I know it's a tough sell: S-expressions.

S-expressions are handy because they are self-describing structures, trivial to process, and easy to merge (as we did in supermon, where we were merging 4500 such streams into one for cluster monitoring). Processing is insensitive to change: new components in the s-expression won't interfere with old processing pipelines. 

So, taking linux meminfo as one example, it comes out like this today:

MemTotal: xyz KB

MemFree: zzz KB

...

S-expressions would look like this

((MemTotal xyz) (MemFree xyz) ...)

N.B.: the s-expression form can be just as compact, byte-wise, as the current form. 

I realize that linux  has made a valiant effort in the last 10+ years to move non-process things  to /sys and they've also enforced one value per file, but that has its own set of problems: you have one system call (minimum) per value. That's slow. For the monitor device we built for linux, we had about 100 values we could pull out at 12,000 hz. (30Khz on plan 9). This allowed us to see some useful artifacts with no measurable impact on the system: we could actually get a global picture of MPI reduce actions, for example, which were invisible with existing monitoring that could only watch at a fraction of a Hz. One value per file proved to be too slow.

The software we used to parse this stuff is here: http://sexpr.sourceforge.net/

it's small and fast. 

Before you ask: yes, we looked at XML. For whatever reason, XML parsers are big, complex, and slow. Matt and I moved to s-expressions after a frustrating couple of weeks working with XML. I don't know why, since XML is  really just 'labelled s-expressions', but there you are. I think it's why XML has fallen out of favor in many places, in favor of other formats. 

I'm not that wedded to s-expressions; any sort of self describing stream that allows for hierarchy (nesting) would be fine. JSON is one possibility.I like the fact that to process JSON data in Go I do this:

import "encoding/json"

and I'm done. It's why I used JSON for the build tool on Harvey; the parsing became Someone Else's Problem, and JSON was just good enough.

Anyway, I wanted to mention this before we got too far. I'd like to fix the synthetic file mess once and for all, and some sort of self-describing, uniform, nested format would be nice -- just don't say "XML" or my head will explode ...

ron

[Davide Libenzi]

JSON is powerful enough to represent complex data, and yet simple to parse with 100 lines of C.

And furthermore a widely used standard.

It is not easy to the eye though.

XML? You kinda of scared me there ☺

--
You received this message because you are subscribed to the Google Groups "Akaros" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akaros+un...@googlegroups.com.
To post to this group, send email to aka...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dan Cross]

On Thu, Oct 29, 2015 at 5:12 PM, ron minnich <rmin...@gmail.com> wrote:

files in Unix /proc have always been a mess. There are tables, tuples, tables and tuples, numbers, ... it's not consistent because it grew in pieces.

This also covers other synthetics, since in the plan 9 model each driver offers its own tree, which is very different from the centralized /sys tree in linux. In plan 9, however, things are similarly inconsistent. Sometimes you get one line of numbers. Sometimes you get a thing that's like a table. Sometimes it's key value pairs. It's a mess. And, once a given file in the synthetic is  changed, sometimes it's impossible to know how to read it.

For synthetics that offer status, e.g. /net/tcp/*/status and so on, I'd like to propose something we did on Linux for the supercomputers that worked really well, and which is even used by HP to this day on their cluster offerings, though I know it's a tough sell: S-expressions.

As a former Lisp hacker, this isn't a touch sell to me. I'm all for a consistent, simple, record-oriented format. As you say, this has long been a deficiency in Unix: systems designed their own goofy formats and ad hoc parsers that were never *quite* the right fit for other systems. Yuck.

S-expressions are handy because they are self-describing structures, trivial to process, and easy to merge (as we did in supermon, where we were merging 4500 such streams into one for cluster monitoring). Processing is insensitive to change: new components in the s-expression won't interfere with old processing pipelines. 

So this is really the crux of it: something that's simple, elegant, extensible, easy to parse.

So, taking linux meminfo as one example, it comes out like this today:

MemTotal: xyz KB

MemFree: zzz KB

...

S-expressions would look like this

((MemTotal xyz) (MemFree xyz) ...)

N.B.: the s-expression form can be just as compact, byte-wise, as the current form. 

I realize that linux  has made a valiant effort in the last 10+ years to move non-process things  to /sys and they've also enforced one value per file, but that has its own set of problems: you have one system call (minimum) per value. That's slow. For the monitor device we built for linux, we had about 100 values we could pull out at 12,000 hz. (30Khz on plan 9). This allowed us to see some useful artifacts with no measurable impact on the system: we could actually get a global picture of MPI reduce actions, for example, which were invisible with existing monitoring that could only watch at a fraction of a Hz. One value per file proved to be too slow.

The software we used to parse this stuff is here: http://sexpr.sourceforge.net/

it's small and fast. 

Before you ask: yes, we looked at XML. For whatever reason, XML parsers are big, complex, and slow. Matt and I moved to s-expressions after a frustrating couple of weeks working with XML. I don't know why, since XML is  really just 'labelled s-expressions', but there you are. I think it's why XML has fallen out of favor in many places, in favor of other formats. 

Tangent: XML fails in many ways because it tries to do too much. That and the weird SGML-inspired syntax. I found, however, that the biggest problem with XML was the split between entities and attributes. Once you're used to s-expressions, you see that there's really no need for attributes and they just add a lot of semantic complexity for little gain.

I'm not that wedded to s-expressions; any sort of self describing stream that allows for hierarchy (nesting) would be fine. JSON is one possibility.I like the fact that to process JSON data in Go I do this:

import "encoding/json"

and I'm done. It's why I used JSON for the build tool on Harvey; the parsing became Someone Else's Problem, and JSON was just good enough.

So this is the thing, though; as much as I love Lisp, I'd rather see us use JSON than s-expressions. They're a lot more familiar, and also largely compatible with the text serialization of e.g. protobufs. S-expr's have a place in my heart, to be sure, but JSON is the better format for this type of thing.

Anyway, I wanted to mention this before we got too far. I'd like to fix the synthetic file mess once and for all, and some sort of self-describing, uniform, nested format would be nice -- just don't say "XML" or my head will explode ...

Ok.

        - Dan C.

 

[Kevin Klues]

+1 for JSON. I use JSON all the time for web development and it's great.

> --
> You received this message because you are subscribed to the Google Groups
> "Akaros" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to akaros+un...@googlegroups.com.
> To post to this group, send email to aka...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
~Kevin

[ron minnich]

OK, you all sold me. JSON it is. Our first test case will be #proc/pid/vmstats.

ron

[Kevin Klues]

Is the idea that *everything* we put in /proc will be in JSON format?
If so, I think that's great. Then we can write standard tools to
process anything and everything in /proc

[ron minnich]

Yes, and even more, every driver in Plan 9 that emits text (e.g. /net/tcp/0/stats) will emit json too. It's all JSON, 

everywhere, and then we only need one parser, not a different parser for every file type. 

It will be a bit of work, but worth it I think.

ron

[barret rhoden]

On 2015-10-29 at 23:27 ron minnich wrote:
> Yes, and even more, every driver in Plan 9 that emits text (e.g.
> /net/tcp/0/stats) will emit json too. It's all JSON,
> everywhere, and then we only need one parser, not a different parser
> for every file type.
>
> It will be a bit of work, but worth it I think.

As someone who regularly uses cat on plan 9 status files, I'd like a
second to think about this before our interface is JSON.

It sounds like this would only apply to certain files within the
synthetic devices, right? e.g. not #ip/tcp/0/ctl, but #ip/tcp/stats or
whatever.

One option here is what I did with mpstat:

case Kmpstatqid:
n = mpstat_read(va, n, offset);
break;
case Kmpstatrawqid:
n = mpstatraw_read(va, n, offset);
break;

There's two files in #kprof, one for humans that is a basic catting,
and another that is designed for a machine. The latter is a candidate
for better serialization.

stuff like this:
/* header line: version, num_cores, tsc freq, state names */
len += snprintf(buf + len, bufsz - len, "v%03d %5d %16llu", 1, num_cores,
system_timing.tsc_freq);

But it certainly is nice to be able to cat mpstat and have it work. An
alternative to having two separate QIDs for the same thing would be an
fcntl operation to change the mode from 'cat' to 'serialized'.


As far as the format for the serialized data, JSON seems fine.

I had been thinking (and perhaps I mentioned this to you, Ron, a month
or two ago) about using protobufs for a bunch of things within Akaros,
both between the kernel and user for various devices and between user
space apps (thinking more performance oriented IPC and it's message
format). I like the idea of having a bunch of .protos as part of the
kernel headers for kernel devices. Likewise, if our daemons export a
certain interface, it'd be nice to have that defined.

I guess the question is whether or not you want structured data with a
defined interface or not for this use case. I don't know too much about
protos and whether or not they'd meet the needs from your original
email (the hierarchical stuff). But they are fast and arch-independent.
I think their clearly defined structure is important for many things
we'll do in the future, but maybe it's not what you're looking for now.

Importantly, both JSON and protobufs would maintain the distributed
system aspect of plan 9.

Barret

[ron minnich]

OK, barrett, good points. This is always the conflict between human readable and machine readable stuff, last seen in errno vs. errstr.

So, we want

- a standard format for programs that lets us read synthetic files with one libary, not thousands of little functions

- human readable 'cat' output.

Merging suggestions:

- JSON format. There are good paths to serializing to other formats we might care about but no need to discuss them all here, right? :-)

- human readable files. So, e.g., cat /proc/pid/status will give us what we have now

- and, /proc/pid/status.json gives us json

sound ok?

ron

[Davide Libenzi]

Isn't it better to leave the kernel emitting JSON, and have a userspace "prettyfier" which could parse the JSON and emit eye-pleasing formats?

--

[Kevin Klues]

On Friday, October 30, 2015, 'Davide Libenzi' via Akaros <aka...@googlegroups.com> wrote:

Isn't it better to leave the kernel emitting JSON, and have a userspace "prettyfier" which could parse the JSON and emit eye-pleasing formats?

This would be my vote too. You couldn't just cat it anymore to see the human readable format, but I'm ok with that, personally.

 

On Fri, Oct 30, 2015 at 10:15 AM, ron minnich <rmin...@gmail.com> wrote:

OK, barrett, good points. This is always the conflict between human readable and machine readable stuff, last seen in errno vs. errstr.

So, we want

- a standard format for programs that lets us read synthetic files with one libary, not thousands of little functions

- human readable 'cat' output.

Merging suggestions:

- JSON format. There are good paths to serializing to other formats we might care about but no need to discuss them all here, right? :-)

- human readable files. So, e.g., cat /proc/pid/status will give us what we have now

- and, /proc/pid/status.json gives us json

sound ok?

ron

--
You received this message because you are subscribed to the Google Groups "Akaros" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akaros+un...@googlegroups.com.
To post to this group, send email to aka...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Akaros" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akaros+un...@googlegroups.com.
To post to this group, send email to aka...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
~Kevin

[ron minnich]

On Fri, Oct 30, 2015 at 10:51 AM 'Davide Libenzi' via Akaros <aka...@googlegroups.com> wrote:

Isn't it better to leave the kernel emitting JSON, and have a userspace "prettyfier" which could parse the JSON and emit eye-pleasing formats?

you might think so, and I might think so, and that guy over there might think so, but not all people will  think so. It's easy to accommodate those people, so I figure we  might as well.

We can pick a subset of files to make human readable in the native case, where everything else will emit .json? How does that sound?

Anyway, '#cons/vmstats.json' is going to be  ... JSON. :-)

ron 

[ron minnich]

On Fri, Oct 30, 2015 at 10:56 AM Kevin Klues <klu...@gmail.com> wrote:

On Friday, October 30, 2015, 'Davide Libenzi' via Akaros <aka...@googlegroups.com> wrote:

Isn't it better to leave the kernel emitting JSON, and have a userspace "prettyfier" which could parse the JSON and emit eye-pleasing formats?

This would be my vote too. You couldn't just cat it anymore to see the human readable format, but I'm ok with that, personally.

What I did learn with the s-expression-based proc output was that you quickly learned to scan it. People are good at that stuff. 

How about we try the JSON path for now and, at some point, we can figure out if we really need the old school format?

ron 

[Kevin Klues]

I think Json is very human readable to begin with. Kind of what you're saying, I guess.

 

ron 

--
You received this message because you are subscribed to the Google Groups "Akaros" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akaros+un...@googlegroups.com.
To post to this group, send email to aka...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

~Kevin

[Dan Cross]

On Fri, Oct 30, 2015 at 1:56 PM, Kevin Klues <klu...@gmail.com> wrote:

On Friday, October 30, 2015, 'Davide Libenzi' via Akaros <aka...@googlegroups.com> wrote:

Isn't it better to leave the kernel emitting JSON, and have a userspace "prettyfier" which could parse the JSON and emit eye-pleasing formats?

This would be my vote too. You couldn't just cat it anymore to see the human readable format, but I'm ok with that, personally.

I'm scratching my head just a tad at the latter part of this statement; why wouldn't it be human readable?  If I boot akaros right now and run:

/ $ cat /net/arp

ether  OK       10.0.2.15 52:54:00:12:34:56

/ $

The format is pretty easily guessable, but is it any more readable than:

{ dev: "ether", status: "OK", ip: "10.0.2.15", mac: "52:54:00:12:34:56" }

?

Similarly,

/ $ cat '#proc/1/status'

       1 busybox              WAITING         0/ $

/ $

Is that very different, than,

{ pid: 1, name: "busybox", status: "WAITING", numtraps: 0 }

?

[Davide Libenzi]

One thing of JSON is that is not TAB-ified. You *could* TAB-ify it, but that misses the point IMHO.

If you have multiple records, that might not be so easy reading.

Also, you example would be more like:

{ "pid": 1, "name": "busybox", "status": "WAITING", "numtraps": 0 }

Don't get me wrong, I like JSON, but I can see how people might find it not easy to read as son as it starts being not trivial.

IMHO hte kernel should not be in the business of creating pretty output.

That is what userspace is for ☺

--

[Kevin Klues]

On Friday, October 30, 2015, Dan Cross <cro...@gmail.com> wrote:

On Fri, Oct 30, 2015 at 1:56 PM, Kevin Klues <klu...@gmail.com> wrote:

On Friday, October 30, 2015, 'Davide Libenzi' via Akaros <aka...@googlegroups.com> wrote:

Isn't it better to leave the kernel emitting JSON, and have a userspace "prettyfier" which could parse the JSON and emit eye-pleasing formats?

This would be my vote too. You couldn't just cat it anymore to see the human readable format, but I'm ok with that, personally.

I'm scratching my head just a tad at the latter part of this statement; why wouldn't it be human readable?  If I boot akaros right now and run:

Sorry, by 'human readable' here, I meant what Barret was calling human readable (I.e. without all the Json formatting). I do, in fact, think Json is very human readable (if not more so).

 

/ $ cat /net/arp

ether  OK       10.0.2.15 52:54:00:12:34:56

/ $

The format is pretty easily guessable, but is it any more readable than:

{ dev: "ether", status: "OK", ip: "10.0.2.15", mac: "52:54:00:12:34:56" }

?

Similarly,

/ $ cat '#proc/1/status'

       1 busybox              WAITING         0/ $

/ $

Is that very different, than,

{ pid: 1, name: "busybox", status: "WAITING", numtraps: 0 }

?

--

You received this message because you are subscribed to the Google Groups "Akaros" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akaros+un...@googlegroups.com.
To post to this group, send email to aka...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
~Kevin

[Kevin Klues]

On Friday, October 30, 2015, 'Davide Libenzi' via Akaros <aka...@googlegroups.com> wrote:

One thing of JSON is that is not TAB-ified. You *could* TAB-ify it, but that misses the point IMHO.

If you have multiple records, that might not be so easy reading.

Also, you example would be more like:

{ "pid": 1, "name": "busybox", "status": "WAITING", "numtraps": 0 }

Don't get me wrong, I like JSON, but I can see how people might find it not easy to read as son as it starts being not trivial.

IMHO hte kernel should not be in the business of creating pretty output.

That is what userspace is for ☺

To me, the compromise would be to have the kernel always emit Json, and then also prettyify it so it is more human readable.

--
~Kevin

[Davide Libenzi]

That means that the kernel needs to have two functions to maintain now, to emit the same content.

If you instead have the kernel always deal with JSON, and have a userspace script which is the gateway for your pretty viewing of internal status, things are tighter in the kernel, and a userspace tool have much more tooling to create pretty output, WRT the kernel.

The other big advantage for JSON in general is that you can enrich the format (adding new fields) w/out breaking pretty-view readers.

[Kevin Klues]

Fair enough. From user space it would then be: 

cat /proc/whatever | json-prettifier

Instead of just

cat /proc/whatever

On Friday, October 30, 2015, 'Davide Libenzi' via Akaros <aka...@googlegroups.com> wrote:

That means that the kernel needs to have two functions to maintain now, to emit the same content.

If you instead have the kernel always deal with JSON, and have a userspace script which is the gateway for your pretty viewing of internal status, things are tighter in the kernel, and a userspace tool have much more tooling to create pretty output, WRT the kernel.

The other big advantage for JSON in general is that you can enrich the format (adding new fields) w/out breaking pretty-view readers.

I would have it always emit nonformatted Json and then run it through a standard prettifier at the time it is read.

--
~Kevin

[ron minnich]

On Fri, Oct 30, 2015 at 11:49 AM Kevin Klues <klu...@gmail.com> wrote:

I would have it always emit nonformatted Json and then run it through a standard prettifier at the time it is read.

 

and we have aliases/scripts for that, sound ok?

So JSON for everything?

ron 

[Davide Libenzi]

I think the prettyfier could even be smarter than that, and select proper pretty format depending on the kind of status being displayed.

So instead on top of maybe being a pipe-through thing, could be like:

$ sys_show --cpuinfo --vmstatus ...

[Barret Rhoden]

On 2015-10-30 at 11:18 Kevin Klues <klu...@gmail.com> wrote:
> I think Json is very human readable to begin with. Kind of what you're
> saying, I guess.

I personally like the existing format for things like mpstat.
Though some existing things like /net/tcp/0/status are an ugly mess.
Things like the ifstats also have device-specific chunks.

I also like being able to just use grep on raw text, yes, even though
grep is a machine.

I liked ron's suggestion about /status and /status.json. Later we could
have status.proto if we want. Or maybe an fcntl switch even later on.
Part of the niceness of that is that it's clear what file speaks what
protocol. Without some other form of distinction, it's a little odd to
me (which means this is just opinion) that /net/tcp/0/ctl is raw text
and /net/tcp/0/status isn't.

I also like the idea of having clearly defined interfaces. That's
another nice benefit of .protos. And later on, if/when we care about
fast, platform-independent IPCs between processes, protos seem like a
nicer deal that JSON.

[Dan Cross]

On Fri, Oct 30, 2015 at 2:24 PM, 'Davide Libenzi' via Akaros <aka...@googlegroups.com> wrote:

One thing of JSON is that is not TAB-ified. You *could* TAB-ify it, but that misses the point IMHO.

If you have multiple records, that might not be so easy reading.

Also, you example would be more like:

{ "pid": 1, "name": "busybox", "status": "WAITING", "numtraps": 0 }

Most JSON parsers don't *require* you to quote the key field (that is, it doesn't have to be a string). Consider the protobuf text format; it's billed as being compatible with JSON. If that's really a sticking point, then, I'd suggest that we formally use the protobuf text serialization format.

Don't get me wrong, I like JSON, but I can see how people might find it not easy to read as son as it starts being not trivial.

Well, if they find that hard to read, they're really going to have a hard time with the protobuf wire format.

IMHO hte kernel should not be in the business of creating pretty output.

We already are. When we talk about using 'cat' to show us human readable output, that's exactly what we are talking about. Take a look at the output of e.g., 'cat /prof/mpstat':

/ $ cat /prof/mpstat

  CPU:                     irq                      kern                      user                      idle

    0:          0.098385 (  0%),          1.250913 (  2%),          0.009430 (  0%),         44.753029 ( 97%)

    1:          0.000000 (  0%),          0.000016 (100%),          0.000000 (  0%),          0.000000 (  0%)

    2:          0.000000 (  0%),          0.000030 (100%),          0.000000 (  0%),          0.000000 (  0%)

    3:          0.000000 (  0%),          0.000031 (100%),          0.000000 (  0%),          0.000000 (  0%)

    4:          0.000000 (  0%),          0.000019 (100%),          0.000000 (  0%),          0.000000 (  0%)

    5:          0.000000 (  0%),          0.000027 (100%),          0.000000 (  0%),          0.000000 (  0%)

    6:          0.000000 (  0%),          0.000022 (  0%),          0.000000 (  0%),         46.111824 ( 99%)

    7:          0.000000 (  0%),          0.000036 (  0%),          0.000000 (  0%),         46.111810 ( 99%)

/ $

Honestly, I don't find this any harder to read:

{stat: {cpu: 0, irq: {sec: 0.112863, pct: 0}, kern: {sec: 1.390669, pct: 2}, user: {sec: 0.011819. pct: 0}, idle: {sec: 57.512032, pct: 97}},

 stat: {cpu: 1, irq: {sec: 0.0, pct: 0}, kern: {sec: 0.000162, pct: 100}, user: {sec: 0.0, pct: 0}, idle: {sec: 0.0, pct: 0}},

 stat: {cpu: 2, irq: {sec: 0.0, pct: 0}, kern: {sec: 0.000176, pct: 100}, user: {sec: 0.0, pct: 0}, idle: {sec: 0.0, pct: 0}},

 stat: {cpu: 3, irq: {sec: 0.0, pct: 0}, kern: {sec: 0.000156, pct: 100}, user: {sec: 0.0, pct: 0}, idle: {sec: 0.0, pct: 0}},

 stat: {cpu: 4, irq: {sec: 0.0, pct: 0}, kern: {sec: 0.000168, pct: 0}, user: {sec: 0.0, pct: 0}, idle: {sec: 59.027276, pct: 99}},

 stat: {cpu: 5, irq: {sec: 0.0, pct: 0}, kern: {sec: 0.000164, pct: 100}, user: {sec: 0.0, pct: 0}, idle: {sec: 0.0, pct: 0}},

 stat: {cpu: 6: irq: {sec: 0.0, pct: 0}, kern: {sec: 0.000010, pct: 0}, user: {sec: 0.0, pct: 0}, idle: {sec: 59.027266, pct: 99}},

 stat: {cpu: 7: irq: {sec: 0.0, pct: 0}, kern: {sec: 0.000163, pct: 0}, user: {sec: 0.0, pct: 0}, idle: {sec: 59.027305, pct: 99}}}

Notably, however, I showed this to Barret and he disagreed with me about readability. Clearly reasonable people can have different opinions.

Note that that's a proto message.

That is what userspace is for ☺

I don't disagree. https://github.com/dancrossnyc/pbpp

However, I have somewhat more concern about binary formats. 45 years ago the Unix people figured out that text formats are useful. All I'm saying is that we shouldn't be so quick to discard that history. Also, the protobuf wire format is non-trivial; it's not easy to create an 'ad hoc' formatter for it, and it implies that we take these data structures and turn them into proto messages; that is, they wouldn't sit next to header files; they generated code *would* be the header file.

        - Dan C.

[Davide Libenzi]

On Fri, Oct 30, 2015 at 12:09 PM, Dan Cross <cro...@gmail.com> wrote:

On Fri, Oct 30, 2015 at 2:24 PM, 'Davide Libenzi' via Akaros <aka...@googlegroups.com> wrote:

One thing of JSON is that is not TAB-ified. You *could* TAB-ify it, but that misses the point IMHO.

If you have multiple records, that might not be so easy reading.

Also, you example would be more like:

{ "pid": 1, "name": "busybox", "status": "WAITING", "numtraps": 0 }

Most JSON parsers don't *require* you to quote the key field (that is, it doesn't have to be a string). Consider the protobuf text format; it's billed as being compatible with JSON. If that's really a sticking point, then, I'd suggest that we formally use the protobuf text serialization format.

No doubt protobuf uses a different format, but the topic here was JSON, for which the "" are mandatory.

The protobuf is more readable IMO, but JSON is more a standard.

[Barret Rhoden]

On 2015-10-30 at 14:19 Dan Cross <cro...@gmail.com> wrote:
> / $ cat /net/arp
> ether OK 10.0.2.15 52:54:00:12:34:56
> / $
>
> The format is pretty easily guessable, but is it any more readable
> than:
>
> { dev: "ether", status: "OK", ip: "10.0.2.15", mac:
> "52:54:00:12:34:56" }

I hope it doesn't wrap in my Akaros console, otherwise the ARP table
will look like a mess if there is more than one entry. =)

We've talked about mpstat, and I wasn't convinced that your JSON for
that was nicer than the existing output. But whatever, me and Drew
were the only ones to use it.

> Similarly,
>
> / $ cat '#proc/1/status'
> 1 busybox WAITING 0/ $
> / $
>
> Is that very different, than,
>
> { pid: 1, name: "busybox", status: "WAITING", numtraps: 0 }

I guess we could make that JSON, then have the shell script 'ps'
convert it back.

btw, s/numtraps/ppid/

My main thing is that I use tools like cat all the time from the
console for all sorts of stuff. That was one of the benefits that we
were sold when we brought in Plan 9's interfaces. I'd rather not lose
that or make it unduly tedious on me.

But, as I've got too much other stuff going on, I'm also not the one
who's going to code it up. I've said my piece on this, and I'll accept
code that does JSON or whatever for the various new status files.

So in summary:
- I'd like raw, console based, text output to work, at least for
existing interfaces
- I'd like clearly defined interfaces, especially if a machine is
processing the output (e.g. .protos in k/i/r/protos or something)
- Don't rule out protos for our long range IPC mechanisms (including
the kernel to user)
- I'm okay with JSON for now.

Barret

[Dan Cross]

As I said, proto's text format bills itself as being compatible with JSON.

The protobuf is more readable IMO, but JSON is more a standard.

[Kevin Klues]

Off topic, but this is exactly how I feel about the code review process at the moment. It's so tedious that I mostly don't bother to contribute.

 

But, as I've got too much other stuff going on, I'm also not the one
who's going to code it up.  I've said my piece on this, and I'll accept
code that does JSON or whatever for the various new status files.

So in summary:
- I'd like raw, console based, text output to work, at least for
  existing interfaces
- I'd like clearly defined interfaces, especially if a machine is
  processing the output (e.g. .protos in k/i/r/protos or something)
- Don't rule out protos for our long range IPC mechanisms (including
  the kernel to user)
- I'm okay with JSON for now.

Barret

--
You received this message because you are subscribed to the Google Groups "Akaros" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akaros+un...@googlegroups.com.
To post to this group, send email to aka...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
~Kevin

[ron minnich]

you mean like a comment series that goes on for, oh, say, 35 patches and days: 

http://review.coreboot.org/#/c/11966/

Comes with the territory :-)

ron

[Kevin Klues]

On Friday, October 30, 2015, ron minnich <rmin...@gmail.com> wrote:

you mean like a comment series that goes on for, oh, say, 35 patches and days: 

http://review.coreboot.org/#/c/11966/

Comes with the territory :-)

I don't mind doing code reviews (I actually enjoy them when things are coming together). Our current process over email just hinders me from doing them effectively so I mostly don't bother commenting.