Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Latest compiler and/or Boost.Variant issue with forward declarations
102 views
Skip to first unread message
Michael Powell
Jan 10, 2019, 1:39:52 PM1/10/19
to
Hello,
I've got an AST I am developing to support JSON parsing using the latest VC++ (15.9.5), however, I am having difficulty persuading the compiler with my forward declarations.
i.e. 1>d:\dev\boost.org\boost_1_69_0\boost\mpl\sizeof.hpp(27): error C2027: use of undefined type 'kingdom::json::object_t'
In terms of JSON AST modeling, the rub appears to be in a couple of places: defining the JSON AST Array, and making the JSON AST Object available to Value. Re: semantic terminology, Value is interchangeable with Element, in view of the JSON grammar.
Here's my attempt at a flattened single source example:
https://wandbox.org/permlink/83c3VXZ4W1DHoEBc
I seem to recall there being future issues with C++ forward declarations, but this seems to be like a bad language design decision to restrict this, especially in light of this kind of approach.
I'm not sure how better to sort this out. At minimum, at least Value needs a forward declaration depending on how I arrange my headers, includes, etc. But then I still run into Value forward declaration undefined type issues.
I'm not sure it's as much a language issue as much as it is possible a Boost.Variant issue, per se.
Suggestions?
Thanks!
Michael Powell
I've got an AST I am developing to support JSON parsing using the latest VC++ (15.9.5), however, I am having difficulty persuading the compiler with my forward declarations.
i.e. 1>d:\dev\boost.org\boost_1_69_0\boost\mpl\sizeof.hpp(27): error C2027: use of undefined type 'kingdom::json::object_t'
In terms of JSON AST modeling, the rub appears to be in a couple of places: defining the JSON AST Array, and making the JSON AST Object available to Value. Re: semantic terminology, Value is interchangeable with Element, in view of the JSON grammar.
Here's my attempt at a flattened single source example:
https://wandbox.org/permlink/83c3VXZ4W1DHoEBc
I seem to recall there being future issues with C++ forward declarations, but this seems to be like a bad language design decision to restrict this, especially in light of this kind of approach.
I'm not sure how better to sort this out. At minimum, at least Value needs a forward declaration depending on how I arrange my headers, includes, etc. But then I still run into Value forward declaration undefined type issues.
I'm not sure it's as much a language issue as much as it is possible a Boost.Variant issue, per se.
Suggestions?
Thanks!
Michael Powell
Mr Flibble
Jan 10, 2019, 1:44:00 PM1/10/19
to
you writing your own? Even I have written one which is implemented in
terms of std::variant:
https://github.com/i42output/neolib/blob/master/include/neolib/json.hpp
/Flibble
--
“You won’t burn in hell. But be nice anyway.” – Ricky Gervais
“I see Atheists are fighting and killing each other again, over who
doesn’t believe in any God the most. Oh, no..wait.. that never happens.” –
Ricky Gervais
"Suppose it's all true, and you walk up to the pearly gates, and are
confronted by God," Bryne asked on his show The Meaning of Life. "What
will Stephen Fry say to him, her, or it?"
"I'd say, bone cancer in children? What's that about?" Fry replied.
"How dare you? How dare you create a world to which there is such misery
that is not our fault. It's not right, it's utterly, utterly evil."
"Why should I respect a capricious, mean-minded, stupid God who creates a
world that is so full of injustice and pain. That's what I would say."
Michael Powell
Jan 10, 2019, 2:16:42 PM1/10/19
to
On Thursday, January 10, 2019 at 1:44:00 PM UTC-5, Mr Flibble wrote:
> On 10/01/2019 18:39, Michael Powell wrote:
> > Hello,
> >
> > I've got an AST I am developing to support JSON parsing using the latest VC++ (15.9.5), however, I am having difficulty persuading the compiler with my forward declarations.
> >
> > i.e. 1>d:\dev\boost.org\boost_1_69_0\boost\mpl\sizeof.hpp(27): error C2027: use of undefined type 'kingdom::json::object_t'
> >
> > In terms of JSON AST modeling, the rub appears to be in a couple of places: defining the JSON AST Array, and making the JSON AST Object available to Value. Re: semantic terminology, Value is interchangeable with Element, in view of the JSON grammar.
> >
> > Here's my attempt at a flattened single source example:
> >
> > https://wandbox.org/permlink/83c3VXZ4W1DHoEBc
> >
> > I seem to recall there being future issues with C++ forward declarations, but this seems to be like a bad language design decision to restrict this, especially in light of this kind of approach.
> >
> > I'm not sure how better to sort this out. At minimum, at least Value needs a forward declaration depending on how I arrange my headers, includes, etc. But then I still run into Value forward declaration undefined type issues.
> >
> > I'm not sure it's as much a language issue as much as it is possible a Boost.Variant issue, per se.
> >
> > Suggestions?
>
> There are plenty of pre-existing C++ JSON libraries out there so why are
> you writing your own? Even I have written one which is implemented in
> terms of std::variant:
>
> https://github.com/i42output/neolib/blob/master/include/neolib/json.hpp
That's an interesting approach. A lot of type definitions going on there, including the self_type, basically the recursive "value type" use case.
> On 10/01/2019 18:39, Michael Powell wrote:
> > Hello,
> >
> > I've got an AST I am developing to support JSON parsing using the latest VC++ (15.9.5), however, I am having difficulty persuading the compiler with my forward declarations.
> >
> > i.e. 1>d:\dev\boost.org\boost_1_69_0\boost\mpl\sizeof.hpp(27): error C2027: use of undefined type 'kingdom::json::object_t'
> >
> > In terms of JSON AST modeling, the rub appears to be in a couple of places: defining the JSON AST Array, and making the JSON AST Object available to Value. Re: semantic terminology, Value is interchangeable with Element, in view of the JSON grammar.
> >
> > Here's my attempt at a flattened single source example:
> >
> > https://wandbox.org/permlink/83c3VXZ4W1DHoEBc
> >
> > I seem to recall there being future issues with C++ forward declarations, but this seems to be like a bad language design decision to restrict this, especially in light of this kind of approach.
> >
> > I'm not sure how better to sort this out. At minimum, at least Value needs a forward declaration depending on how I arrange my headers, includes, etc. But then I still run into Value forward declaration undefined type issues.
> >
> > I'm not sure it's as much a language issue as much as it is possible a Boost.Variant issue, per se.
> >
> > Suggestions?
>
> There are plenty of pre-existing C++ JSON libraries out there so why are
> you writing your own? Even I have written one which is implemented in
> terms of std::variant:
>
> https://github.com/i42output/neolib/blob/master/include/neolib/json.hpp
I did consider std::variant at one point on another project, however boost::variant is a requirement of sorts since I will be turning that around in a Boost Spirit Qi grammar.
Appreciate the feedback, though.
james...@alumni.caltech.edu
Jan 10, 2019, 4:59:58 PM1/10/19
to
On Thursday, January 10, 2019 at 1:39:52 PM UTC-5, Michael Powell wrote:
> Hello,
>
> I've got an AST I am developing to support JSON parsing using the latest VC++ (15.9.5), however, I am having difficulty persuading the compiler with my forward declarations.
>
> i.e. 1>d:\dev\boost.org\boost_1_69_0\boost\mpl\sizeof.hpp(27): error C2027: use of undefined type 'kingdom::json::object_t'
>
> In terms of JSON AST modeling, the rub appears to be in a couple of places: defining the JSON AST Array, and making the JSON AST Object available to Value. Re: semantic terminology, Value is interchangeable with Element, in view of the JSON grammar.
>
> Here's my attempt at a flattened single source example:
>
> https://wandbox.org/permlink/83c3VXZ4W1DHoEBc
Your example contains no instances of the identifier "kingdom", so it
> Hello,
>
> I've got an AST I am developing to support JSON parsing using the latest VC++ (15.9.5), however, I am having difficulty persuading the compiler with my forward declarations.
>
> i.e. 1>d:\dev\boost.org\boost_1_69_0\boost\mpl\sizeof.hpp(27): error C2027: use of undefined type 'kingdom::json::object_t'
>
> In terms of JSON AST modeling, the rub appears to be in a couple of places: defining the JSON AST Array, and making the JSON AST Object available to Value. Re: semantic terminology, Value is interchangeable with Element, in view of the JSON grammar.
>
> Here's my attempt at a flattened single source example:
>
> https://wandbox.org/permlink/83c3VXZ4W1DHoEBc
cannot be the actual source code that causes that error message shown
above. Can you provide code, as short and as simple as possible, that
does generate that message when compiled? It would be much easier to
diagnose your problem if you could provide a proper example of it.
The bottom of that web page contains error messages generated from your
example code. Taking the first message, it points out that on line 100,
your code refers to other.sign, when the relevant type has no member
named "sign". Could you explain why you thought that it should have a
member named "sign"?
Michael Powell
Jan 10, 2019, 5:39:55 PM1/10/19
to
On Thursday, January 10, 2019 at 4:59:58 PM UTC-5, james...@alumni.caltech.edu wrote:
> On Thursday, January 10, 2019 at 1:39:52 PM UTC-5, Michael Powell wrote:
> > Hello,
> >
> > I've got an AST I am developing to support JSON parsing using the latest VC++ (15.9.5), however, I am having difficulty persuading the compiler with my forward declarations.
> >
> > i.e. 1>d:\dev\boost.org\boost_1_69_0\boost\mpl\sizeof.hpp(27): error C2027: use of undefined type 'kingdom::json::object_t'
> >
> > In terms of JSON AST modeling, the rub appears to be in a couple of places: defining the JSON AST Array, and making the JSON AST Object available to Value. Re: semantic terminology, Value is interchangeable with Element, in view of the JSON grammar.
> >
> > Here's my attempt at a flattened single source example:
> >
> > https://wandbox.org/permlink/83c3VXZ4W1DHoEBc
>
> Your example contains no instances of the identifier "kingdom", so it
> cannot be the actual source code that causes that error message shown
> above. Can you provide code, as short and as simple as possible, that
> does generate that message when compiled? It would be much easier to
> diagnose your problem if you could provide a proper example of it.
That was the error I received under local build. The Wandbox example is an attempt at a MRE.
> On Thursday, January 10, 2019 at 1:39:52 PM UTC-5, Michael Powell wrote:
> > Hello,
> >
> > I've got an AST I am developing to support JSON parsing using the latest VC++ (15.9.5), however, I am having difficulty persuading the compiler with my forward declarations.
> >
> > i.e. 1>d:\dev\boost.org\boost_1_69_0\boost\mpl\sizeof.hpp(27): error C2027: use of undefined type 'kingdom::json::object_t'
> >
> > In terms of JSON AST modeling, the rub appears to be in a couple of places: defining the JSON AST Array, and making the JSON AST Object available to Value. Re: semantic terminology, Value is interchangeable with Element, in view of the JSON grammar.
> >
> > Here's my attempt at a flattened single source example:
> >
> > https://wandbox.org/permlink/83c3VXZ4W1DHoEBc
>
> Your example contains no instances of the identifier "kingdom", so it
> cannot be the actual source code that causes that error message shown
> above. Can you provide code, as short and as simple as possible, that
> does generate that message when compiled? It would be much easier to
> diagnose your problem if you could provide a proper example of it.
> The bottom of that web page contains error messages generated from your
> example code. Taking the first message, it points out that on line 100,
> your code refers to other.sign, when the relevant type has no member
> named "sign". Could you explain why you thought that it should have a
> member named "sign"?
https://wandbox.org/permlink/EZ4DtRXFlOm20Ojs
I've worked through some of this in my local project, sort of, and now landing in Boost Spirit Qi territory, however, what I am finding is that the forward declarations are a problem with it comes to leveraging Boost.Variant. The best I could come up with was to use loosely joined Tuples, Vectors of Tuples, etc, when it comes to modeling the JSON Object and so forth. It seems a bit clunky to me, but it will probably work.
Concerning Sign itself, yes, I am planning to support an enhanced parsing of the JSON specification that includes, potentially, floating point, NaN, [+/-]Infinity, and so on. That is, unless there is something about natural C++ language support for Double Infinity that I'm unaware of.
Cheers.
Michael Powell
Jan 10, 2019, 6:06:49 PM1/10/19
to
https://wandbox.org/permlink/P9Xel2Oiiy0O8Ygr
Which I confirmed in local build. The issues I am visiting now are to do with Boost Spirit Qi relating to "incompatible Skippers".
However, it seems like to me forward declarations are a problem, but I don't have a ready answer as to why.
James Kuyper
Jan 11, 2019, 9:52:10 AM1/11/19
to
On 1/10/19 18:06, Michael Powell wrote:
...
what your comment even means.
> However, it seems like to me forward declarations are a problem, but I don't have a ready answer as to why.
What gives you the impression that forward declarations might be a
problem? Be specific: complete compilable code examples would be best.
...
> Best I could come up with was something like this, it builds and runs anyway.
>
> https://wandbox.org/permlink/P9Xel2Oiiy0O8Ygr
>
> Which I confirmed in local build. The issues I am visiting now are to do with Boost Spirit Qi relating to "incompatible Skippers".
That would be best pursued in a forum devoted to Boost. I have no idea
>
> https://wandbox.org/permlink/P9Xel2Oiiy0O8Ygr
>
> Which I confirmed in local build. The issues I am visiting now are to do with Boost Spirit Qi relating to "incompatible Skippers".
what your comment even means.
> However, it seems like to me forward declarations are a problem, but I don't have a ready answer as to why.
problem? Be specific: complete compilable code examples would be best.
Öö Tiib
Jan 11, 2019, 10:09:19 AM1/11/19
to
I have read JSON to be specified like that (putting terms
into double quotes):
* "object" is an unordered set of "name" / "value" pairs (that can
be empty set).
* "array" is an ordered collection of "values" (that can be empty
array).
* "name" is "string" uniquely identifying "value" in "object".
* "value" is one of "string", "number", "true", "false" or "null".
* "string" is ... and so on.
Looking at your code "object" seems missing. Why?
Usually when receiving JSON then it is UTF-8 file
consisting of one, root object. I would expect it
to be something like:
using object = std::unordered set<name,value>;
Other things are renamed to "str_t", "key_t",
"numeric_t" why to rename those? Also the "_t"
feels pointless cargo-cult waste of two characters.
It gives no benefits, makes it just harder and more
confusing to communicate about the thing.
Michael Powell
Jan 11, 2019, 11:00:04 AM1/11/19
to
What's your source for that?
> Looking at your code "object" seems missing. Why?
> Usually when receiving JSON then it is UTF-8 file
> consisting of one, root object. I would expect it
> to be something like:
using object_t = std::vector<boost::tuple<key_t, value_t>>;
The awkward part is that value_t needs to be a fully declared type at the moment Boost.Variant finds it in the template specialization. I have, however, taken to declaring the type alias immediately thereafter so that at least subscribers to value_t see it as the expected object_t.
> using object = std::unordered set<name,value>;
> Other things are renamed to "str_t", "key_t",
> "numeric_t" why to rename those? Also the "_t"
> feels pointless cargo-cult waste of two characters.
> It gives no benefits, makes it just harder and more
> confusing to communicate about the thing.
Best regards,
Michael Powell
Michael Powell
Jan 11, 2019, 11:48:52 AM1/11/19
to
On Friday, January 11, 2019 at 10:09:19 AM UTC-5, Öö Tiib wrote:
The problem is that the JSON specification expects that Value may map itself as the value of the Object Members, however that gets modeled.
1>c:\program files (x86)\microsoft visual studio\2017\enterprise\vc\tools\msvc\14.16.27023\include\type_traits(719): error C2139: 'kingdom::json::value_t': an undefined class is not allowed as an argument to compiler intrinsic type trait '__is_empty'
1>path\to\src\kingdom.json.parser\value.hpp(41): note: see declaration of 'kingdom::json::value_t'
1>c:\program files (x86)\microsoft visual studio\2017\enterprise\vc\tools\msvc\14.16.27023\include\xutility(264): note: see reference to variable template 'const bool is_empty_v<kingdom::json::value_t>' being compiled
1>c:\program files (x86)\microsoft visual studio\2017\enterprise\vc\tools\msvc\14.16.27023\include\unordered_set(27): note: see reference to class template instantiation 'std::_Uhash_compare<_Kty,_Hasher,_Keyeq>' being compiled
1> with
1> [
1> _Kty=kingdom::json::str_t,
1> _Hasher=kingdom::json::value_t,
1> _Keyeq=std::equal_to<kingdom::json::str_t>
1> ]
Intended usage...
struct value_t : boost::variant<
null_t
, boolean_t
, str_t
, floating_point_number_t
, integer_number_t
, std::unordered_set<str_t, value_t>
// ^^^^^^^
, std::vector<value_t>
// ^^^^^^^ interestingly, this is fine...
> {
// ...
};
Perhaps an Unordered Map, Multimap, etc, are better? Suggestions?
Michael Powell
Jan 11, 2019, 11:53:46 AM1/11/19
to
Daniel
Jan 11, 2019, 1:04:14 PM1/11/19
to
On Thursday, January 10, 2019 at 2:16:42 PM UTC-5, Michael Powell wrote:
>
> boost::variant is a requirement of sorts since I will be turning that
> around in a Boost Spirit Qi grammar.
>
You might want to compare with John W. Wilkinson's JSON parser based on Boost Spirit, https://www.codeproject.com/Articles/20027/%2FArticles%2F20027%2FJSON-
>
> boost::variant is a requirement of sorts since I will be turning that
> around in a Boost Spirit Qi grammar.
>
Spirit-A-C-JSON-Parser-Generator-Implemented. This does work, and at one time had a significant user base. It doesn't seem to be currently supported though, not since 2014.
I wouldn't personally recommend basing a JSON parser on Boost Spirit, as it
would be very slow. You can see some of my own benchmarks on Windows 10
https://github.com/danielaparker/json_benchmarks/blob/master/report/performance.md
I seem to recall Milo Yip's benchmarks https://github.com/miloyip/nativejson-benchmark also showed JSON Spirit to
be very slow, but I don't see it on the latest list.
Daniel
Michael Powell
Jan 11, 2019, 1:26:42 PM1/11/19
to
> Daniel
Richard
Jan 11, 2019, 4:43:49 PM1/11/19
to
[Please do not mail me a copy of your followup]
Daniel <daniel...@gmail.com> spake the secret code
<99fbc5f1-3648-4528...@googlegroups.com> thusly:
>I wouldn't personally recommend basing a JSON parser on Boost Spirit, as it
>would be very slow. You can see some of my own benchmarks on Windows 10
>
>https://github.com/danielaparker/json_benchmarks/blob/master/report/performance.md
Has anyone done any root cause analysis as to why the spirit
implementation is so slow? Lots of spirit parsing has been very fast,
e.g. faster than atoi.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>
Daniel <daniel...@gmail.com> spake the secret code
<99fbc5f1-3648-4528...@googlegroups.com> thusly:
>I wouldn't personally recommend basing a JSON parser on Boost Spirit, as it
>would be very slow. You can see some of my own benchmarks on Windows 10
>
>https://github.com/danielaparker/json_benchmarks/blob/master/report/performance.md
implementation is so slow? Lots of spirit parsing has been very fast,
e.g. faster than atoi.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>
Daniel
Jan 11, 2019, 6:03:54 PM1/11/19
to
On Friday, January 11, 2019 at 4:43:49 PM UTC-5, Richard wrote:
>
> Has anyone done any root cause analysis as to why the spirit
> implementation is so slow? Lots of spirit parsing has been very fast,
> e.g. faster than atoi.
Well, parsing 900MB of JSON text (my benchmark) isn't totally analogous to
>
> Has anyone done any root cause analysis as to why the spirit
> implementation is so slow? Lots of spirit parsing has been very fast,
> e.g. faster than atoi.
what atoi does. Maybe this stackoverflow item has some applicability:
https://stackoverflow.com/questions/13343874/boost-spirit-qi-slow
It suggests that "the performance bottleneck is the string handling of Spirit qi. All other data types seem to be quite fast."
I found an earlier tag of Milo Yip's benchmarks which has JSON Spirit,
https://github.com/miloyip/nativejson-benchmark/tree/v1.0.0
with smaller benchmark files, between 600 and 1200KB. JSON Spirit's relative performance is a little bit better on those, but not by much, e.g. 8 x slower than cJSON vs 12 times slower.
Daniel
Öö Tiib
Jan 14, 2019, 7:39:30 AM1/14/19
to
wordage that you used? My source is every
specification that I have read. Whatever language
or interface that supports JSON uses Name (not
Key), String (not Str) and Number (not Numeric).
Look at json.org it also uses those:
https://www.json.org/
It is difficult to discuss stuff when terminology is
pointlessly skewed by some part.
> > Looking at your code "object" seems missing. Why?
> > Usually when receiving JSON then it is UTF-8 file
> > consisting of one, root object. I would expect it
> > to be something like:
>
> At least at the moment, Object is represented by:
>
> using object_t = std::vector<boost::tuple<key_t, value_t>>;
> The awkward part is that value_t needs to be a fully declared type at the moment Boost.Variant finds it in the template specialization. I have, however, taken to declaring the type alias immediately thereafter so that at least subscribers to value_t see it as the expected object_t.
That means we have to use another layer of
indirection at spots of breaking the recursion.
With boost::variant there is possibility to make
recursive variants. Unfortunately these are tricky
to make compile on some compilers and then
heap-allocate under the hood.
I would instead go with std::variant and
std::unique_ptr to not fully defined types at
spots of breaking recursion. It is simple enough to
use first at spots chosen by "best readability".
Later the spots are easier to change and also
management of owned object of unique_ptr is
simpler to customize for to optimize. That "later"
in sense that it is performance optimization ... for
time when the thing works and so there is point
to optimize it at all.
I am in doubt about Boost.Spirit ... IOW why it?
When I tried it then it compiled awkwardly slowly
and the compiling diagnostics were horrible when
it failed to compile. Parse-time error handling was
quite inconvenient to hook into it. Resulting parser
performed about 3 times worse than equivalent
parser made by other parser generator. That all
made it feel like lose-lose-lose choice. It was
about 4 or 5 years ago, maybe spirit has improved
in some aspects since then but I doubt it.
>
> > using object = std::unordered set<name,value>;
>
> It's an excellent point in and of itself. I was unaware of Unordered Set, per se; that would be preferable, I think; it would certainly be more concise than Vector of Member Tuples. The main issue there would be whether I could synthesize its elements or otherwise emplace them during parsing. As others mentioned, yes, more of a question for Boost and/or Boost.Spirit forums.
unordered_map<name, value>. Its interface is
most convenient. For example value v to name n
in object o can be set just with "o[n] = v;".
The o[n] returns reference to value or default-
constructed value when it didn't exist.
It is likely that vector of pairs is performing better
when objects have only few members and
unorded_map is better when objects have lot of
members but making such choices or using
both (as variants) is again about performance,
so leave for later. Choose by convenience of
interface for task first and only when all tests
run then tinker with performance.
Öö Tiib
Jan 14, 2019, 8:35:32 AM1/14/19
to
that names must be unique within object.
The standard:
http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf
does not require those be unique:
| The JSON syntax does not impose any restrictions on the
| strings used as names, does not require that name strings
| be unique, and does not assign any significance to the
| ordering of name/value pairs.
However most parsers silently (and few noisily) fail when
object has several members with same name. I would go
with a map and would throw on multiple members with
same name. One should use array member instead of
multiple members with same name.
> I think you probably meant std::map, or at least std::unordered_map. That would be a closer implementation detail to the object member associative array.
Michael Powell
Jan 14, 2019, 11:03:29 AM1/14/19
to
> > > Looking at your code "object" seems missing. Why?
> > > Usually when receiving JSON then it is UTF-8 file
> > > consisting of one, root object. I would expect it
> > > to be something like:
> >
> > At least at the moment, Object is represented by:
> >
> > using object_t = std::vector<boost::tuple<key_t, value_t>>;
>
> Ok. It wasn't when I looked at your code.
> > The awkward part is that value_t needs to be a fully declared type at the moment Boost.Variant finds it in the template specialization. I have, however, taken to declaring the type alias immediately thereafter so that at least subscribers to value_t see it as the expected object_t.
>
> Yes, the whole idea of JSON is deeply recursive.
> That means we have to use another layer of
> indirection at spots of breaking the recursion.
> With boost::variant there is possibility to make
> recursive variants. Unfortunately these are tricky
> to make compile on some compilers and then
> heap-allocate under the hood.
>
> I would instead go with std::variant and
> std::unique_ptr to not fully defined types at
> spots of breaking recursion. It is simple enough to
> use first at spots chosen by "best readability".
> Later the spots are easier to change and also
> management of owned object of unique_ptr is
> simpler to customize for to optimize. That "later"
> in sense that it is performance optimization ... for
> time when the thing works and so there is point
> to optimize it at all.
> I am in doubt about Boost.Spirit ... IOW why it?
> When I tried it then it compiled awkwardly slowly
> and the compiling diagnostics were horrible when
> it failed to compile. Parse-time error handling was
> quite inconvenient to hook into it. Resulting parser
> performed about 3 times worse than equivalent
> parser made by other parser generator. That all
> made it feel like lose-lose-lose choice. It was
> about 4 or 5 years ago, maybe spirit has improved
> in some aspects since then but I doubt it.
> >
> > > using object = std::unordered set<name,value>;
> >
> > It's an excellent point in and of itself. I was unaware of Unordered Set, per se; that would be preferable, I think; it would certainly be more concise than Vector of Member Tuples. The main issue there would be whether I could synthesize its elements or otherwise emplace them during parsing. As others mentioned, yes, more of a question for Boost and/or Boost.Spirit forums.
>
> Hmm. Yes, My mistake ... actually it should be
> unordered_map<name, value>. Its interface is
> most convenient. For example value v to name n
> in object o can be set just with "o[n] = v;".
> The o[n] returns reference to value or default-
> constructed value when it didn't exist.
> It is likely that vector of pairs is performing better
> when objects have only few members and
> unorded_map is better when objects have lot of
> members but making such choices or using
> both (as variants) is again about performance,
> so leave for later. Choose by convenience of
> interface for task first and only when all tests
> run then tinker with performance.
>
> >
> > > Other things are renamed to "str_t", "key_t",
> > > "numeric_t" why to rename those? Also the "_t"
> > > feels pointless cargo-cult waste of two characters.
> > > It gives no benefits, makes it just harder and more
> > > confusing to communicate about the thing.
> >
> > Thanks for the feedback.
> >
> > Best regards,
> >
> > Michael Powell
Cheers,
Michael Powell
Michael Powell
Jan 14, 2019, 11:06:11 AM1/14/19
to
On Monday, January 14, 2019 at 8:35:32 AM UTC-5, Öö Tiib wrote:
> On Friday, 11 January 2019 18:53:46 UTC+2, Michael Powell wrote:
> > On Friday, January 11, 2019 at 11:48:52 AM UTC-5, Michael Powell wrote:
> > > On Friday, January 11, 2019 at 10:09:19 AM UTC-5, Öö Tiib wrote:
> > >
> > > Perhaps an Unordered Map, Multimap, etc, are better? Suggestions?
>
> Use multimap if you want to parse without (typical) constraint
> that names must be unique within object.
>
> The standard:
> http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf
> does not require those be unique:
>
> | The JSON syntax does not impose any restrictions on the
> | strings used as names, does not require that name strings
> | be unique, and does not assign any significance to the
> | ordering of name/value pairs.
Fair enough. I've actually read further that "uniqueness" is more of an implementation detail, but the spec itself does not restrict that aspect.
> On Friday, 11 January 2019 18:53:46 UTC+2, Michael Powell wrote:
> > On Friday, January 11, 2019 at 11:48:52 AM UTC-5, Michael Powell wrote:
> > > On Friday, January 11, 2019 at 10:09:19 AM UTC-5, Öö Tiib wrote:
> > >
> > > Perhaps an Unordered Map, Multimap, etc, are better? Suggestions?
>
> Use multimap if you want to parse without (typical) constraint
> that names must be unique within object.
>
> The standard:
> http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf
> does not require those be unique:
>
> | The JSON syntax does not impose any restrictions on the
> | strings used as names, does not require that name strings
> | be unique, and does not assign any significance to the
> | ordering of name/value pairs.
> However most parsers silently (and few noisily) fail when
> object has several members with same name. I would go
> with a map and would throw on multiple members with
> same name. One should use array member instead of
> multiple members with same name.
> > I think you probably meant std::map, or at least std::unordered_map. That would be a closer implementation detail to the object member associative array.
>
> Yes, my mistake, meant std::unordered_map.
Cheers,
Michael Powell
Michael Powell
Jan 14, 2019, 12:40:49 PM1/14/19
to
On Monday, January 14, 2019 at 11:06:11 AM UTC-5, Michael Powell wrote:
One thing I am finding with the types being so raw and "closer to the metal", so to speak, is just now natural it feels to build out JSON examples, especially for the variants, their automatic conversions, etc.
For instance:
struct value_t : boost::variant<
null_t
, boolean_t
, str_t
, floating_point_number_t
, integer_number_t
, std::map<str_t, value_t>
, std::vector<value_t>
> {
using object_type = std::map<str_t, value_t>;
using member_type = std::pair<str_t, value_t>;
using array_type = std::vector<value_t>;
using base_type = boost::variant<
, array_type
>;
// ...
};
using object_t = value_t::object_type;
using member_t = value_t::member_type;
using array_t = value_t::array_type;
Some example snippets, all of which auto-magically convert to the value_t, plus with the advent of initializer lists and so forth, just feels very natural at a C++ language level IMO:
null_value
// Yes, I could support the language level bool, true, false, but for parsing purposes I chose this for the time being.
boolean_true
boolean_false
array_t{}
array_t{ null_value, null_value }
array_t{ boolean_false, boolean_true }
object_t{}
These are just one level distilled from my unit tests, and I could extend that out any number of levels.
One thing I am finding with the types being so raw and "closer to the metal", so to speak, is just now natural it feels to build out JSON examples, especially for the variants, their automatic conversions, etc.
For instance:
struct value_t : boost::variant<
null_t
, boolean_t
, str_t
, floating_point_number_t
, integer_number_t
, std::vector<value_t>
> {
using object_type = std::map<str_t, value_t>;
using member_type = std::pair<str_t, value_t>;
using array_type = std::vector<value_t>;
using base_type = boost::variant<
null_t
, boolean_t
, str_t
, floating_point_number_t
, integer_number_t
, object_type
, boolean_t
, str_t
, floating_point_number_t
, integer_number_t
, array_type
>;
// ...
};
using object_t = value_t::object_type;
using member_t = value_t::member_type;
using array_t = value_t::array_type;
Some example snippets, all of which auto-magically convert to the value_t, plus with the advent of initializer lists and so forth, just feels very natural at a C++ language level IMO:
null_value
// Yes, I could support the language level bool, true, false, but for parsing purposes I chose this for the time being.
boolean_true
boolean_false
array_t{}
array_t{ null_value, null_value }
array_t{ boolean_false, boolean_true }
object_t{}
These are just one level distilled from my unit tests, and I could extend that out any number of levels.
Öö Tiib
Jan 15, 2019, 9:54:51 AM1/15/19
to
am experienced programmer ... I read code quickly.
But I am not parapsychological super-human so can't
guess missing parts. For example like that:
#include <iostream>
#include <map>
#include <memory>
#include <string>
#include <variant>
#include <vector>
namespace json
{
using Null = std::monostate;
using True = std::true_type;
using False = std::false_type;
using Number = double;
using String = std::string;
struct Value;
using ValuePtr = std::unique_ptr<Value>;
using Array = std::vector<ValuePtr>;
using Name = String;
using Object = std::map<Name, ValuePtr>;
struct Value : std::variant<Null, True, False, Number, String, Array, Object> {};
}
int main()
{
std::cout << "Sketch done 26 lines!\n";
}
It has everything what JSON has and if to copy-paste it then it compiles
and runs on C++17 compiler. Then it can be discussed if, how, when and
why to rearrange or restructure it and to turn these using/typedefs into
real classes.
>
> Some example snippets, all of which auto-magically convert to the value_t, plus with the advent of initializer lists and so forth, just feels very natural at a C++ language level IMO:
>
> null_value
> // Yes, I could support the language level bool, true, false, but for parsing purposes I chose this for the time being.
> boolean_true
> boolean_false
> array_t{}
> array_t{ null_value, null_value }
> array_t{ boolean_false, boolean_true }
> object_t{}
C++ code.
> These are just one level distilled from my unit tests, and I could extend that out any number of levels.
does it mean.
Michael Powell
Jan 15, 2019, 12:29:32 PM1/15/19
to
> > These are just one level distilled from my unit tests, and I could extend that out any number of levels.
>
> I don't understand what is "one level distilled" and what
> does it mean.
james...@alumni.caltech.edu
Jan 15, 2019, 1:20:44 PM1/15/19
to
On Tuesday, January 15, 2019 at 12:29:32 PM UTC-5, Michael Powell wrote:
...
example doesn't provide enough information to allow people to provide
that help. This is a very common, and very frustrating situation. In my
experience, people who don't understand why they're running into
problems, and select the part of their code that they think is relevant,
are virtually certain to choose a part of their code that doesn't
include the actual problem. You are NOT a counter-example.
One of the error messages you provided, for instance, says "'kingdom::json::value_t': an undefined class is not allowed as an
kingdom::json, or kingdom::json::value_t, which makes it impossible to
determine what's actually wrong with your code that triggered that error
message. value_t not being defined inside of kingdom::json is the
simplest possible explanation for that message, but your description
doesn't even suggest that you checked out that possibility, must less
describe what you found when you did check it out.
The compileable code he's asking for might contain the needed
information; at the very least, it could prompt a discussion that would
lead to a request for the needed information.
Even when I've explained why additional information is needed, I've
seldom seen anybody actually provide it. So I've started surveying these
people to find out their motivations. As one of these people, could you
explain why it is that you refuse to provide the information that your
prospective helpers tell you that they need in order to provide the help
that you yourself requested?
...
> As I stated these are just snippets. It wasn't meant to be "compilable code", but rather an illustrative example. That's it.
That's the problem - you're asking for help, and your illustrative
example doesn't provide enough information to allow people to provide
that help. This is a very common, and very frustrating situation. In my
experience, people who don't understand why they're running into
problems, and select the part of their code that they think is relevant,
are virtually certain to choose a part of their code that doesn't
include the actual problem. You are NOT a counter-example.
One of the error messages you provided, for instance, says "'kingdom::json::value_t': an undefined class is not allowed as an
argument to compiler intrinsic type trait '__is_empty' "
Yet the code you provide contains no definition for kingdom,
kingdom::json, or kingdom::json::value_t, which makes it impossible to
determine what's actually wrong with your code that triggered that error
message. value_t not being defined inside of kingdom::json is the
simplest possible explanation for that message, but your description
doesn't even suggest that you checked out that possibility, must less
describe what you found when you did check it out.
The compileable code he's asking for might contain the needed
information; at the very least, it could prompt a discussion that would
lead to a request for the needed information.
Even when I've explained why additional information is needed, I've
seldom seen anybody actually provide it. So I've started surveying these
people to find out their motivations. As one of these people, could you
explain why it is that you refuse to provide the information that your
prospective helpers tell you that they need in order to provide the help
that you yourself requested?
Michael Powell
Jan 15, 2019, 1:27:18 PM1/15/19
to
On Tuesday, January 15, 2019 at 1:20:44 PM UTC-5, james...@alumni.caltech.edu wrote:
> On Tuesday, January 15, 2019 at 12:29:32 PM UTC-5, Michael Powell wrote:
> ...
> > As I stated these are just snippets. It wasn't meant to be "compilable code", but rather an illustrative example. That's it.
>
> That's the problem - you're asking for help, and your illustrative
> example doesn't provide enough information to allow people to provide
> that help. This is a very common, and very frustrating situation. In my
> experience, people who don't understand why they're running into
> problems, and select the part of their code that they think is relevant,
> are virtually certain to choose a part of their code that doesn't
> include the actual problem. You are NOT a counter-example.
That's the thing. You are misreading the message here. This was not a request for questions, etc. This is a summary/take-away's.
> On Tuesday, January 15, 2019 at 12:29:32 PM UTC-5, Michael Powell wrote:
> ...
> > As I stated these are just snippets. It wasn't meant to be "compilable code", but rather an illustrative example. That's it.
>
> That's the problem - you're asking for help, and your illustrative
> example doesn't provide enough information to allow people to provide
> that help. This is a very common, and very frustrating situation. In my
> experience, people who don't understand why they're running into
> problems, and select the part of their code that they think is relevant,
> are virtually certain to choose a part of their code that doesn't
> include the actual problem. You are NOT a counter-example.
I've overcome the forward declarations and and standing on a viable approach.
Full steam ahead.
Thanks for the feedback!
Michael Powell
Jan 19, 2019, 5:30:45 PM1/19/19
to
I didn't check, but do you have that benchmark? I'm not sure how relevant or useful it would be to run that through my parser grammar for what I am trying to accomplish here, but it might interesting to respond with the results nonetheless.
Thanks!
> Daniel
Daniel
Jan 19, 2019, 6:45:57 PM1/19/19
to
On Saturday, January 19, 2019 at 5:30:45 PM UTC-5, Michael Powell wrote:
> On Friday, January 11, 2019 at 6:03:54 PM UTC-5, Daniel wrote:
> >
> >
>> I found an earlier tag of Milo Yip's benchmarks which has JSON Spirit,
>> https://github.com/miloyip/nativejson-benchmark/tree/v1.0.0
>> with smaller benchmark files, between 600 and 1200KB. JSON Spirit's
>> relative performance is a little bit better on those, but not by much, e.g. 8 x slower than cJSON vs 12 times slower.
>
> Hello,
>
> I didn't check, but do you have that benchmark?
Have what? I've posted the links to all the benchmarks I referred to.
> On Friday, January 11, 2019 at 6:03:54 PM UTC-5, Daniel wrote:
> >
> >
>> I found an earlier tag of Milo Yip's benchmarks which has JSON Spirit,
>> https://github.com/miloyip/nativejson-benchmark/tree/v1.0.0
>> with smaller benchmark files, between 600 and 1200KB. JSON Spirit's
>> relative performance is a little bit better on those, but not by much, e.g. 8 x slower than cJSON vs 12 times slower.
>
> Hello,
>
> I didn't check, but do you have that benchmark?
Daniel
Michael Powell
Jan 19, 2019, 7:34:30 PM1/19/19
to
> Daniel
Daniel
Jan 19, 2019, 7:50:28 PM1/19/19
to
On Saturday, January 19, 2019 at 7:34:30 PM UTC-5, Michael Powell wrote:
> Oh, I see; I guess I misunderstood and read that to mean you had a
> specifically "large" test case apart from the benchmarks.
>
I do, but I earlier provided a link to that :-) Anyway, here's the full set:
> Oh, I see; I guess I misunderstood and read that to mean you had a
> specifically "large" test case apart from the benchmarks.
>
Parsing and serializing text and integers:
https://github.com/danielaparker/json_benchmarks/blob/master/report/performance.md
Parsing and serializing text and doubles:
https://github.com/danielaparker/json_benchmarks/blob/master/report/performance_fp.md
Compliance comparisons:
https://danielaparker.github.io/json_benchmarks/
Just to note, my primary interest in these benchmarks is to evaluate how
jsoncons, of which I am the author, measures in comparison to other
libraries that use different architectures, especially on large files.
Daniel
0 new messages