A library for large amounts of read-only strings?

[Richard]

[Please do not mail me a copy of your followup]

I'm wondering if there is a C++ library that might fit this use case.

I have large amounts of text I need to scan through in a read-only
manner. It needs to be carved up into individual chunks for various
processing purposes, so it can't be easily handled as just a big char
array.

Handling individual chunks with a string_view is fine, but then there
is the manual management of the dynamically allocated storage (or
mmap-style file mapping) that holds the actual text. (Since
string_view is a non-owning representation.) I could use std::string,
but I think I'd get better memory behavior by allocating large,
fixed-size buffers (probably matched to the virtual memory page size)
on demand and carving them up for the various strings.

I was cooking up my own "rope" class (essentially a vector<string_view>)
and realized that it would be more ideal if each individual string_view
could reference count the associated buffer so that buffers would
naturally return themselves to a storage pool when no longer referenced
by any string_view.

In my use case I have lots and lots of read-only strings that I obtain
from either files or a network socket. I have the occasional user
entered string for which std::string is sufficient.

I'm not aware of any such library, but I thought I'd ping out to see
if anything rings a bell.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

[Scott Lurndal]

legaliz...@mail.xmission.com (Richard) writes:
>[Please do not mail me a copy of your followup]
>
>I'm wondering if there is a C++ library that might fit this use case.
>
>I have large amounts of text I need to scan through in a read-only
>manner. It needs to be carved up into individual chunks for various
>processing purposes, so it can't be easily handled as just a big char
>array.

Sounds like a job for lex(1) or flex(1).

[Chris M. Thomasson]

On 4/25/2023 8:56 AM, Richard wrote:
> [Please do not mail me a copy of your followup]
>
> I'm wondering if there is a C++ library that might fit this use case.
>
> I have large amounts of text I need to scan through in a read-only
> manner. It needs to be carved up into individual chunks for various
> processing purposes, so it can't be easily handled as just a big char
> array.

Is the text formatted in a certain way? How many "processing purposes"
do you have, what do they do? Break the text into sentences, paragraphs
and outlines? Or, does this text describe an object, or objects, in a
formatted way? Think of loading up a blender file to bring a model into
a game. It has textures, colors, animations, bones, ect...

Do you have to read the whole text in order to carve it up? Or can you
create a chunk on the fly as you are performing the scanning and
generating chunks phase, so to speak...?

> Handling individual chunks with a string_view is fine, but then there
> is the manual management of the dynamically allocated storage (or
> mmap-style file mapping) that holds the actual text. (Since
> string_view is a non-owning representation.) I could use std::string,
> but I think I'd get better memory behavior by allocating large,
> fixed-size buffers (probably matched to the virtual memory page size)
> on demand and carving them up for the various strings.

Think of a local chunk indexing into a larger super chunk, so to speak. ;^)

> I was cooking up my own "rope" class (essentially a vector<string_view>)
> and realized that it would be more ideal if each individual string_view
> could reference count the associated buffer so that buffers would
> naturally return themselves to a storage pool when no longer referenced
> by any string_view.



> In my use case I have lots and lots of read-only strings that I obtain
> from either files or a network socket. I have the occasional user
> entered string for which std::string is sufficient.

Are these read-only strings kept in a single large file with some sort
of delimiter to separate them?

[Bonita Montero]

Am 25.04.2023 um 17:56 schrieb Richard:
> [Please do not mail me a copy of your followup]
>
> I'm wondering if there is a C++ library that might fit this use case.
>
> I have large amounts of text I need to scan through in a read-only
> manner. It needs to be carved up into individual chunks for various
> processing purposes, so it can't be easily handled as just a big char
> array.
>
> Handling individual chunks with a string_view is fine, but then there
> is the manual management of the dynamically allocated storage (or
> mmap-style file mapping) that holds the actual text. (Since
> string_view is a non-owning representation.) I could use std::string,
> but I think I'd get better memory behavior by allocating large,
> fixed-size buffers (probably matched to the virtual memory page size)
> on demand and carving them up for the various strings.
>
> I was cooking up my own "rope" class (essentially a vector<string_view>)
> and realized that it would be more ideal if each individual string_view
> could reference count the associated buffer so that buffers would
> naturally return themselves to a storage pool when no longer referenced
> by any string_view.
>
> In my use case I have lots and lots of read-only strings that I obtain
> from either files or a network socket. I have the occasional user
> entered string for which std::string is sufficient.
>
> I'm not aware of any such library, but I thought I'd ping out to see
> if anything rings a bell.

I have something related to this topic. And reading in several lines
one line at a time is quite slow with the implementations of the C++
standard library. So I came up with the idea of writing a function
that I called linify() that has a generic callback that takes the
lines as a pair of start and end iterators. Here is the code:

#pragma once
#include <type_traits>
#include <iterator>

template<std::input_iterator InputIt, typename IsEnd, typename Consumer>
requires std::is_integral_v<std::iter_value_t<InputIt>>
&& requires( IsEnd isEnd, InputIt it ) { { isEnd( it ) } ->
std::convertible_to<bool>; }
&& requires( Consumer consumer, InputIt inputIt ) { { consumer(
inputIt, inputIt ) }; }
void linify( InputIt begin, IsEnd isEnd, Consumer consumer )
{
using namespace std;
if( isEnd( begin ) ) [[unlikely]]
return;
for( InputIt scn = begin; ; )
for( InputIt lineBegin = scn; ; )
if( *scn == '\n' ) [[unlikely]]
{
consumer( lineBegin, scn );
if( isEnd( ++scn ) )
return;
break;
}
else if( *scn == '\r' ) [[unlikely]]
{
consumer( lineBegin, scn );
if( isEnd( ++scn ) || *scn == '\n' && isEnd( ++scn ) )
return;
break;
}
else if( isEnd( ++scn ) ) [[unlikely]]
{
consumer( lineBegin, scn );
return;
}
}

template<std::input_iterator InputIt, typename Consumer>
requires std::is_integral_v<std::iter_value_t<InputIt>>
&& requires( Consumer consumer, InputIt inputIt ) { { consumer(
inputIt, inputIt ) }; }
inline void linify( InputIt begin, InputIt end, Consumer consumer )
{
linify( begin, [&]( InputIt it ) -> bool { return it == end; }, consumer );
}

template<std::input_iterator InputIt, typename Consumer>
requires std::is_integral_v<std::iter_value_t<InputIt>>
&& requires( Consumer consumer, InputIt inputIt ) { { consumer(
inputIt, inputIt ) }; }
inline void linify( InputIt begin, Consumer consumer )
{
linify( begin, [&]( InputIt it ) -> bool { return !*it; }, consumer );
}


Since the lambda of the consumer exists only once globally for the
entire application and has its own unique type, which means that
the instantiation of linify() also only exists once, the linify()
function and the lambda are compiled to where it is called .

[Richard]

[Please do not mail me a copy of your followup]

"Chris M. Thomasson" <chris.m.t...@gmail.com> spake the secret code
<u29g0v$118vm$1...@dont-email.me> thusly:

>[asked a ton of questions but made no mention of any existing library]

Do you have a library in mind or are you trying to build one?

I can build one just fine for myself, but it's best to ask around to
see if one already exists.

[Mut...@dastardlyhq.com]

On Wed, 26 Apr 2023 07:11:47 +0200
Bonita Montero <Bonita....@gmail.com> wrote:
>template<std::input_iterator InputIt, typename IsEnd, typename Consumer>
> requires std::is_integral_v<std::iter_value_t<InputIt>>
> && requires( IsEnd isEnd, InputIt it ) { { isEnd( it ) } ->
>std::convertible_to<bool>; }
> && requires( Consumer consumer, InputIt inputIt ) { { consumer(
>inputIt, inputIt ) }; }
>void linify( InputIt begin, IsEnd isEnd, Consumer consumer )

Serious question - do you really think this is a sane function prototype or
anything approaching intelligable code?

[Malcolm McLean]

On Tuesday, 25 April 2023 at 16:57:15 UTC+1, Richard wrote:
> [Please do not mail me a copy of your followup]
>
> I'm wondering if there is a C++ library that might fit this use case.
>
> I have large amounts of text I need to scan through in a read-only
> manner. It needs to be carved up into individual chunks for various
> processing purposes, so it can't be easily handled as just a big char
> array.
>
> Handling individual chunks with a string_view is fine, but then there
> is the manual management of the dynamically allocated storage (or
> mmap-style file mapping) that holds the actual text. (Since
> string_view is a non-owning representation.) I could use std::string,
> but I think I'd get better memory behavior by allocating large,
> fixed-size buffers (probably matched to the virtual memory page size)
> on demand and carving them up for the various strings.
>
> I was cooking up my own "rope" class (essentially a vector<string_view>)
> and realized that it would be more ideal if each individual string_view
> could reference count the associated buffer so that buffers would
> naturally return themselves to a storage pool when no longer referenced
> by any string_view.
>
> In my use case I have lots and lots of read-only strings that I obtain
> from either files or a network socket. I have the occasional user
> entered string for which std::string is sufficient.
>
> I'm not aware of any such library, but I thought I'd ping out to see
> if anything rings a bell.
>

It depends how you are processing these strings, and whether you need to do
it in just noticeable time on a personal computer, or as fast as possible on
a server.
If you are doing very many searches on a fixed body of text, then the structure to
use is a suffix tree. It's about ten times as memory intensive as a flat string,
but it allows for searching in constant time. However a suffix tree isn't easy to
implement, and is overkill for most text processing.

[Bonita Montero]

This prototype makes that the iterator and the consumer function
objects have the required properties through C++20 conceots. This
prevents any errors from inside the function body, which makes the
code easier to handle for people using the code.
If you don't like concepts you could delete them. But don't wonder
if you have typical template type property errors from inside the
function.
I think concepts are easy to read.

[Chris M. Thomasson]

On 4/25/2023 11:22 PM, Richard wrote:
> [Please do not mail me a copy of your followup]
>
> "Chris M. Thomasson" <chris.m.t...@gmail.com> spake the secret code
> <u29g0v$118vm$1...@dont-email.me> thusly:
>
>> [asked a ton of questions but made no mention of any existing library]
>
> Do you have a library in mind or are you trying to build one?

Humm... Good question. I do not really have an existing library in mind
right off the bat. Perhaps if certain questions were answered.

> I can build one just fine for myself, but it's best to ask around to
> see if one already exists.

Indeed. You might need to create something that is specialized to your
needs. Is this just a large read only file?

[Mut...@dastardlyhq.com]

On Wed, 26 Apr 2023 20:03:07 +0200

Bonita Montero <Bonita....@gmail.com> wrote:
>Am 26.04.2023 um 10:22 schrieb Mut...@dastardlyhq.com:
>> On Wed, 26 Apr 2023 07:11:47 +0200
>> Bonita Montero <Bonita....@gmail.com> wrote:
>>> template<std::input_iterator InputIt, typename IsEnd, typename Consumer>
>>> requires std::is_integral_v<std::iter_value_t<InputIt>>
>>> && requires( IsEnd isEnd, InputIt it ) { { isEnd( it ) } ->
>>> std::convertible_to<bool>; }
>>> && requires( Consumer consumer, InputIt inputIt ) { { consumer(
>>> inputIt, inputIt ) }; }
>>> void linify( InputIt begin, IsEnd isEnd, Consumer consumer )
>>
>> Serious question - do you really think this is a sane function prototype or
>> anything approaching intelligable code?
>
>This prototype makes that the iterator and the consumer function
>objects have the required properties through C++20 conceots. This
>prevents any errors from inside the function body, which makes the
>code easier to handle for people using the code.

Seriously?

>If you don't like concepts you could delete them. But don't wonder
>if you have typical template type property errors from inside the
>function.
>I think concepts are easy to read.

This prototype is illegible line noise and any half decent project manager
would throw it out if he saw it in his teams code and ask the person to write
something legible and maintainable.

[Bonita Montero]

Am 27.04.2023 um 09:47 schrieb Mut...@dastardlyhq.com:

> This prototype is illegible line noise and any half decent project manager
> would throw it out if he saw it in his teams code and ask the person to write
> something legible and maintainable.

Concepts aren't noise but help deveopers using generic code not to make
mistkes with would otherwise lead to inspecting the generic function's
body. And concepts are rather easy to read.

[Mut...@dastardlyhq.com]

Concepts in general might be easy to read, your code isn't. Its a convoluted
mess. Someone looking at code for the first time shouldn't have to decode the
function prototypes when they'll have enough work figuring out the code itself.
Function prototypes shouldn't be grokked in seconds.

[Mut...@dastardlyhq.com]

shouldn't -> should

[Bonita Montero]

Am 27.04.2023 um 17:29 schrieb Mut...@dastardlyhq.com:

> Concepts in general might be easy to read, your code isn't.

What you call superflous is concepted code.
Try BASIC !

[Bonita Montero]

Am 27.04.2023 um 17:29 schrieb Mut...@dastardlyhq.com:

> Concepts in general might be easy to read, your code isn't. Its a convoluted
> mess. Someone looking at code for the first time shouldn't have to decode the
> function prototypes when they'll have enough work figuring out the code itself.
> Function prototypes shouldn't be grokked in seconds.

If you have a function prototype with generic type you've got to look
into the documentation what are the required properties for that types.
With concepts you often doesn't need that because the prototype itself
shows what the requirements are. I can read such prototypes in 30s, you
can't.
There's a personality trait in the Big Five personality test called
openness for new experiences; you get a negative score on that.

[Mut...@dastardlyhq.com]

Its bollocks, thats what it is. There's zero need for anything like that in
a function prototype unless you're trying to be a showoff dick which you
manage so well.

>Try BASIC !

I wrote a version of it and I didn't need any of that shit anywhere in the
interpreter code.

[Mut...@dastardlyhq.com]

Having to wade through someones showboating bullshit is not an "experience" I
or many other devs want.

[Bonita Montero]

Am 27.04.2023 um 18:08 schrieb Mut...@dastardlyhq.com:

> Its bollocks, thats what it is. There's zero need for anything like that
> in a function prototype unless you're trying to be a showoff dick which
> you manage so well.

The generic types are missing sth. essential: what properties they have.
So concepts specify this and make the code more readable.

> I wrote a version of it and I didn't need any of that shit anywhere
> in the interpreter code.

You don't have the personality to develop software.

[Bonita Montero]

Depends on the mentioned personality trait. You have a set of
fixed patterns of thought and action that you don't deviate
from. That's bad for such an innovative job, especially with
a language like C++ you use like been stuck on C++98.

[Mut...@dastardlyhq.com]

On Thu, 27 Apr 2023 18:12:40 +0200
Bonita Montero <Bonita....@gmail.com> wrote:
>Am 27.04.2023 um 18:08 schrieb Mut...@dastardlyhq.com:
>
>> Its bollocks, thats what it is. There's zero need for anything like that
>> in a function prototype unless you're trying to be a showoff dick which
>> you manage so well.
>
>The generic types are missing sth. essential: what properties they have.

Yes, so essential that 30+ years after C++ was created they're finally
being introduced. How did we ever manage without them?

>So concepts specify this and make the code more readable.

LOL! :) You're priceless sometimes.

>> I wrote a version of it and I didn't need any of that shit anywhere
>> in the interpreter code.
>
>You don't have the personality to develop software.

Your psychological analysis is as bad as your code.

[Mut...@dastardlyhq.com]

On Thu, 27 Apr 2023 18:14:28 +0200
Bonita Montero <Bonita....@gmail.com> wrote:

>Am 27.04.2023 um 18:11 schrieb Mut...@dastardlyhq.com:
>> Having to wade through someones showboating bullshit is not an "experience"
>> I or many other devs want.
>
>Depends on the mentioned personality trait. You have a set of
>fixed patterns of thought and action that you don't deviate
>from. That's bad for such an innovative job, especially with
>a language like C++ you use like been stuck on C++98.

You can't differentiate shiny-shiny from useful. You're like a kid in a toy
shop. But whatever, go write your incomprehensible code. You'll not pass
many job tests writing that crap.

[Bonita Montero]

Am 27.04.2023 um 18:16 schrieb Mut...@dastardlyhq.com:

> On Thu, 27 Apr 2023 18:12:40 +0200

>> The generic types are missing sth. essential: what properties they have.

> Yes, so essential that 30+ years after C++ was created they're finally
> being introduced. How did we ever manage without them?

This was introduced because if you have surrounding code that uses a
generic function with types which haven't the required properties you
get a lot of errors from inside the generic function. With MSVC, clang++
and g++ you now get descriptive errors which show which constraint is
violated. You don't even have to understand how to write concepts to
understand the resulting errors.

> LOL! :) You're priceless sometimes.

Idiot.

> Your psychological analysis is as bad as your code.

I'm right on that because you're overburdened with a lot of new things.

[Bonita Montero]

Am 27.04.2023 um 18:18 schrieb Mut...@dastardlyhq.com:

> You can't differentiate shiny-shiny from useful. ...

I described why concepts are useful and if you don't understand
that you're a bad developer.

[Felix Palmen]

* Bonita Montero <Bonita....@gmail.com>:

As there's also a way to declare a concept separately (while, hold on
your breath, giving them meaningful names!), meant to avoid such an
illegible mess for more complex cases, that looks like moving the
goalpost.

--
Dipl.-Inform. Felix Palmen <fe...@palmen-it.de> ,.//..........
{web} http://palmen-it.de {jabber} [see email] ,//palmen-it.de
{pgp public key} http://palmen-it.de/pub.txt // """""""""""
{pgp fingerprint} 6936 13D5 5BBF 4837 B212 3ACC 54AD E006 9879 F231

[Bonita Montero]

Am 27.04.2023 um 18:28 schrieb Felix Palmen:

> As there's also a way to declare a concept separately (while, hold on
> your breath, giving them meaningful names!), meant to avoid such an
> illegible mess for more complex cases, that looks like moving the
> goalpost.

This may make sense if you define a concept several times,
e.g. if you have a generic class where you need the concept
for each defined method, but certainly not if you only specify
the requirements of the concept once.

[Felix Palmen]

* Bonita Montero <Bonita....@gmail.com>:

IOW, you don't get the concept (sic) of "readable code".

[Bonita Montero]

Am 27.04.2023 um 18:39 schrieb Felix Palmen:

> IOW, you don't get the concept (sic) of "readable code".

That depends on the skills.
For me that's easy to read, you're overburdened with that.

[Felix Palmen]

* Bonita Montero <Bonita....@gmail.com>:

Now *that* is utter bullshit. I really doubt you ever did serious
software development.

[Bonita Montero]

Am 27.04.2023 um 18:47 schrieb Felix Palmen:
> * Bonita Montero <Bonita....@gmail.com>:
>> Am 27.04.2023 um 18:39 schrieb Felix Palmen:
>>
>>> IOW, you don't get the concept (sic) of "readable code".
>>
>> That depends on the skills.
>> For me that's easy to read, you're overburdened with that.
>
> Now *that* is utter bullshit. I really doubt you ever did serious
> software development.

I'm 51 and I'm programming since I'm 10 and I'm skilled far
beyond you. What impression do you think you make on others,
except for those who are not better at programming than you ?

[Felix Palmen]

* Bonita Montero <Bonita....@gmail.com>:

> I'm 51 and I'm programming since I'm 10 and I'm skilled far
> beyond you. What impression do you think you make on others,
> except for those who are not better at programming than you ?

Ah! A "rock star programmer". Yeah, we all know that kind. Strongest
point is their self-esteem ^^

[Bonita Montero]

Am 27.04.2023 um 18:53 schrieb Felix Palmen:
> * Bonita Montero <Bonita....@gmail.com>:
>> I'm 51 and I'm programming since I'm 10 and I'm skilled far
>> beyond you. What impression do you think you make on others,
>> except for those who are not better at programming than you ?
>
> Ah! A "rock star programmer". Yeah, we all know that kind.
> Strongest point is their self-esteem ^^

Using such concepted code isn't being a rockstar programmer.
If that is too complex for you chose BASIC.

[Felix Palmen]

* Bonita Montero <Bonita....@gmail.com>:

You obviously have no idea what the term means. No, it's not a
compliment. lol.

[Scott Lurndal]

fe...@palmen-it.de (Felix Palmen) writes:
>* Bonita Montero <Bonita....@gmail.com>:
>> Am 27.04.2023 um 18:39 schrieb Felix Palmen:
>>
>>> IOW, you don't get the concept (sic) of "readable code".
>>
>> That depends on the skills.
>> For me that's easy to read, you're overburdened with that.
>
>Now *that* is utter bullshit. I really doubt you ever did serious
>software development.

Indeed, in the real world one attempts to make the code readable
to all colleagues (and for open source, the whole world), regardless
of each colleagues skill level.

[Daniel]

On Thursday, April 27, 2023 at 2:19:06 PM UTC-4, Scott Lurndal wrote:

> Indeed, in the real world one attempts to make the code readable
> to all colleagues (and for open source, the whole world), regardless
> of each colleagues skill level.

For some definition of readability. Would you regard this open source
code as particularly readable?

https://github.com/martinus/robin-hood-hashing/blob/master/src/include/robin_hood.h

It's an excellent implementation that I've drawn on for other work, but I
wouldn't regard it as particularly readable, I think that's in the nature of C++.
Particularly when it's required to support compilers from gcc 4.8 and
VS 2015 to present, and C++ 11 to latest, on Ubunto, macOS, and
Windows, as many open source projects have to do.

I'm curious what open source projects you had in mind that you thought
were readable by colleagues of lessor skill level.

I think readability is more easily attained on application projects, which
have more constrained requirements, and are users of rather than
creators of utilities.

Daniel

[Scott Lurndal]

Daniel <daniel...@gmail.com> writes:
>On Thursday, April 27, 2023 at 2:19:06=E2=80=AFPM UTC-4, Scott Lurndal wrot=
>e:
>
>> Indeed, in the real world one attempts to make the code readable=20
>> to all colleagues (and for open source, the whole world), regardless=20

>> of each colleagues skill level.
>

>For some definition of readability. Would you regard this open source=20
>code as particularly readable? =20
>
>https://github.com/martinus/robin-hood-hashing/blob/master/src/include/robi=

>n_hood.h
>
>It's an excellent implementation that I've drawn on for other work, but I

>wouldn't regard it as particularly readable, I think that's in the nature o=

>f C++.
>Particularly when it's required to support compilers from gcc 4.8 and
>VS 2015 to present, and C++ 11 to latest, on Ubunto, macOS, and
>Windows, as many open source projects have to do.
>
>I'm curious what open source projects you had in mind that you thought
>were readable by colleagues of lessor skill level.

Linux was the first that came to mind.

[Daniel]

On Thursday, April 27, 2023 at 2:46:59 PM UTC-4, Scott Lurndal wrote:

> Daniel <daniel...@gmail.com> writes:

> >I'm curious what open source projects you had in mind that you thought
> >were readable by colleagues of lessor skill level.
> Linux was the first that came to mind.

Any C++ ones?

Daniel

[Scott Lurndal]

Daniel <daniel...@gmail.com> writes:
>On Thursday, April 27, 2023 at 2:46:59=E2=80=AFPM UTC-4, Scott Lurndal wrot=
>e:
>> Daniel <daniel...@gmail.com> writes:=20
>
>> >I'm curious what open source projects you had in mind that you thought=
>=20

>> >were readable by colleagues of lessor skill level.
>> Linux was the first that came to mind.
>
>Any C++ ones?

gcc.

[Felix Palmen]

* Daniel <daniel...@gmail.com>:

> It's an excellent implementation that I've drawn on for other work,
> but I wouldn't regard it as particularly readable, I think that's in
> the nature of C++.

C++ certainly is a language where writing readable code can be a bit
challenging. It's still worth the effort.

> Particularly when it's required to support compilers from gcc 4.8 and
> VS 2015 to present, and C++ 11 to latest, on Ubunto, macOS, and
> Windows, as many open source projects have to do.

Supporting older compilers is often a bad idea. Supporting different
operating systems is another story, but this can be done in readable
ways.

> I'm curious what open source projects you had in mind that you thought
> were readable by colleagues of lessor skill level.

IMHO, "skill level" is a strawman in this discussion anyways. Readable
code is code that expresses the intent. Separating things and naming
things are key ingredients for that. From such readable code, everyone
regardless of "skill level" will profit.

[Chris M. Thomasson]

OT: Anyone for Applesoft BASIC?

https://www.calormen.com/jsbasic/

;^)

[Daniel]

On Thursday, April 27, 2023 at 3:48:16 PM UTC-4, Felix Palmen wrote:

> Supporting older compilers is often a bad idea.

I'm getting the impression that you have opinions, but do not have an
open source project that you are the owner of and that has tens of
thousands of users or more, and dependent artifacts :-)

New projects can say this is the earliest version we support, and it
can be a late one. Older projects have responsibilities.

> Supporting different
> operating systems is another story, but this can be done in readable
> ways.

Best regards,
Daniel

[Chris M. Thomasson]

On 4/27/2023 9:16 AM, Mut...@dastardlyhq.com wrote:
> On Thu, 27 Apr 2023 18:12:40 +0200
> Bonita Montero <Bonita....@gmail.com> wrote:
>> Am 27.04.2023 um 18:08 schrieb Mut...@dastardlyhq.com:
>>
>>> Its bollocks, thats what it is. There's zero need for anything like that
>>> in a function prototype unless you're trying to be a showoff dick which
>>> you manage so well.
>>
>> The generic types are missing sth. essential: what properties they have.
>
> Yes, so essential that 30+ years after C++ was created they're finally
> being introduced. How did we ever manage without them?
>
>> So concepts specify this and make the code more readable.
>
> LOL! :) You're priceless sometimes.

Bonita is an interesting person. Seemingly "loves" to insult, but....
Bonita is anything but stupid. One time I loosely compared her to the
cruel puppet:

https://www.thezorklibrary.com/history/cruel_puppet.html

;^o

[Chris M. Thomasson]

On 4/27/2023 11:45 AM, Daniel wrote:
> On Thursday, April 27, 2023 at 2:19:06 PM UTC-4, Scott Lurndal wrote:
>
>> Indeed, in the real world one attempts to make the code readable
>> to all colleagues (and for open source, the whole world), regardless
>> of each colleagues skill level.
>
> For some definition of readability. Would you regard this open source
> code as particularly readable?
>
> https://github.com/martinus/robin-hood-hashing/blob/master/src/include/robin_hood.h

[...]

I can handle it! ROBIN_HOOD_TRACE_ENABLED, no problem. ;^)

[Felix Palmen]

* Daniel <daniel...@gmail.com>:

> I'm getting the impression that you have opinions, but do not have an

Oh, you have "opinions" for sure. Have fun with them.

[Keith Thompson]

fe...@palmen-it.de (Felix Palmen) writes:
> * Bonita Montero <Bonita....@gmail.com>:
>> I'm 51 and I'm programming since I'm 10 and I'm skilled far
>> beyond you. What impression do you think you make on others,
>> except for those who are not better at programming than you ?
>
> Ah! A "rock star programmer". Yeah, we all know that kind. Strongest
> point is their self-esteem ^^

Felix, a lot of us have filtered out Bonita's posts. Please consider
not replying to Bonita unless you're actually talkinga about C++.

--
Keith Thompson (The_Other_Keith) Keith.S.T...@gmail.com
Working, but not speaking, for XCOM Labs
void Void(void) { Void(); } /* The recursive call of the void */

[Daniel]

On Thursday, April 27, 2023 at 3:39:33 PM UTC-4, Scott Lurndal wrote:
> Daniel <daniel...@gmail.com> writes:

> >> >I'm curious what open source projects you had in mind that you thought=

> >Any C++ ones?

> gcc.

That's not a bad example. The compiler code looks quite readable to my
eye, possibly because it uses more straight forward approaches than the
standard library needs to support. No uses of stateful allocators in the
compiler code!

I also had a look at the standard map implementation

https://github.com/gcc-mirror/gcc/blob/master/libstdc%2B%2B-v3/include/bits/stl_map.h

which has it's challenges for readability, but which I think compares very favourably to
comparable code in Visual Studio or boost. For boost, it's too many implementation
files, it becomes exhausting to follow.

Given the size of and multiple versions in the gcc code base, the directory structure
seems well organized and I found it straight forward to find what I was interested in.

I noticed that implementation classes for the standard library classes are collected in a
directory named "bits", I'm curious, is "bits" an acronym for something, or does it
simply mean "bits of code"?

Daniel

[Daniel]

On Thursday, April 27, 2023 at 4:16:06 PM UTC-4, Chris M. Thomasson wrote:

> Bonita is an interesting person. Seemingly "loves" to insult, but....
> Bonita is anything but stupid.

Totally agree. The group would be the poorer without Bonita. The
discussions you've had with Bonita have been some of the most
interesting discussions in the group. Regrettably, often on topics
that I don't feel qualified to contribute to.

Best regards,
Daniel

[Bonita Montero]

Am 27.04.2023 um 20:18 schrieb Scott Lurndal:

> Indeed, in the real world one attempts to make the code readable
> to all colleagues (and for open source, the whole world), regardless
> of each colleagues skill level.

My code is easy to read if you know C++20 concepts.

[Öö Tiib]

I you are such a rockstar programmer then perhaps provide cite of that
opinion.

[Bonita Montero]

Am 28.04.2023 um 06:58 schrieb Öö Tiib:


> I you are such a rockstar programmer then perhaps provide cite of that
> opinion.

I use concepts how they're meant, i.e. constraining all generic
types in a way that the code gets meaningful errors if the caller's
type don't match that constraints.

[Malcolm McLean]

It's a general problem with modern C++. Unless you are very familiar with
the language, code is hard to read, because it's not similar to other
languages. However you can argue that that is the case for C++ - there's
not much point in a language which just has a slightly different syntax for
counted loops but is otherwise essentially C.

[Malcolm McLean]

On Friday, 28 April 2023 at 05:58:36 UTC+1, Öö Tiib wrote:

Bonita's prototype
template<std::input_iterator InputIt, typename IsEnd, typename Consumer>
requires std::is_integral_v<std::iter_value_t<InputIt>>
&& requires( IsEnd isEnd, InputIt it ) { { isEnd( it ) } ->
std::convertible_to<bool>; }
&& requires( Consumer consumer, InputIt inputIt ) { { consumer(
inputIt, inputIt ) }; }

We've got three templated type, InputIt, isEnd, and Consumer.

InputIt must be an input iterator.

The we've got the constriants.
InputIt must have an integral value type
isEnd must be a callable type which returns a boolean compatible type whe called with an InputIt
Consumer must be a callable typewhich takes two InputIts.

It's not too hard.

[Mut...@dastardlyhq.com]

On Thu, 27 Apr 2023 18:21:09 +0200
Bonita Montero <Bonita....@gmail.com> wrote:

>Am 27.04.2023 um 18:16 schrieb Mut...@dastardlyhq.com:
>
>> On Thu, 27 Apr 2023 18:12:40 +0200
>

>>> The generic types are missing sth. essential: what properties they have.
>
>> Yes, so essential that 30+ years after C++ was created they're finally
>> being introduced. How did we ever manage without them?
>

>This was introduced because if you have surrounding code that uses a
>generic function with types which haven't the required properties you
>get a lot of errors from inside the generic function. With MSVC, clang++
>and g++ you now get descriptive errors which show which constraint is
>violated. You don't even have to understand how to write concepts to
>understand the resulting errors.

So you're saying that a new construct needs to be added to the language in
order to allow sane compile errors? Riiiiight...

>> LOL! :) You're priceless sometimes.
>

>Idiot.

>
>> Your psychological analysis is as bad as your code.
>

>I'm right on that because you're overburdened with a lot of new things.

No, I'm just able to filter out the crap.

[Mut...@dastardlyhq.com]

On Thu, 27 Apr 2023 13:15:50 -0700
"Chris M. Thomasson" <chris.m.t...@gmail.com> wrote:
>On 4/27/2023 9:16 AM, Mut...@dastardlyhq.com wrote:
>> On Thu, 27 Apr 2023 18:12:40 +0200
>> Bonita Montero <Bonita....@gmail.com> wrote:
>>> Am 27.04.2023 um 18:08 schrieb Mut...@dastardlyhq.com:
>>>
>>>> Its bollocks, thats what it is. There's zero need for anything like that
>>>> in a function prototype unless you're trying to be a showoff dick which
>>>> you manage so well.
>>>
>>> The generic types are missing sth. essential: what properties they have.
>>
>> Yes, so essential that 30+ years after C++ was created they're finally
>> being introduced. How did we ever manage without them?
>>
>>> So concepts specify this and make the code more readable.
>>
>> LOL! :) You're priceless sometimes.
>
>Bonita is an interesting person. Seemingly "loves" to insult, but....
>Bonita is anything but stupid. One time I loosely compared her to the
>cruel puppet:

Oh he's (and its almost certainly a man) not stupid. Just has no understanding
of other people and likes to show off.

[Mut...@dastardlyhq.com]

On Thu, 27 Apr 2023 18:50:30 +0200
Bonita Montero <Bonita....@gmail.com> wrote:
>Am 27.04.2023 um 18:47 schrieb Felix Palmen:
>> * Bonita Montero <Bonita....@gmail.com>:
>>> Am 27.04.2023 um 18:39 schrieb Felix Palmen:
>>>
>>>> IOW, you don't get the concept (sic) of "readable code".
>>>
>>> That depends on the skills.
>>> For me that's easy to read, you're overburdened with that.
>>
>> Now *that* is utter bullshit. I really doubt you ever did serious
>> software development.
>
>I'm 51 and I'm programming since I'm 10 and I'm skilled far
>beyond you. What impression do you think you make on others,

You think you're skilled because you can play around with various language
constructs. Software development involves writing code that actually does
something useful beyond some demonstation snippet. What have you done?

>except for those who are not better at programming than you ?

You're like the amateur guitar player who can play a few riffs from [some band]
really well to impress people at a party. But put him on on a stage and ask him
to do an entire concert and he'd fall apart.

[Mut...@dastardlyhq.com]

On Thu, 27 Apr 2023 11:45:01 -0700 (PDT)
Daniel <daniel...@gmail.com> wrote:
>On Thursday, April 27, 2023 at 2:19:06=E2=80=AFPM UTC-4, Scott Lurndal wrot=
>e:
>
>> Indeed, in the real world one attempts to make the code readable=20
>> to all colleagues (and for open source, the whole world), regardless=20

>> of each colleagues skill level.
>

>For some definition of readability. Would you regard this open source=20
>code as particularly readable? =20
>
>https://github.com/martinus/robin-hood-hashing/blob/master/src/include/robi=
>n_hood.h

Templates mixed up with macros. Ugh.

[Mut...@dastardlyhq.com]

On Fri, 28 Apr 2023 01:55:18 -0700 (PDT)
Malcolm McLean <malcolm.ar...@gmail.com> wrote:
>On Friday, 28 April 2023 at 05:58:36 UTC+1, =C3=96=C3=B6 Tiib wrote:
>> On Friday, 28 April 2023 at 04:36:15 UTC+3, Bonita Montero wrote:=20
>> > Am 27.04.2023 um 20:18 schrieb Scott Lurndal:=20
>> >=20
>> > > Indeed, in the real world one attempts to make the code readable=20
>> > > to all colleagues (and for open source, the whole world), regardless=
>=20
>> > > of each colleagues skill level.=20

>> > My code is easy to read if you know C++20 concepts.

>> I you are such a rockstar programmer then perhaps provide cite of that=20

>> opinion.
>Bonita's prototype
>template<std::input_iterator InputIt, typename IsEnd, typename Consumer>
>requires std::is_integral_v<std::iter_value_t<InputIt>>
>&& requires( IsEnd isEnd, InputIt it ) { { isEnd( it ) } ->
>std::convertible_to<bool>; }
>&& requires( Consumer consumer, InputIt inputIt ) { { consumer(

>inputIt, inputIt ) }; }=20

>
>We've got three templated type, InputIt, isEnd, and Consumer.
>
>InputIt must be an input iterator.
>
>The we've got the constriants.
>InputIt must have an integral value type

>isEnd must be a callable type which returns a boolean compatible type whe c=

>alled with an InputIt
>Consumer must be a callable typewhich takes two InputIts.
>
>It's not too hard.

You think having to spend time figuring out metacode in a function prototype
is acceptable?

[Bonita Montero]

I straightened the code a bit:

#pragma once
#include <type_traits>
#include <iterator>
#include <concepts>

template<typename Iterator, typename Consumer>
concept linify_input_iterator =
std::input_iterator<Iterator>
&& requires( Iterator inputIt ) { { *inputIt == char() } ->
std::convertible_to<bool>; }
&& requires( Consumer consumer, Iterator inputIt ) { { consumer(
inputIt, inputIt ) }; };

template<typename Iterator, typename IsEnd, typename Consumer>
requires linify_input_iterator<Iterator, Consumer>
&& requires( IsEnd isEnd, Iterator it ) { { isEnd( it ) } ->
std::convertible_to<bool>; }
void linify( Iterator begin, IsEnd isEnd, Consumer consumer )
{
using namespace std;
if( isEnd( begin ) ) [[unlikely]]
return;
for( Iterator scn = begin; ; )
for( Iterator lineBegin = scn; ; )
if( *scn == '\n' ) [[unlikely]]
{
consumer( lineBegin, scn );
if( isEnd( ++scn ) )
return;
break;
}
else if( *scn == '\r' ) [[unlikely]]
{
consumer( lineBegin, scn );
if( isEnd( ++scn ) || *scn == '\n' && isEnd( ++scn ) )
return;
break;
}
else if( isEnd( ++scn ) ) [[unlikely]]
{
consumer( lineBegin, scn );
return;
}
}

template<typename Iterator, typename Consumer>
requires linify_input_iterator<Iterator, Consumer>
inline void linify( Iterator begin, Iterator end, Consumer consumer )
{
linify( begin, [&]( Iterator it ) -> bool { return it == end; },
consumer );
}

template<typename Iterator, typename Consumer>
requires linify_input_iterator<Iterator, Consumer>
inline void linify( Iterator begin, Consumer consumer )
{
linify( begin, [&]( Iterator it ) -> bool { return !*it; }, consumer );
}


The common parts of all three functions are now in a concept,
the individual parts of the first function are still there.

[Bonita Montero]

Am 28.04.2023 um 11:15 schrieb Mut...@dastardlyhq.com:

> You think having to spend time figuring out metacode in a function prototype
> is acceptable?

This makes the code more readable and usable since you directly see
what are the required constraints on the generic types and if you
supply variables that don't have these properties you get more mea-
ningful errors.

[Bonita Montero]

Am 28.04.2023 um 11:01 schrieb Mut...@dastardlyhq.com:

> So you're saying that a new construct needs to be added to the language in
> order to allow sane compile errors? Riiiiight...

Generic code may lead to errors from inside the functions.
So you have a more meaningful error from outside the function
and you won't have to deal with the details.

[Malcolm McLean]

On Friday, 28 April 2023 at 10:01:53 UTC+1, Mut...@dastardlyhq.com wrote:
> On Thu, 27 Apr 2023 18:21:09 +0200
> Bonita Montero <Bonita....@gmail.com> wrote:
> >Am 27.04.2023 um 18:16 schrieb Mut...@dastardlyhq.com:
> >
> >> On Thu, 27 Apr 2023 18:12:40 +0200
> >
> >>> The generic types are missing sth. essential: what properties they have.
> >
> >> Yes, so essential that 30+ years after C++ was created they're finally
> >> being introduced. How did we ever manage without them?
> >
> >This was introduced because if you have surrounding code that uses a
> >generic function with types which haven't the required properties you
> >get a lot of errors from inside the generic function. With MSVC, clang++
> >and g++ you now get descriptive errors which show which constraint is
> >violated. You don't even have to understand how to write concepts to
> >understand the resulting errors.
>
> So you're saying that a new construct needs to be added to the language in
> order to allow sane compile errors? Riiiiight...
>

It's because of the way that templates developed in C++ from what was originally
a very limited idea to a sophisticated system. The syntax will accept any type.
However the surrounding code can only rarely accept any type whatsoever.
Worse, you can sometimes have subtle bugs, for example when passing an
integral type to a function which needs to do real arithmetic. The compiler is
of course incapable of understanding this.

[Mut...@dastardlyhq.com]

On Fri, 28 Apr 2023 02:49:10 -0700 (PDT)
Malcolm McLean <malcolm.ar...@gmail.com> wrote:
>On Friday, 28 April 2023 at 10:01:53 UTC+1, Mut...@dastardlyhq.com wrote:
>> On Thu, 27 Apr 2023 18:21:09 +0200

>> So you're saying that a new construct needs to be added to the language in
>> order to allow sane compile errors? Riiiiight...
>>
>It's because of the way that templates developed in C++ from what was
>originally
>a very limited idea to a sophisticated system. The syntax will accept any type.
>
>However the surrounding code can only rarely accept any type whatsoever.
>Worse, you can sometimes have subtle bugs, for example when passing an
>integral type to a function which needs to do real arithmetic. The compiler is
>of course incapable of understanding this.

Thats why template specialisation was created.

[Mut...@dastardlyhq.com]

Compiler writers are quite capable of making their compilers give useful
errors. Clang is a lot better at it than gcc for example , the latter whom
often dumps pages of borderline unparsable garbage to the screen. Thats not
a problem with C++, its a problem with gcc.

[Bonita Montero]

Am 28.04.2023 um 12:27 schrieb Mut...@dastardlyhq.com:

> Compiler writers are quite capable of making their compilers give useful

> errors. ...

Do you ever have debugged generic function-objects inside generic code
like you use ? The errors are much less useful than coming from a con-
cept. And you even don't have to understand concepts but just the error.

> Clang is a lot better at it than gcc for example, ...

You don't know what you're talking about. Imagine I wouldn't have con-
cepted my code and the isEnd Function-Object doesn't return a bool or
the consumer function-object doesn't accept two of the given forward
-iterators. The errors are more complicated to read than those resul-
ting from a concept. And if you have code like of common runtimes it
looks at Swahili to inspect that; then I'd rather perfer clean concepts.

[Mut...@dastardlyhq.com]

On Fri, 28 Apr 2023 12:57:41 +0200
Bonita Montero <Bonita....@gmail.com> wrote:
>Am 28.04.2023 um 12:27 schrieb Mut...@dastardlyhq.com:
>
>> Compiler writers are quite capable of making their compilers give useful
>> errors. ...
>
>Do you ever have debugged generic function-objects inside generic code
>like you use ? The errors are much less useful than coming from a con-
>cept. And you even don't have to understand concepts but just the error.
>
>> Clang is a lot better at it than gcc for example, ...
>
>You don't know what you're talking about. Imagine I wouldn't have con-

You're an arrogant ass sometimes. Every piece of personal code I write I compile
on MacOS and Linux (clang and gcc) and clang is far better at error reporting
than gcc in generic code but there's still plenty of room for improvement.

[Bonita Montero]

Am 28.04.2023 um 13:04 schrieb Mut...@dastardlyhq.com:

> You're an arrogant ass sometimes. Every piece of personal code I write I compile
> on MacOS and Linux (clang and gcc) and clang is far better at error reporting
> than gcc in generic code but there's still plenty of room for improvement.

The errors of any compiler when supplying types that don't match the
criteria are a lot harder to read than when the compiler says you which
properties are missing according to the concept - with any compiler.