Let cat use multiple cores for parallel processing.

[hongy...@gmail.com]

Today, I noticed that some of the discussions here are entitled

```
can the "cat" command use multi-processes"
```

At the bottom of the discussion, I see the following comment and script snippet:


Update On further thought, this will only prefetch up to the pipe buffer size, 64k. The following modification will pull all of the files into memory; if they can't fit, don't use it:
```
#!/bin/bash

fd=3
for file in *
do
eval exec $fd\<<(content=$(<$file); echo "$content")

fd=$((fd + 1))
done

out_fd=3
while [ "$out_fd" -lt "$fd" ]
do
cat <&${out_fd}
out_fd=$((out_fd + 1))
done
```

But I'm still not sure whether I can let `cat` use multiple cores for parallel processing. Any hints will be highly appreciated.

Regards,
HZ

[hongy...@gmail.com]

On Tuesday, April 19, 2022 at 7:27:48 AM UTC+8, hongy...@gmail.com wrote:
> Today, I noticed that some of the discussions here are entitled

See here [1] for the original discussion.

[1] https://unix.stackexchange.com/questions/81705/can-the-cat-command-use-multi-processes

[David W. Hodgins]

On Mon, 18 Apr 2022 19:27:45 -0400, hongy...@gmail.com <hongy...@gmail.com> wrote:
> But I'm still not sure whether I can let `cat` use multiple cores for parallel processing. Any hints will be highly appreciated.

The cat command is supposed to concatenate one or more files, optionally with some
modifications such as suppressing repeated empty output lines, and write them to a
single output. The limited processing available in the cat command ensures it will
be i/o bound, not cpu bound. The first bottleneck the cat command has to deal with
is the write speed. Since there is only one output, there is no possible benefit,
and lots of complications, to try to use more than one core.

To even think of using multiple cores for the cat command, strongly suggests you
should be using some other command.

What are you trying to accomplish? Is this just a test question for some course?

Ah. Just noticed your headers. You're using google groups, not usenet. After
a quick search, the thread you're referring to is from 8 years ago, and it's
answered in that thread, that there is no benefit.

Regards, Dave Hodgins.

[David W. Hodgins]

The cat command is not designed to merge files and is the wrong tool to use to
try and do so.

The sort command has a merge option, which is suitable for use if the files are
each already sorted.

Regards, Dave Hodgins

[Aragorn]

On 18.04.2022 at 20:18, David W. Hodgins scribbled:

> Ah. Just noticed your headers. You're using google groups, not
> usenet. After a quick search, the thread you're referring to is from
> 8 years ago, and it's answered in that thread, that there is no
> benefit.

It's Hongyi Zhao, the longstanding resident help vampire. Don't expect
any clarification from him, ever. He only asks questions — whether
on-topic or off-topic, he doesn't care — but he'll never answer any
question asked of him, and he'll never tell you what he's trying to
accomplish.

The worst part is that everyone here in comp.unix.shell is always eager
to reply to him, regardless of what he's asking, just to show everyone
else how well they know the different POSIX standards, without ever
questioning why Hongyi can't be bothered reading the manuals or doing
his own research, not to mention what he's trying to concoct with his
experiments.

And well, ultimately, he's a Google Grouper. That should say it all.
<shrug>


--
With respect,
= Aragorn =