Write to file in 8 places simultaneously

[Frederick Gotham]

First, let me give an example of code I wrote this morning to write to RAM in 8 places simultaneously:

unsigned constexpr g_amount_threads = 8;

#include <cstddef>
using std::size_t;

#include <cstdint> /* uint8_t, uint64_t */

#include <numeric> /* iota */

#include <thread> /* thread */

typedef std::uint8_t Element_Type;

size_t constexpr g_total_bytes = 4294967296ull; // Four gigabytes

size_t constexpr g_total_elements = g_total_bytes / sizeof(Element_Type);

size_t constexpr g_elements_per_thread = g_total_elements / g_amount_threads;

Element_Type *g_p_mem = nullptr;

void Thread_Entry_Point_Set_Memory(unsigned const thread_number)
{
Element_Type *const p = g_p_mem + (g_elements_per_thread * thread_number);

Element_Type *const q = p + g_elements_per_thread;

std::iota(p, q, static_cast<Element_Type>(g_elements_per_thread * thread_number));
}

#include <iostream>
using std::cout;
using std::endl;

auto main(void) -> int
{
g_p_mem = new Element_Type[g_total_elements];

std::thread *ptrs_threads[g_amount_threads];

for (unsigned i = 0; i != g_amount_threads; ++i)
{
ptrs_threads[i] = new std::thread(Thread_Entry_Point_Set_Memory,i);
}

// Now join all the threads

for (auto &p : ptrs_threads)
{
p->join();

delete p;
}

delete [] g_p_mem;
}


I have measured the differences in execution time with 1 thread, with 2 threads, with 4 threads, etc..

What I want to do now is create a very large file (4 GB) on the hard disk and write to it in 8 different places simultaneously. I know that I can create 8 different files and deal with them independently, but I just want one file.

I think this would be possible under Unix using the POSIX functions open, pwrite, writenv. Has anyone tried it?

[Paavo Helde]

This is basically testing the main memory throughput. For curiosity,
what where the results?

>
> What I want to do now is create a very large file (4 GB) on the hard disk and write to it in 8 different places simultaneously. I know that I can create 8 different files and deal with them independently, but I just want one file.

And this is testing either main memory throughput or hard disk
throughput, depending on the amount of free memory in the computer.

>
> I think this would be possible under Unix using the POSIX functions open, pwrite, writenv. Has anyone tried it?

If you aim for max speed and are willing to use non-portable code, you
should probably consider mmap.

[Frederick Gotham]

On Thursday, January 2, 2020 at 9:47:01 AM UTC, Frederick Gotham wrote:

> I think this would be possible under Unix using the POSIX functions open, pwrite, writenv. Has anyone tried it?

Making a few changes to my original code, here's what I have so far for writing to disk. I'm not getting consistent behaviour yet (the output files have a different sha256 sum depending on how many cores I use). Anyway here's what I have so far:

unsigned constexpr g_amount_threads = 8;

char const g_filepath[] = "/home/fgotham/special.txt";

#include <cstddef>
using std::size_t;

#include <cstdint> /* uint8_t, uint64_t */

#include <numeric> /* iota */

#include <thread> /* thread */

#include <stdio.h> /* fopen, fseek, fputc, fclose */

#include <iostream>
using std::cout;
using std::endl;

/* The next three lines are for POSIX functions: open, pwrite, close */
#include <unistd.h>
#include <sys/stat.h>
#include <fcntl.h>

struct Thread_Local_Stuff {
std::thread *ptr_thread;
int fd;
};

Thread_Local_Stuff g_thread_stuffs[g_amount_threads];

typedef std::uint8_t Element_Type;

size_t constexpr g_total_bytes = 4294967296ull; // Four gigabytes

size_t constexpr g_bytes_per_thread = g_total_bytes / g_amount_threads;

size_t constexpr g_total_elements = g_total_bytes / sizeof(Element_Type);

size_t constexpr g_elements_per_thread = g_total_elements / g_amount_threads;

Element_Type *g_p_mem = nullptr;

void Thread_Entry_Point(unsigned const thread_number)

{
Element_Type *const p = g_p_mem + (g_elements_per_thread * thread_number);

Element_Type *const q = p + g_elements_per_thread;

std::iota(p, q, static_cast<Element_Type>(g_elements_per_thread * thread_number));

pwrite( g_thread_stuffs[thread_number].fd, p, g_bytes_per_thread, p - g_p_mem );
}

auto main(void) -> int
{
// First create the very large file

FILE *pfile = fopen(g_filepath, "w");
fseek(pfile, g_total_bytes - 1u, SEEK_SET);
fputc('\0', pfile);
fclose(pfile);

// Now start the threads

g_p_mem = new Element_Type[g_total_elements];

for (unsigned i = 0; i != g_amount_threads; ++i)
{

g_thread_stuffs[i].fd = open(g_filepath, O_WRONLY|O_SYNC);
g_thread_stuffs[i].ptr_thread = new std::thread(Thread_Entry_Point,i);

}

// Now join all the threads

for (auto &e : g_thread_stuffs)
{
e.ptr_thread->join();

delete e.ptr_thread;

close(e.fd);
}

delete [] g_p_mem;
}

[Frederick Gotham]

On Thursday, January 2, 2020 at 10:46:53 AM UTC, Paavo Helde wrote:

> This is basically testing the main memory throughput. For curiosity,
> what where the results?

One thread is 12 seconds.

Eight threads is 1.8 seconds.

> And this is testing either main memory throughput or hard disk
> throughput, depending on the amount of free memory in the computer.

And depending on where the bottle necks are.

> If you aim for max speed and are willing to use non-portable code, you
> should probably consider mmap.

I'm gonna play with the POSIX functions so more before I look up mmap.

[Jorgen Grahn]

On Thu, 2020-01-02, Frederick Gotham wrote:
> On Thursday, January 2, 2020 at 10:46:53 AM UTC, Paavo Helde wrote:

...

>> If you aim for max speed and are willing to use non-portable code, you
>> should probably consider mmap.
>
> I'm gonna play with the POSIX functions so more before I look up mmap.

mmap is a POSIX function, too.

/Jorgen

--
// Jorgen Grahn <grahn@ Oo o. . .
\X/ snipabacken.se> O o .

[Bonita Montero]

> std::thread *ptrs_threads[g_amount_threads];
> for (unsigned i = 0; i != g_amount_threads; ++i)
> {
> ptrs_threads[i] = new std::thread(Thread_Entry_Point_Set_Memory,i);
> }

Use
vector<thread> threads( g_amount_threads ).
for( thread &t : threads )
t = move( thread( threadFn, params ) );