Cool !

[Bonita Montero]

I can have native FILETIME in C++:

#include <Windows.h>
#include <iostream>
#include <chrono>

using namespace std;
using namespace chrono;

int main()
{
int64_t sinceEpoch = file_clock::now().time_since_epoch().count(), ft;
do
sinceEpoch = file_clock::now().time_since_epoch().count(),
GetSystemTimeAsFileTime( &(FILETIME &)ft );
while( sinceEpoch != ft );
cout << "C++: " << sinceEpoch << endl;
cout << "Win32: " << ft << endl;
}

[Bonita Montero]

I disassembled GetSystemTimeAsFileTime() some years ago and it used
x86's RDTSC internally. But now it switched to a memory-mapped hard-
ware register with about millisecond resolution. Before it had 100ns
-resolution.

#include <Windows.h>
#include <iostream>
#include <chrono>

using namespace std;
using namespace chrono;

int main()
{

constexpr int64_t ROUNDS = 100'000'000;
auto tGet = []()
{
int64_t t;
GetSystemTimeAsFileTime( &(FILETIME &)t );
return t;
};
int64_t tSum = 0, n = 0, tBefore = tGet();
for( int64_t r = ROUNDS, t, tDiff; r--; )
t = tGet(),
tDiff = t - tBefore,
tSum += tDiff,
n += (bool)tDiff,
tBefore = t;
cout << (double)tSum / n << endl;
}

[Bonita Montero]

I got it: they don't use the TSC anymore because there might be
still machines with invariant TSC and because the TSC would not
be contignous across virtual machine migrations. That's while
there's GetSystemTimePreciseAsFileTime, which is still up to
the TSC.
But the're for sure no x64-machines with invariant TSC. And to
manage virtual machine migrations Intel could have added an off-
set added to the TSC before it is read which is set by the kernel.

#include <Windows.h>
#include <iostream>
#include <chrono>

using namespace std;
using namespace chrono;

int main()
{

auto tDo = []( char const *what, void (*fn)( FILETIME * ) )

{
constexpr int64_t ROUNDS = 100'000'000;

auto tGet = [&]()
{
int64_t t;
fn( &(FILETIME &)t );

return t;
};
int64_t tSum = 0, n = 0, tBefore = tGet();
for( int64_t r = ROUNDS, t, tDiff; r--; )
t = tGet(),
tDiff = t - tBefore,
tSum += tDiff,
n += (bool)tDiff,
tBefore = t;
cout << (double)tSum / n << endl;

};
tDo( "precise: ", GetSystemTimePreciseAsFileTime );
tDo( "relaxed: ", GetSystemTimeAsFileTime );
}

[Scott Lurndal]

Bonita Montero <Bonita....@gmail.com> writes:

>But the're for sure no x64-machines with invariant TSC.

Perhaps that is a mistranslation from deutsch?

All modern x64 processors support invariant (and nonstop) TSC. That
doesn't mean that there are no x86 processors
running production code that do not support invariant TSC.

[Bonita Montero]

Am 09.08.2023 um 15:57 schrieb Scott Lurndal:
> Bonita Montero <Bonita....@gmail.com> writes:
>
>> But the're for sure no x64-machines with invariant TSC.
>
> Perhaps that is a mistranslation from deutsch?

No, that's still the reason. I know that x86 CPUs have
had invariant TSCs for a long time, x64 CPUs anyway.

I wonder if the argument that the TSC isn't used because it's
not portable across VM boundaries makes sense. The TSC is con-
verted into a timestamp that indicates the 100ns steps since
1/1/1601, and the offset that is selected for this is certainly
not absolutely precise. Also, migrating a VM takes quite a long
time and never guarantees exactly the same time. In this res-
pect, it would not be a problem to map GetSystemTimeAsFileTime()
directly to GetSystemTimePreciseAsFileTime(). The Microsoft
engineers probably didn't think about it.

[Scott Lurndal]

Bonita Montero <Bonita....@gmail.com> writes:
>Am 09.08.2023 um 15:57 schrieb Scott Lurndal:
>> Bonita Montero <Bonita....@gmail.com> writes:
>>
>>> But the're for sure no x64-machines with invariant TSC.
>>
>> Perhaps that is a mistranslation from deutsch?
>
>No, that's still the reason. I know that x86 CPUs have
>had invariant TSCs for a long time, x64 CPUs anyway.
>
>I wonder if the argument that the TSC isn't used because it's
>not portable across VM boundaries makes sense.

I suspect it is because nanosecond resolution on a filesystem
timestamp is neither possible nor useful.

[Bonita Montero]

Am 09.08.2023 um 18:01 schrieb Scott Lurndal:
> Bonita Montero <Bonita....@gmail.com> writes:
>> Am 09.08.2023 um 15:57 schrieb Scott Lurndal:
>>> Bonita Montero <Bonita....@gmail.com> writes:
>>>
>>>> But the're for sure no x64-machines with invariant TSC.
>>>
>>> Perhaps that is a mistranslation from deutsch?
>>
>> No, that's still the reason. I know that x86 CPUs have
>> had invariant TSCs for a long time, x64 CPUs anyway.
>>
>> I wonder if the argument that the TSC isn't used because it's
>> not portable across VM boundaries makes sense.
>
> I suspect it is because nanosecond resolution on a filesystem
> timestamp is neither possible nor useful.

FILETIME isn't at nanosecond-resolution but in 100ns-steps. If you'd
have a 100ns interval timestamp sine 1/1/1601 00:00.0 the timestamp
would have ended 7/21/2185 23:34:33.71. Microsoft expects that Windows
wil last longer (about 30828). ;-)

[Scott Lurndal]

That kind of resolution is only available if the underlying filesystem
metadata (e.g. inode) supports it. Most filesystems (particularly
legacy filesystems) support one second resolution. There may be some
that support sub-second (perhaps 10ms) resolution, but that is far
from universal.

What would the use case be for higher resolution filesystem timestamps stored
in metadata (which increases the size of the metadata, reducing
the amount of storage available for data by some small factor)?

[Bonita Montero]

Am 10.08.2023 um 16:44 schrieb Scott Lurndal:

> That kind of resolution is only available if the underlying filesystem
> metadata (e.g. inode) supports it. Most filesystems (particularly
> legacy filesystems) support one second resolution. There may be some
> that support sub-second (perhaps 10ms) resolution, but that is far
> from universal.

file_clock::timestamp with that resolution could be used independent
from any file-operations, just like results from GetSystemTimeAsFileTime
is usually not used for file operations.

> What would the use case be for higher resolution filesystem timestamps

> stored in metadata (which increases the size of the metadata, ...

No one cares for three 64 bit values for the file's metadata.

[Scott Lurndal]

Sure they do. If supporting 64-bit values causes the metadata size
to require additional storage space on the volume, that's a cost.

In any case, none of the existing filesystems, widely used, would
be willing to make such a change, so your point is moot.

[Bonita Montero]

Am 10.08.2023 um 17:49 schrieb Scott Lurndal:

> Sure they do. If supporting 64-bit values causes the metadata size
> to require additional storage space on the volume, that's a cost.

Wo really cares for that ? Imagine you have a small cluster size of
4kB and you have the smallest possible size of one cluster, then the
additional overhead is 12 byte against 4kB, that's 3 per thousand,
dimishing when the file becomes bigger. If the cluster size is larger
the part is even smaller.

> In any case, none of the existing filesystems, widely used,
> would be willing to make such a change, so your point is moot.

You couldn't change existing filesstems for that without
changing the API. This would never change for Unix systems.

[David Brown]

ext4 has nanosecond resolution for its timestamps - and that is, I would
argue, quite a widely used filesystem.

[Bonita Montero]

Am 11.08.2023 um 08:28 schrieb David Brown:

> ext4 has nanosecond resolution for its timestamps - and
> that is, I would argue, quite a widely used filesystem.

And what's the API to use that ?
For sure not stat(), because that uses time_t timestamps.

[Ben Bacarisse]

Bonita Montero <Bonita....@gmail.com> writes:

> Am 11.08.2023 um 08:28 schrieb David Brown:
>
>> ext4 has nanosecond resolution for its timestamps - and
>> that is, I would argue, quite a widely used filesystem.
>
> And what's the API to use that ?

Various, including stat(2).

> For sure not stat(), because that uses time_t timestamps.

Modern stat(2) uses struct timespec fields with #defines that make
st_[acm]time access the 'seconds' part of the timespec.

--
Ben.

[Paavo Helde]

Some quick googling says:

"Since kernel 2.5.48, the stat structure supports nanosecond resolution
for the three file timestamp fields."

Glibc is exposing these fields in form stat.st_mtim.tv_nsec et al.