Missing timestamps in Debian repository

[Steffen Grunewald]

Hello,

a while ago I had reported (at the Stammtisch) that I found diverging timestamps
of packages provided via BeeGFS repositories.
It took me a while to reproduce this, but here's my observations:

(1) Check https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/ - in the
browser, all timestamps are identical (to the minute).

(2) Run
$ wget -m -nvpdH https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/ 2>&1 | tee wget.log
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:10:03 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/ [5462] -> "index.html" [1]
2021-10-05 17:10:03 URL:https://www.beegfs.io/robots.txt [912/912] -> "robots.txt" [1]
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:10:03 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/?C=N;O=D [5462] -> "index.html?C=N;O=D" [1]
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:10:03 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/?C=M;O=A [5462] -> "index.html?C=M;O=A" [1]
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:10:03 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/?C=S;O=A [5462] -> "index.html?C=S;O=A" [1]
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:10:03 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/?C=D;O=A [5462] -> "index.html?C=D;O=A" [1]
2021-10-05 17:10:03 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-client-compat_7.2.4_all.deb [302320/302320] -> "beegfs-client-compat_7.2.4_all.deb" [1]
2021-10-05 17:10:03 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-client-dev_7.2.4_amd64.deb [10324/10324] -> "beegfs-client-dev_7.2.4_amd64.deb" [1]
2021-10-05 17:10:03 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-client-dkms_7.2.4_all.deb [308040/308040] -> "beegfs-client-dkms_7.2.4_all.deb" [1]
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:10:03 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-client_7.2.4_all.deb [314848/314848] -> "beegfs-client_7.2.4_all.deb" [1]
2021-10-05 17:10:03 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-common_7.2.4_amd64.deb [10716/10716] -> "beegfs-common_7.2.4_amd64.deb" [1]
2021-10-05 17:10:03 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-helperd-dbg_7.2.4_amd64.deb [9177508/9177508] -> "beegfs-helperd-dbg_7.2.4_amd64.deb" [1]
2021-10-05 17:10:03 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-helperd_7.2.4_amd64.deb [336752/336752] -> "beegfs-helperd_7.2.4_amd64.deb" [1]
2021-10-05 17:10:04 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-meta-dbg_7.2.4_amd64.deb [58398992/58398992] -> "beegfs-meta-dbg_7.2.4_amd64.deb" [1]
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:10:04 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-meta_7.2.4_amd64.deb [1331804/1331804] -> "beegfs-meta_7.2.4_amd64.deb" [1]
2021-10-05 17:10:04 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-mgmtd-dbg_7.2.4_amd64.deb [23301264/23301264] -> "beegfs-mgmtd-dbg_7.2.4_amd64.deb" [1]
2021-10-05 17:10:04 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-mgmtd_7.2.4_amd64.deb [664388/664388] -> "beegfs-mgmtd_7.2.4_amd64.deb" [1]
2021-10-05 17:10:04 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-mirror-md-dbg_7.2.4_amd64.deb [457302/457302] -> "beegfs-mirror-md-dbg_7.2.4_amd64.deb" [1]
2021-10-05 17:10:04 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-mirror-md_7.2.4_amd64.deb [18622/18622] -> "beegfs-mirror-md_7.2.4_amd64.deb" [1]
2021-10-05 17:10:04 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-mon-dbg_7.2.4_amd64.deb [13679240/13679240] -> "beegfs-mon-dbg_7.2.4_amd64.deb" [1]
2021-10-05 17:10:04 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-mon-grafana_7.2.4_all.deb [9264/9264] -> "beegfs-mon-grafana_7.2.4_all.deb" [1]
2021-10-05 17:10:05 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-mon_7.2.4_amd64.deb [445276/445276] -> "beegfs-mon_7.2.4_amd64.deb" [1]
2021-10-05 17:10:05 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-storage-dbg_7.2.4_amd64.deb [29509784/29509784] -> "beegfs-storage-dbg_7.2.4_amd64.deb" [1]
2021-10-05 17:10:05 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-storage_7.2.4_amd64.deb [809692/809692] -> "beegfs-storage_7.2.4_amd64.deb" [1]
2021-10-05 17:10:05 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-utils-dbg_7.2.4_amd64.deb [51936184/51936184] -> "beegfs-utils-dbg_7.2.4_amd64.deb" [1]
2021-10-05 17:10:05 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-utils-dev_7.2.4_amd64.deb [8332/8332] -> "beegfs-utils-dev_7.2.4_amd64.deb" [1]
2021-10-05 17:10:05 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-utils_7.2.4_amd64.deb [1349484/1349484] -> "beegfs-utils_7.2.4_amd64.deb" [1]
2021-10-05 17:10:05 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs_7.2.4_amd64.build [57127/57127] -> "beegfs_7.2.4_amd64.build" [1]
2021-10-05 17:10:05 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs_7.2.4_amd64.buildinfo [14158/14158] -> "beegfs_7.2.4_amd64.buildinfo" [1]
2021-10-05 17:10:05 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs_7.2.4_amd64.changes [8767/8767] -> "beegfs_7.2.4_amd64.changes" [1]
2021-10-05 17:10:06 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beehive_7.2.4.deb [6463264/6463264] -> "beehive_7.2.4.deb" [1]
2021-10-05 17:10:06 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beeond-thirdparty-gpl_7.2.4_amd64.deb [65590/65590] -> "beeond-thirdparty-gpl_7.2.4_amd64.deb" [1]
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:10:06 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beeond_7.2.4_amd64.deb [21134/21134] -> "beeond_7.2.4_amd64.deb" [1]
2021-10-05 17:10:06 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/libbeegfs-ib_7.2.4_amd64.deb [517766/517766] -> "libbeegfs-ib_7.2.4_amd64.deb" [1]
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:10:06 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/?C=N;O=A [5462] -> "index.html?C=N;O=A" [1]
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:10:06 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/?C=M;O=D [5462] -> "index.html?C=M;O=D" [1]
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:10:06 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/?C=S;O=D [5462] -> "index.html?C=S;O=D" [1]
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:10:06 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/?C=D;O=D [5462] -> "index.html?C=D;O=D" [1]
FINISHED --2021-10-05 17:10:06--
Total wall clock time: 2.9s
Downloaded: 38 files, 190M in 2.0s (94.3 MB/s)

(3) After a while, rerun
$ wget -m -nvpdH https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/ 2>&1 | tee wget2.log
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:13:18 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/ [5462] -> "index.html" [1]
2021-10-05 17:13:18 URL:https://www.beegfs.io/robots.txt [912/912] -> "robots.txt.1" [1]
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:13:18 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/?C=N;O=D [5462] -> "index.html?C=N;O=D" [1]
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:13:18 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/?C=M;O=A [5462] -> "index.html?C=M;O=A" [1]
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:13:18 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/?C=S;O=A [5462] -> "index.html?C=S;O=A" [1]
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:13:18 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/?C=D;O=A [5462] -> "index.html?C=D;O=A" [1]
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:13:19 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-client_7.2.4_all.deb [314848/314848] -> "beegfs-client_7.2.4_all.deb" [1]
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:13:19 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-meta_7.2.4_amd64.deb [1331804/1331804] -> "beegfs-meta_7.2.4_amd64.deb" [1]
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:13:19 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beeond_7.2.4_amd64.deb [21134/21134] -> "beeond_7.2.4_amd64.deb" [1]
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:13:19 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/?C=N;O=A [5462] -> "index.html?C=N;O=A" [1]
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:13:19 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/?C=M;O=D [5462] -> "index.html?C=M;O=D" [1]
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:13:19 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/?C=S;O=D [5462] -> "index.html?C=S;O=D" [1]
Last-modified header missing -- time-stamps turned off.
2021-10-05 17:13:19 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/?C=D;O=D [5462] -> "index.html?C=D;O=D" [1]
FINISHED --2021-10-05 17:13:19--
Total wall clock time: 1.2s
Downloaded: 13 files, 1.6M in 0.2s (9.19 MB/s)


Something must be special about these three packages, making them lack that
"last-modified" header.

To be honest, I can't spot a pattern :-( in particular as
https://www.beegfs.io/release/beegfs_7.2.4/dists/rhel8/x86_64/ shows the same
behaviour but for *different packages* (beegfs-helperd-debuginfo-7.2.4-el8.x86_64.rpm
and beegfs-utils-debuginfo-7.2.4-el8.x86_64.rpm), and other distros have *other
packages affected*.


Any idea?

Cheers,
Steffen


--
Steffen Grunewald, Cluster Administrator
Max Planck Institute for Gravitational Physics (Albert Einstein Institute)
Am Mühlenberg 1 * D-14476 Potsdam-Golm * Germany
~~~
Fon: +49-331-567 7274
Mail: steffen.grunewald(at)aei.mpg.de
~~~

[Steffen Grunewald]

On Thu, 2021-10-07 at 10:10:16 +0200, Steffen Grunewald wrote:
> Hello,
>
> a while ago I had reported (at the Stammtisch) that I found diverging timestamps
> of packages provided via BeeGFS repositories.
> It took me a while to reproduce this, but here's my observations:
>

> $ wget -m -nvpdH https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/

The situation seems to have changed slightly, now the pattern is more uniform after
setting up a new mirror and running the same mirror command multiple times:

$ wget -m -nvp https://www.beegfs.io/release/beegfs_7.2.4/ 2>&1 | grep -v index.html | grep URL:
2021-10-27 09:46:51 URL:https://www.beegfs.io/robots.txt [912/912] -> "www.beegfs.io/robots.txt" [1]
2021-10-27 09:46:54 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/alma8/noarch/beegfs-client-7.2.4-el8.noarch.rpm [410012/410012] -> "www.beegfs.io/release/beegfs_7.2.4/dists/alma8/noarch/beegfs-client-7.2.4-el8.noarch.rpm" [1]
2021-10-27 09:46:55 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/alma8/x86_64/beegfs-meta-7.2.4-el8.x86_64.rpm [1418908/1418908] -> "www.beegfs.io/release/beegfs_7.2.4/dists/alma8/x86_64/beegfs-meta-7.2.4-el8.x86_64.rpm" [1]
2021-10-27 09:46:55 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/alma8/x86_64/beeond-7.2.4-el8.x86_64.rpm [25220/25220] -> "www.beegfs.io/release/beegfs_7.2.4/dists/alma8/x86_64/beeond-7.2.4-el8.x86_64.rpm" [1]
2021-10-27 09:46:55 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/bionic/amd64/beegfs-client_7.2.4_all.deb [314916/314916] -> "www.beegfs.io/release/beegfs_7.2.4/dists/bionic/amd64/beegfs-client_7.2.4_all.deb" [1]
2021-10-27 09:46:55 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/bionic/amd64/beegfs-meta_7.2.4_amd64.deb [1343664/1343664] -> "www.beegfs.io/release/beegfs_7.2.4/dists/bionic/amd64/beegfs-meta_7.2.4_amd64.deb" [1]
2021-10-27 09:46:56 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/bionic/amd64/beeond_7.2.4_amd64.deb [21142/21142] -> "www.beegfs.io/release/beegfs_7.2.4/dists/bionic/amd64/beeond_7.2.4_amd64.deb" [1]
2021-10-27 09:46:56 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/buster/amd64/beegfs-client_7.2.4_all.deb [314848/314848] -> "www.beegfs.io/release/beegfs_7.2.4/dists/buster/amd64/beegfs-client_7.2.4_all.deb" [1]
2021-10-27 09:46:56 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/buster/amd64/beegfs-meta_7.2.4_amd64.deb [1331804/1331804] -> "www.beegfs.io/release/beegfs_7.2.4/dists/buster/amd64/beegfs-meta_7.2.4_amd64.deb" [1]
2021-10-27 09:46:57 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/buster/amd64/beeond_7.2.4_amd64.deb [21134/21134] -> "www.beegfs.io/release/beegfs_7.2.4/dists/buster/amd64/beeond_7.2.4_amd64.deb" [1]
2021-10-27 09:46:57 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-client_7.2.4_all.deb [314848/314848] -> "www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-client_7.2.4_all.deb" [1]
2021-10-27 09:46:57 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-meta_7.2.4_amd64.deb [1331804/1331804] -> "www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beegfs-meta_7.2.4_amd64.deb" [1]
2021-10-27 09:46:58 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beeond_7.2.4_amd64.deb [21134/21134] -> "www.beegfs.io/release/beegfs_7.2.4/dists/deb10/amd64/beeond_7.2.4_amd64.deb" [1]
2021-10-27 09:46:58 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb9/amd64/beegfs-client_7.2.4_all.deb [316068/316068] -> "www.beegfs.io/release/beegfs_7.2.4/dists/deb9/amd64/beegfs-client_7.2.4_all.deb" [1]
2021-10-27 09:46:58 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb9/amd64/beegfs-meta_7.2.4_amd64.deb [1347466/1347466] -> "www.beegfs.io/release/beegfs_7.2.4/dists/deb9/amd64/beegfs-meta_7.2.4_amd64.deb" [1]
2021-10-27 09:46:59 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/deb9/amd64/beeond_7.2.4_amd64.deb [21062/21062] -> "www.beegfs.io/release/beegfs_7.2.4/dists/deb9/amd64/beeond_7.2.4_amd64.deb" [1]
2021-10-27 09:46:59 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/focal/amd64/beegfs-client_7.2.4_all.deb [314824/314824] -> "www.beegfs.io/release/beegfs_7.2.4/dists/focal/amd64/beegfs-client_7.2.4_all.deb" [1]
2021-10-27 09:46:59 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/focal/amd64/beegfs-meta_7.2.4_amd64.deb [1342276/1342276] -> "www.beegfs.io/release/beegfs_7.2.4/dists/focal/amd64/beegfs-meta_7.2.4_amd64.deb" [1]
2021-10-27 09:46:59 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/focal/amd64/beeond_7.2.4_amd64.deb [21142/21142] -> "www.beegfs.io/release/beegfs_7.2.4/dists/focal/amd64/beeond_7.2.4_amd64.deb" [1]
2021-10-27 09:47:00 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/rhel7/noarch/beegfs-client-7.2.4-el7.noarch.rpm [405660/405660] -> "www.beegfs.io/release/beegfs_7.2.4/dists/rhel7/noarch/beegfs-client-7.2.4-el7.noarch.rpm" [1]
2021-10-27 09:47:01 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/rhel7/x86_64/beegfs-meta-7.2.4-el7.x86_64.rpm [1321264/1321264] -> "www.beegfs.io/release/beegfs_7.2.4/dists/rhel7/x86_64/beegfs-meta-7.2.4-el7.x86_64.rpm" [1]
2021-10-27 09:47:01 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/rhel7/x86_64/beeond-7.2.4-el7.x86_64.rpm [20868/20868] -> "www.beegfs.io/release/beegfs_7.2.4/dists/rhel7/x86_64/beeond-7.2.4-el7.x86_64.rpm" [1]
2021-10-27 09:47:01 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/rhel8/noarch/beegfs-client-7.2.4-el8.noarch.rpm [409852/409852] -> "www.beegfs.io/release/beegfs_7.2.4/dists/rhel8/noarch/beegfs-client-7.2.4-el8.noarch.rpm" [1]
2021-10-27 09:47:02 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/rhel8/x86_64/beegfs-meta-7.2.4-el8.x86_64.rpm [1418904/1418904] -> "www.beegfs.io/release/beegfs_7.2.4/dists/rhel8/x86_64/beegfs-meta-7.2.4-el8.x86_64.rpm" [1]
2021-10-27 09:47:02 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/rhel8/x86_64/beeond-7.2.4-el8.x86_64.rpm [25220/25220] -> "www.beegfs.io/release/beegfs_7.2.4/dists/rhel8/x86_64/beeond-7.2.4-el8.x86_64.rpm" [1]
2021-10-27 09:47:03 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/rocky/noarch/beegfs-client-7.2.4-el8.noarch.rpm [409844/409844] -> "www.beegfs.io/release/beegfs_7.2.4/dists/rocky/noarch/beegfs-client-7.2.4-el8.noarch.rpm" [1]
2021-10-27 09:47:04 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/rocky/x86_64/beegfs-meta-7.2.4-el8.x86_64.rpm [1419080/1419080] -> "www.beegfs.io/release/beegfs_7.2.4/dists/rocky/x86_64/beegfs-meta-7.2.4-el8.x86_64.rpm" [1]
2021-10-27 09:47:04 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/rocky/x86_64/beeond-7.2.4-el8.x86_64.rpm [25220/25220] -> "www.beegfs.io/release/beegfs_7.2.4/dists/rocky/x86_64/beeond-7.2.4-el8.x86_64.rpm" [1]
2021-10-27 09:47:04 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/sles15/noarch/beegfs-client-7.2.4-sles15.noarch.rpm [371040/371040] -> "www.beegfs.io/release/beegfs_7.2.4/dists/sles15/noarch/beegfs-client-7.2.4-sles15.noarch.rpm" [1]
2021-10-27 09:47:05 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/sles15/x86_64/beegfs-meta-7.2.4-sles15.x86_64.rpm [28583748/28583748] -> "www.beegfs.io/release/beegfs_7.2.4/dists/sles15/x86_64/beegfs-meta-7.2.4-sles15.x86_64.rpm" [1]
2021-10-27 09:47:05 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/sles15/x86_64/beeond-7.2.4-sles15.x86_64.rpm [23988/23988] -> "www.beegfs.io/release/beegfs_7.2.4/dists/sles15/x86_64/beeond-7.2.4-sles15.x86_64.rpm" [1]
2021-10-27 09:47:05 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/stretch/amd64/beegfs-client_7.2.4_all.deb [316068/316068] -> "www.beegfs.io/release/beegfs_7.2.4/dists/stretch/amd64/beegfs-client_7.2.4_all.deb" [1]
2021-10-27 09:47:06 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/stretch/amd64/beegfs-meta_7.2.4_amd64.deb [1347466/1347466] -> "www.beegfs.io/release/beegfs_7.2.4/dists/stretch/amd64/beegfs-meta_7.2.4_amd64.deb" [1]
2021-10-27 09:47:06 URL:https://www.beegfs.io/release/beegfs_7.2.4/dists/stretch/amd64/beeond_7.2.4_amd64.deb [21062/21062] -> "www.beegfs.io/release/beegfs_7.2.4/dists/stretch/amd64/beeond_7.2.4_amd64.deb" [1]

I'm wondering what makes beeond, beegfs-client and beegfs-meta so special
that the server refuses to provide a timestamp (while one exists in the
directory listing)?

[alexand...@thinkparq.com]

Hi Steffen.

Thanks a lot for debugging and letting us know your observations regarding the latest Debian repository.
Unfortunately, I can't answer your question right now, but I'll contact the related department and let them know about the issues...

Please excuse the problems setting up a BeeGFS mirror!


Thanks a lot

Kind regards

Alex

***Please note our change of address below***

Alexander Lutz I System Engineer I p: +49 (0) 631 27757 6301
ThinkParQ GmbH I Trippstadter Strasse 113 I 67663 Kaiserslautern I Germany
CEO: Frank Herold I COB: Dr. Franz-Josef Pfreundt I Registered: Amtsgericht Kaiserslautern HRB 31565 I VAT-ID-No.:DE 292001792

Confidentiality Notice: This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential or proprietary information.


-----Ursprüngliche Nachricht-----
Von: fhgfs...@googlegroups.com <fhgfs...@googlegroups.com> Im Auftrag von Steffen Grunewald
Gesendet: Mittwoch, 27. Oktober 2021 09:55
An: BeeGFS user list <fhgfs...@googlegroups.com>
Betreff: Re: [beegfs-user] Missing timestamps in Debian repository

--
You received this message because you are subscribed to the Google Groups "beegfs-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fhgfs-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/fhgfs-user/20211027075456.jug6fujph65rhbtj%40eddie.aei.mpg.de.

[Steffen Grunewald]

On Wed, 2021-10-27 at 11:19:10 +0200, alexand...@thinkparq.com wrote:
> Hi Steffen.
>
> Thanks a lot for debugging and letting us know your observations regarding the latest Debian repository.
> Unfortunately, I can't answer your question right now, but I'll contact the related department and let them know about the issues...
>
> Please excuse the problems setting up a BeeGFS mirror!

Same behaviour with 7.2.5 - exactly limited to beegfs-client, beegfs-meta and beeond packages,
for all distros. I'd expect a web server to be kind of "filename agnostic", not to specially
handle such a set of files - so I suspect something must be present in the internals which
I'm unable to see.

Quite puzzled,

[Steffen Grunewald]

On Mon, 2021-11-08 at 14:50:33 +0100, Steffen Grunewald wrote:
> On Wed, 2021-10-27 at 11:19:10 +0200, alexand...@thinkparq.com wrote:
> > Hi Steffen.
> >
> > Thanks a lot for debugging and letting us know your observations regarding the latest Debian repository.
> > Unfortunately, I can't answer your question right now, but I'll contact the related department and let them know about the issues...
> >
> > Please excuse the problems setting up a BeeGFS mirror!
>
> Same behaviour with 7.2.5 - exactly limited to beegfs-client, beegfs-meta and beeond packages,
> for all distros. I'd expect a web server to be kind of "filename agnostic", not to specially
> handle such a set of files - so I suspect something must be present in the internals which
> I'm unable to see.

Getting closer? -

$ lynx -dump -head https://www.beegfs.io/release/beegfs_7.2.5/dists/deb9/amd64/beegfs-client_7.2.5_all.deb
HTTP/1.1 200 OK
Date: Mon, 08 Nov 2021 14:01:52 GMT
Server: Apache/2.4.29 (Ubuntu)
Set-Cookie: PHPSESSID=4pkuqms25cm1vofkf5g9vg9242; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Pragma: no-cache
Content-Disposition: attachment; filename=beegfs-client_7.2.5_all.deb
Content-Description: File Transfer
Content-Length: 316336
Connection: close
Content-Type: application/octet-stream


$ lynx -dump -head https://www.beegfs.io/release/beegfs_7.2.5/dists/deb9/amd64/beegfs-storage_7.2.5_amd64.deb
HTTP/1.1 200 OK
Date: Mon, 08 Nov 2021 14:02:05 GMT
Server: Apache/2.4.29 (Ubuntu)
Last-Modified: Fri, 05 Nov 2021 15:39:03 GMT
ETag: "ca978-5d00c715ff004"
Accept-Ranges: bytes
Content-Length: 829816
Connection: close
Content-Type: application/x-debian-package


$ lynx -dump -head https://www.beegfs.io/release/beegfs_7.2.5/dists/deb9/amd64/beegfs-meta_7.2.5_amd64.deb
HTTP/1.1 200 OK
Date: Mon, 08 Nov 2021 14:02:42 GMT
Server: Apache/2.4.29 (Ubuntu)
Set-Cookie: PHPSESSID=kopibde728qlq7ko5blfq0tnrf; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Pragma: no-cache
Content-Disposition: attachment; filename=beegfs-meta_7.2.5_amd64.deb
Content-Description: File Transfer
Content-Length: 1345786
Connection: close
Content-Type: application/octet-stream


$ lynx -dump -head https://www.beegfs.io/release/beegfs_7.2.5/dists/deb9/amd64/beegfs-mon_7.2.5_amd64.deb
HTTP/1.1 200 OK
Date: Mon, 08 Nov 2021 14:03:08 GMT
Server: Apache/2.4.29 (Ubuntu)
Last-Modified: Fri, 05 Nov 2021 15:39:03 GMT
ETag: "6f968-5d00c71608c44"
Accept-Ranges: bytes
Content-Length: 457064
Connection: close
Content-Type: application/x-debian-package


1st and 3rd package affected: come with an expired cookie, cache-control, and "octet-stream" content-type, while
2nd and 4th unaffected: come with a last-modified header and correct content-type.
Index "files" appear to be generated dynamically, and reflect the true timestamps for all files (AFAICT), thus
the underlying filesystem structure seems to be correct.
This cannot happen by accident, there must be server-side rules that single out some of the packages - and,
perhaps, a reason for them.
Does this ring a bell with someone, eventually?


Cheers,
Steffen