retrieving date of remote object

[L A Walsh]

I'm downloading packages for my distro into a cache area on my local
system.

I note there is an option to set if-modified-since as well a to get
a cache-freshness value in getting back a response, but the cachefreshness
value seems to always return 1, as it is the number of seconds
old since the item was retrieved.

Thing it, I'd like to get the time& date stamp on the remote item.

Is there some header value that contains the time (in epoch seconds)
for the remote item?

I am also familiar with a variant of the 'GET' method -- 'mirror' which
will set the value of my copy to that on the remote -- so something
must have the actual time stamp of the remote object.

I'm just not sure what call or interface I need to use to get at
it directly.

While the 'mirror' option might be good in the long term, until I'm
sure all my bits and pieces fit together I'd like to be able to
display the local and remote time stamps on the object.

Anyone know the call/header value to use for getting at the remote
object's last-modified time directly?

Thanks!

[L A Walsh]

On 2019/10/21 17:39, Bradley Dean wrote:
>
> Using if-modified-since relies on the server you're querying supporting that
----
It "seems" it must support it separately, "I guess"...because (see below)...

>> Anyone know the call/header value to use for getting at the remote
>> object's last-modified time directly?
>>
>

> You can check what headers are available by using a HEAD request before you GET - then you can make the call to download or not.
>
I printed out all the headers -- unfortunately there was no
'last-modified'
value for the object (even though the directory listing for it shows a
timestamp). Nevertheless, somehow it has to know since the 'mirror' method
seems to work.

Something weird might be going on since there is a squid-proxy
between me
and the content. Very strange. Note that the mirror method works, but
when I
tried to do similar, manually, I only got that my local copy was 'fresh'
until
the cache expiration time had passed, then it downloaded it again.

But with the mirror method, it seems to only download it once and
sync with the remote. Strange.


One thing I was curious about -- when I call mirror (vs. get), I get
back
an HTTP::Response object. One of the fields of that object is the
'content' value. I'm wondering if that is read in dynamically if
'accessed', since I'm
pulling down RPM packages which can be rather large (the 10 largest packages
are over 1GB each!). It certainly wouldn't just store them in the response
object, by default (would it?), since if I had references to several objects
at the same time, in memory, that could be a huge memory load.

I can see I'm gonna want to find a way to filter some of these packages
before I mirror them since I don't really need the debuginfo for
multiple kernel
versions I'm not running nor various cross-compiler packages (Oi!)

Whether dynamic or not, guess I'll find out -- can't believe how huge
some of those packages are!

Thanks!