Debmirror

[Polyna-Maude Racicot-Summerside]

Hi !
I've did a mirror with debmirror.
All seem good when I look at the folders and files.
But when I do apt-update it complains about contents-amd64 file missing !?

Got some ideas?
--
Polyna-Maude R.-Summerside
-Be smart, Be wise, Support opensource development

[to...@tuxteam.de]

On Sat, Jun 05, 2021 at 02:34:12AM -0400, Polyna-Maude Racicot-Summerside wrote:
> Hi !
> I've did a mirror with debmirror.
> All seem good when I look at the folders and files.
> But when I do apt-update it complains about contents-amd64 file missing !?
>
> Got some ideas?

At the top level of each release there should be one file Contents-<arch>
(possibly gzipped) for each architecture you want to support. Have a look
at e.g. http://deb.debian.org/debian/dists/bullseye/main/

Cheers
- t

[Polyna-Maude Racicot-Summerside]

Hi,

I already know this one.
But there seem to be a problem with debmirror not copying some files.
I found a bug report relating to this.

I've now tried using apt-mirror.
I'll see if I get the same.

[Polyna-Maude Racicot-Summerside]

Hi,

On 2021-06-05 3:24 a.m., to...@tuxteam.de wrote:

I used to make my own mirror using aptly but as it need to sign with a
new key all the mirror it does, it does take lot of time and that's
excessive for the full debian repository with backports and source.

So I wanted a simple tool using debmirror.
As it seem to cause problem, I'll see what does apt-mirror does as a job.
If it doesn't work then I'll go back to aptly (this require modifying a
bit of the install procedure so that the installer recognize my own key).

[to...@tuxteam.de]

On Sat, Jun 05, 2021 at 03:27:52AM -0400, Polyna-Maude Racicot-Summerside wrote:
> Hi,
>
> On 2021-06-05 3:24 a.m., to...@tuxteam.de wrote:

[...]

> > At the top level of each release there should be one file Contents-<arch>

[...]

> I already know this one.
> But there seem to be a problem with debmirror not copying some files.
> I found a bug report relating to this.

Hm. Looking through the debmirror bugs, there are a couple of them which
might be candidates. But all of them look either fixed or suspiciously
old.

Is that Contents file missing in your mirror? What happens if you put
one in by hand?

> I've now tried using apt-mirror.
> I'll see if I get the same.

Yes, perhaps this is a way forward. Good luck :)

Cheers
- t

[Andrew M.A. Cater]

On Sat, Jun 05, 2021 at 02:34:12AM -0400, Polyna-Maude Racicot-Summerside wrote:

If none of the debmirror and similar work well, consider ftpsync.
It's extremely reliable, runs as one process and *just works* for
me and several others. It's written in shell script. It also provides
verbose login.

One config file will mirror a complete Debian mirror using rsync.
Because it's rsync based, you can rerun it if you need to so you can
run it several times to build up a mirror over the course of a week
to start with, for example.

I ran mine from a cron job four times a day - updates are now pushed
to me from a main Debian mirror.

I wrote this up for Planet Debian at:

http://flosslinuxblog.blogspot.com/2020/02/rebuilding-mirror-software-mirroring-of.html

and

http://flosslinuxblog.blogspot.com/2020/02/rebuilding-mirror-fedora-epel-mirroring.html

[I now only mirror Debian and Ubuntu - notably, it is recommended that you do
NOT mirror debian-security - having single known mirrors for these on the
Internet prevents you from accidentally mirroring out of date packages].

Hope this helps

Andy C.

[Francisco M Neto]

Hello!

On 6/5/21 4:27 AM, Polyna-Maude Racicot-Summerside wrote:
>>> I've did a mirror with debmirror.
>>> All seem good when I look at the folders and files.
>>> But when I do apt-update it complains about contents-amd64 file missing !?
>>>
>>> Got some ideas?
>>

> I already know this one.
> But there seem to be a problem with debmirror not copying some files.
> I found a bug report relating to this.
>
> I've now tried using apt-mirror.
> I'll see if I get the same.

You also might be interested in having a look at package aptly.
--
[]'s,

Francisco M Neto <fmn...@usp.br>

3E58 1655 9A3D 5D78 9F90
CFF1 D30B 1694 D692 FBF0

[Francisco M Neto]

Hello again,

On 6/5/21 4:33 AM, Polyna-Maude Racicot-Summerside wrote:
> I used to make my own mirror using aptly but as it need to sign with a
> new key all the mirror it does, it does take lot of time and that's
> excessive for the full debian repository with backports and source.

Eh, I hadn't seen this message when I sent my last reply. My apologies.

Maybe apt-mirror is the better alternative then.

Other sync programs might work too (ftpsync, rsync) but they might be
less effective since they don't know about debian mirrors.

[Andrew M.A. Cater]

On Thu, Jun 10, 2021 at 01:34:56PM -0300, Francisco M Neto wrote:
> Hello again,
>
> On 6/5/21 4:33 AM, Polyna-Maude Racicot-Summerside wrote:
> > I used to make my own mirror using aptly but as it need to sign with a
> > new key all the mirror it does, it does take lot of time and that's
> > excessive for the full debian repository with backports and source.
>
> Eh, I hadn't seen this message when I sent my last reply. My apologies.
>
> Maybe apt-mirror is the better alternative then.
>
> Other sync programs might work too (ftpsync, rsync) but they might be
> less effective since they don't know about debian mirrors.
>

Hi Francisco,

My advice to you would be the same as the advice I gave to Polyna: ftpsync
"just works" for most things and relies only on rsync.

It's what I and several others use - it's Debian native and is widely
understood and used.

All the very best,

Andy Cater

[Francisco M Neto]

On 6/10/21 4:39 PM, Andrew M.A. Cater wrote:
> My advice to you would be the same as the advice I gave to Polyna: ftpsync
> "just works" for most things and relies only on rsync.
>
> It's what I and several others use - it's Debian native and is widely
> understood and used.

Oh, I'm not looking for that. I was just trying to help the OP.

Cheers!

[C M Reinehr]

On 6/5/21 1:34 AM, Polyna-Maude Racicot-Summerside wrote:
> Hi !
> I've did a mirror with debmirror.
> All seem good when I look at the folders and files.
> But when I do apt-update it complains about contents-amd64 file missing !?
>
> Got some ideas?

I don't recall that you've ever mentioned why you want a mirror so this
suggestion may not be appropriate but I have used approx for a number of
years now with great success:

> Approx is an HTTP-based Debian archive server.  It fetches packages
> from remote repositories on demand, and caches them for local use.
>
> Approx saves time and network bandwidth if you need to install or
> upgrade Debian software for a number of machines on a local network. 
> Each package is downloaded from a remote site only once, regardless of
> how many local clients install it.  The approx cache typically
> requires a few gigabytes of disk space.
>
> Approx also simplifies the administration of client machines:
> repository locations need only be changed in approx’s configuration
> file, not in every client’s /etc/apt/sources.list file.
>
http://www.debianadmin.com/upgrade-multiple-debian-systems-with-approx.html

Sincerely,

CMR

Linux distribution Debian 10.3 "Buster"
----
“If you want total security, go to prison. There you’re fed,
clothed, given medical care, and so on. The only thing lacking … is
freedom.” ~ Dwight D. Eisenhower
------

[Polyna-Maude Racicot-Summerside]

Hi,

>> Approx is an HTTP-based Debian archive server.  It fetches packages
>> from remote repositories on demand, and caches them for local use.
>>
>> Approx saves time and network bandwidth if you need to install or
>> upgrade Debian software for a number of machines on a local network. 
>> Each package is downloaded from a remote site only once, regardless of
>> how many local clients install it.  The approx cache typically
>> requires a few gigabytes of disk space.
>>
>> Approx also simplifies the administration of client machines:
>> repository locations need only be changed in approx’s configuration
>> file, not in every client’s /etc/apt/sources.list file.
>>
> http://www.debianadmin.com/upgrade-multiple-debian-systems-with-approx.html
>
> Sincerely,
>
> CMR
>

Thanks, I'll see if this does what I want.
I need a local mirror on my external HD so I can easily install system
without the reliance on a Internet connection.

Also, I'm somewhat looking to offer a possible Debian mirror to the
community. I just need some research regarding how does configuration
work on my rented rack server.

Thanks