Use rsync to update the local mirror of debian with the downloaded dvd files.

[Hongyi Zhao]

Hi all,

Currently, I've a local Debian mirror for stable/squeeze, and I also have
the up-to-date DVD iso files for this distribution. See below for detail:

1- The local mirror of debian is located at the following location on my
computer:

werner@debian-asus:~/software/LocalRepo/apt-mirror/mirror/
mirror.bjtu.edu.cn/debian$ ls
dists pool

2- The DVD iso files for this distribution are located at the following
location on my computer:

werner@debian-asus:~/software/LocalRepo/jigdo/iso-dvd/stable$ ls
debian-6.0.6-amd64-DVD-1.iso debian-6.0.6-amd64-DVD-5.iso
debian-6.0.6-amd64-DVD-2.iso debian-6.0.6-amd64-DVD-6.iso
debian-6.0.6-amd64-DVD-3.iso debian-6.0.6-amd64-DVD-7.iso
debian-6.0.6-amd64-DVD-4.iso debian-6.0.6-amd64-DVD-8.iso

Now, I want to use these dvd files to update the local mirror of debian,
all of the things need to do is sync the newer files in each dists and
pool directories on the DVD files with the local mirror of debian. So I
use the following script named rsync_localrepo.sh for this purpose:

werner@debian-asus:~$ cat rsync_localrepo.sh
#!/bin/bash

if [ -d /media/cdrom ]; then sudo umount /media/cdrom; fi

for i in *iso
do
sudo mount -o loop -t iso9660 /home/werner/software/LocalRepo/jigdo/iso-
dvd/stable/${i} /media/cdrom
rsync -avuP --temp-dir=/tmp /media/cdrom/dists/ /home/werner/software/
LocalRepo/apt-mirror/mirror/mirror.bjtu.edu.cn/debian/dists/
rsync -avuP --temp-dir=/tmp /media/cdrom/pool/ /home/werner/software/
LocalRepo/apt-mirror/mirror/mirror.bjtu.edu.cn/debian/pool/
sudo umount /media/cdrom
done

But, when I run this script, I meet the following issues:

---------------
werner@debian-asus:~$ ./rsync_localrepo.sh
umount: /media/cdrom: not mounted
Usage: mount -V : print version
mount -h : print this help
mount : list mounted filesystems
mount -l : idem, including volume labels
So far the informational part. Next the mounting.
The command is `mount [-t fstype] something somewhere'.
Details found in /etc/fstab may be omitted.
mount -a [-t|-O] ... : mount all stuff from /etc/fstab
mount device : mount device at the known place
mount directory : mount known device here
mount -t type dev dir : ordinary mount command
Note that one does not really mount a device, one mounts
a filesystem (of the given type) found on the device.
One can also mount an already visible directory tree elsewhere:
mount --bind olddir newdir
or move a subtree:
mount --move olddir newdir
One can change the type of mount containing the directory dir:
mount --make-shared dir
mount --make-slave dir
mount --make-private dir
mount --make-unbindable dir
One can change the type of all the mounts in a mount subtree
containing the directory dir:
mount --make-rshared dir
mount --make-rslave dir
mount --make-rprivate dir
mount --make-runbindable dir
A device can be given by name, say /dev/hda1 or /dev/cdrom,
or by label, using -L label or by uuid, using -U uuid .
Other options: [-nfFrsvw] [-o options] [-p passwdfd].
For many more details, say man 8 mount .
sending incremental file list
rsync: change_dir "/media/cdrom/dists" failed: No such file or directory
(2)

sent 12 bytes received 12 bytes 48.00 bytes/sec
total size is 0 speedup is 0.00
rsync error: some files/attrs were not transferred (see previous errors)
(code 23) at main.c(1060) [sender=3.0.7]
sending incremental file list
rsync: change_dir "/media/cdrom/pool" failed: No such file or directory
(2)

sent 12 bytes received 12 bytes 48.00 bytes/sec
total size is 0 speedup is 0.00
rsync error: some files/attrs were not transferred (see previous errors)
(code 23) at main.c(1060) [sender=3.0.7]
umount: /media/cdrom: not mounted
---------------

Could someone please give me some hints on these issues? Thanks in
advance.

Regards

[Dave Gibson]

Hongyi Zhao <hongy...@gmail.com> wrote:
> Hi all,
>
> Currently, I've a local Debian mirror for stable/squeeze, and I also have
> the up-to-date DVD iso files for this distribution. See below for detail:
>
> 1- The local mirror of debian is located at the following location on my
> computer:
>
> werner@debian-asus:~/software/LocalRepo/apt-mirror/mirror/
> mirror.bjtu.edu.cn/debian$ ls
> dists pool
>
> 2- The DVD iso files for this distribution are located at the following
> location on my computer:
>
> werner@debian-asus:~/software/LocalRepo/jigdo/iso-dvd/stable$ ls
> debian-6.0.6-amd64-DVD-1.iso debian-6.0.6-amd64-DVD-5.iso
> debian-6.0.6-amd64-DVD-2.iso debian-6.0.6-amd64-DVD-6.iso
> debian-6.0.6-amd64-DVD-3.iso debian-6.0.6-amd64-DVD-7.iso
> debian-6.0.6-amd64-DVD-4.iso debian-6.0.6-amd64-DVD-8.iso
>
> Now, I want to use these dvd files to update the local mirror of debian,
> all of the things need to do is sync the newer files in each dists and
> pool directories on the DVD files with the local mirror of debian. So I
> use the following script named rsync_localrepo.sh for this purpose:
>
> werner@debian-asus:~$ cat rsync_localrepo.sh
> #!/bin/bash
>
> if [ -d /media/cdrom ]; then sudo umount /media/cdrom; fi

Directories aren't removed when the filesystems mounted on them are
unmounted. (Unless the mount is being managed by some sort of auto-
mounter.)

Also, if a filesystem is mounted there's a possibility that it is in
use so umount may fail.

>
> for i in *iso
> do
> sudo mount -o loop -t iso9660 /home/werner/software/LocalRepo/jigdo/iso-
> dvd/stable/${i} /media/cdrom

The path is in the wrong place.

for i in /home/werner/.../iso-dvd/stable/*iso
do
sudo ... "$i" /media/cdrom

In the absence of any matches for *iso in $PWD, $i will be '*iso' in the
loop. Parameter expansion would then give:

sudo mount -o loop -t iso9660 /home/.../stable/*iso /media/cdrom

Finally globbing, which would take place due to $i *NOT* *BEING* *QUOTED,*
would result in the shell invoking mount with a list of all the files
matching *iso in the .../stable/ directory:

sudo mount -o loop -t iso9660 \
/home/.../stable/debian-6.0.6-amd64-DVD-1.iso \
/home/.../stable/debian-6.0.6-amd64-DVD-2.iso \
/home/.../stable/debian-6.0.6-amd64-DVD-3.iso \
/home/.../stable/debian-6.0.6-amd64-DVD-4.iso \
/home/.../stable/debian-6.0.6-amd64-DVD-5.iso \
/home/.../stable/debian-6.0.6-amd64-DVD-6.iso \
/home/.../stable/debian-6.0.6-amd64-DVD-7.iso \
/home/.../stable/debian-6.0.6-amd64-DVD-8.iso \
/media/cdrom

The remaining errors are a consequence of the mount failure.

> rsync -avuP --temp-dir=/tmp /media/cdrom/dists/ /home/werner/software/
> LocalRepo/apt-mirror/mirror/mirror.bjtu.edu.cn/debian/dists/
> rsync -avuP --temp-dir=/tmp /media/cdrom/pool/ /home/werner/software/
> LocalRepo/apt-mirror/mirror/mirror.bjtu.edu.cn/debian/pool/
> sudo umount /media/cdrom
> done

[...]

Untested.

#! /bin/sh -

repo=$HOME/software/LocalRepo

isodir=$repo/jigdo/iso-dvd/stable

mirror=$repo/apt-mirror/mirror/mirror.bjtu.edu.cn/debian

isomount=/media/cdrom

if mount | grep -q " on $isomount\\>" ; then
echo "Mount point '$isomount' in use." 1>&2
exit 1
fi

if [ ! -d "$isodir" ]; then
echo "Missing DVD filesystem image directory: '$isodir'" 1>&2
exit 1
fi

mkdir -v -p -- "$mirror" || exit 1

sudo mkdir -v -p -- "$isomount" || exit 1

for i in "$isodir"/*.iso ; do
if sudo mount -o loop -t iso9660 "$i" "$isomount" ; then
rsync -avuP --temp-dir=/tmp \
"$isomount/dists" "$isomount/pool" "$mirror/"
rc=$?
# die if can't unmount -- avoid mount heaps.
sudo umount "$isomount" || exit 1
# Uncomment next line to treat rsync errors as fatal.
# [ $rc -eq 0 ] || exit 1
fi
done

[Thomas 'PointedEars' Lahn]

Hongyi Zhao wrote:

> Currently, I've a local Debian mirror for stable/squeeze, and I also have
> the up-to-date DVD iso files for this distribution. See below for detail:
>
> 1- The local mirror of debian is located at the following location on my
> computer:
>
> werner@debian-asus:~/software/LocalRepo/apt-mirror/mirror/
> mirror.bjtu.edu.cn/debian$ ls
> dists pool
>
> 2- The DVD iso files for this distribution are located at the following
> location on my computer:
>
> werner@debian-asus:~/software/LocalRepo/jigdo/iso-dvd/stable$ ls
> debian-6.0.6-amd64-DVD-1.iso debian-6.0.6-amd64-DVD-5.iso
> debian-6.0.6-amd64-DVD-2.iso debian-6.0.6-amd64-DVD-6.iso
> debian-6.0.6-amd64-DVD-3.iso debian-6.0.6-amd64-DVD-7.iso
> debian-6.0.6-amd64-DVD-4.iso debian-6.0.6-amd64-DVD-8.iso
>
> Now, I want to use these dvd files to update the local mirror of debian,
> all of the things need to do is sync the newer files in each dists and

> pool directories on the DVD files with the local mirror of debian. […]

Debian is a package-based GNU/Linux distribution that can be updated
automatically via the Internet. It does not appear to be very clever or
useful to synchronize whole DVD images when you can have your own mirror of
the repository with up-to-date packages of the distribution on an as-needed
basis virtually for free.

> So I use the following script named rsync_localrepo.sh for this purpose:
>
> werner@debian-asus:~$ cat rsync_localrepo.sh
> #!/bin/bash
>
> if [ -d /media/cdrom ]; then sudo umount /media/cdrom; fi
>
> for i in *iso

That should be *.iso

> do
> sudo mount -o loop -t iso9660 /home/werner/software/LocalRepo/jigdo/iso-
> dvd/stable/${i} /media/cdrom
> rsync -avuP --temp-dir=/tmp /media/cdrom/dists/ /home/werner/software/
> LocalRepo/apt-mirror/mirror/mirror.bjtu.edu.cn/debian/dists/
> rsync -avuP --temp-dir=/tmp /media/cdrom/pool/ /home/werner/software/
> LocalRepo/apt-mirror/mirror/mirror.bjtu.edu.cn/debian/pool/
> sudo umount /media/cdrom
> done
>
> But, when I run this script, I meet the following issues:
>
> ---------------
> werner@debian-asus:~$ ./rsync_localrepo.sh
> umount: /media/cdrom: not mounted

/media/cdrom is considered a directory (-d), but apparently it is not in
mtab(5), therefore umount(8) fails. The reason it is not in mtab(5) is
probably that /media is managed by another layer.

RTFM. ISTM that the order of arguments is wrong; in the manpage examples
for "THE LOOP DEVICE", the -o option always comes last, after the image
argument, the mount point argument and the -t option.

In addition to that, you could try to loop-mount the image into a
subdirectory of /mnt instead of media (which is usually not automatically
managed), but AISB it is far easier to set up a local repository mirror and
only update the packages that are needed by its users.

> sending incremental file list
> rsync: change_dir "/media/cdrom/dists" failed: No such file or directory
> (2)
>
> sent 12 bytes received 12 bytes 48.00 bytes/sec
> total size is 0 speedup is 0.00
> rsync error: some files/attrs were not transferred (see previous errors)
> (code 23) at main.c(1060) [sender=3.0.7]
> sending incremental file list
> rsync: change_dir "/media/cdrom/pool" failed: No such file or directory
> (2)
>
> sent 12 bytes received 12 bytes 48.00 bytes/sec
> total size is 0 speedup is 0.00
> rsync error: some files/attrs were not transferred (see previous errors)
> (code 23) at main.c(1060) [sender=3.0.7]

You are rsyncing regardless whether mount(8) worked. Use either `if…fi' or
`&&' to avoid that.

> umount: /media/cdrom: not mounted

You are umount(8)ing regardless whether any of the above worked.

> Could someone please give me some hints on these issues? Thanks in
> advance.

This is not the first time that you have showed yourself to be completely
clueless. Try to learn structured thinking and coding for a change. RTFM,
STFW.

Running the script's shell with the -x switch should help. Other than that,
none of this has to do with shell programming. And BTW, *G*NU is *N*ot
*U*nix.

--
PointedEars

Twitter: @PointedEars2
Please do not Cc: me. / Bitte keine Kopien per E-Mail.

[Lew Pitcher]

On Friday 26 October 2012 15:49, in comp.unix.shell, Point...@web.de
wrote:
[snip]

>
> RTFM. ISTM that the order of arguments is wrong; in the manpage examples
> for "THE LOOP DEVICE", the -o option always comes last, after the image
> argument, the mount point argument and the -t option.

I'm afraid that the mount(8) man page has led you astray

# ls -l
total 268404
drwxr-xr-x 2 lpitcher users 4096 2012-10-26 16:00 kav
-rw-r--r-- 1 lpitcher users 274565120 2012-05-30 15:17 kav_rescue_10.iso

# file kav_rescue_10.iso
kav_rescue_10.iso: ISO 9660 CD-ROM filesystem data UDF filesystem data
(unknown version, id 'NSR01') 'KRD10 ' (bootable)

# ls -l kav
total 0

# # ****** Note the order of the mount options below ******

# mount -t iso9660 -o ro,loop kav_rescue_10.iso kav && echo Success
Success

# ls -l kav
total 4
dr-xr-xr-x 1 root root 2048 2010-04-16 08:00 boot
-r-xr-xr-x 1 root root 0 2012-02-07 03:00 livecd
dr-xr-xr-x 1 root root 2048 2012-02-09 03:00 rescue

# umount kav && echo Umount succeeded
Umount succeeded

# ls -l kav
total 0

The mount above follows the 4th format of the mount(8) command shown in the
SYNOPSIS

MOUNT(8) Linux Programmer's Manual MOUNT(8)

NAME
mount - mount a file system

SYNOPSIS
mount [-lhV]

mount -a [-fFnrsvw] [-t vfstype] [-O optlist]
mount [-fnrsvw] [-o options [,...]] device | dir
mount [-fnrsvw] [-t vfstype] [-o options] device dir


[snip]

--
Lew Pitcher
"In Skills, We Trust"

[Thomas 'PointedEars' Lahn]

Lew Pitcher wrote:

> [Thomas 'PointedEars' Lahn wrote:]

> [snip]
>>
>> RTFM. ISTM that the order of arguments is wrong; in the manpage examples
>> for "THE LOOP DEVICE", the -o option always comes last, after the image
>> argument, the mount point argument and the -t option.
>
> I'm afraid that the mount(8) man page has led you astray

> […]

Yes, with

$ mount --version
mount from util-linux 2.20.1 (with libblkid and selinux support)

it works on GNU/Linux with options in either order indeed (as it should).
However, something *is* wrong here with the way the OP's mount(8) is
invoked, else it would not have displayed its command-line help.

[Hongyi Zhao]

On Fri, 26 Oct 2012 21:49:04 +0200, Thomas 'PointedEars' Lahn wrote:

> And BTW, *G*NU is *N*ot *U*nix.

What do you mean by saying the above sentence?

Thanks a lot, in fact, I've solved this issue by using the -x option
within bash.

Regards