yum image creating failed on suse docker (probably on all suse)
133 views
Skip to first unread message
Roman Grigoryev
May 28, 2018, 6:45:32 AM5/28/18
to kiwi
Hi,
I tried to prepare oem image on suse leap 43.2 in docker as workaround of centos dependency issues.
But it failed with output:
----------------------------------------------------
Installing : 1:grub2-tools-extra-2.02-0.65.el7.centos.2.x86_64 143/147
Installing : 1:grub2-efi-x64-2.02-0.65.el7.centos.2.x86_64 144/147
Installing : shim-x64-12-1.el7.centos.x86_64 145/147
Installing : yum-plugin-priorities-1.1.31-42.el7.noarch 146/147 ]: 10:42:19 | bootstrap:
Installing : 1:grub2-efi-x64-modules-2.02-0.65.el7.centos.2.noarch 147/147 ]: 10:42:20 | bootstrap:
[ INFO ]: Processing: [########################################] 100%
[ ERROR ]: 10:42:20 | KiwiBootStrapPhaseFailed: Bootstrap package installation failed: Main config did not have a requires_policy attr. before setopt
Traceback (most recent call last):
File "/usr/bin/yum", line 29, in <module>
yummain.user_main(sys.argv[1:], exit_code=True)
File "/usr/share/yum-cli/yummain.py", line 288, in user_main
errcode = main(args)
File "/usr/share/yum-cli/yummain.py", line 223, in main
return_code = base.doTransaction()
File "/usr/share/yum-cli/cli.py", line 599, in doTransaction
resultobject = self.runTransaction(cb=cb)
File "/usr/lib64/python2.7/site-packages/yum/__init__.py", line 1565, in runTransaction
self.rpmdb.dropCachedDataPostTransaction(list(self.tsInfo))
File "/usr/lib64/python2.7/site-packages/yum/rpmsack.py", line 329, in dropCachedDataPostTransaction
self._cached_rpmdb_mtime = os.path.getmtime(rpmdbfname)
File "/usr/lib64/python2.7/genericpath.py", line 62, in getmtime
return os.stat(filename).st_mtime
OSError: [Errno 2] No such file or directory: '/tmp/74-oem/var/lib/rpm/Packages'
[ INFO ]: 10:42:20 | Cleaning up SystemPrepare instance
[ DEBUG ]: 10:42:20 | EXEC: [mountpoint /tmp/74-oem/var/cache/kiwi]
[ DEBUG ]: 10:42:20 | EXEC: [mountpoint /tmp/74-oem/var/cache/kiwi]
[ DEBUG ]: 10:42:20 | EXEC: [umount -l /tmp/74-oem/var/cache/kiwi]
[ DEBUG ]: 10:42:20 | EXEC: [mountpoint /tmp/74-oem/sys]
--------------------------------------------------
In same time config.xml works ok on centos. Probably it could because of 'yum' difference or
Installing : 1:grub2-efi-x64-2.02-0.65.el7.centos.2.x86_64 144/147
Installing : shim-x64-12-1.el7.centos.x86_64 145/147
Installing : yum-plugin-priorities-1.1.31-42.el7.noarch 146/147 ]: 10:42:19 | bootstrap:
Installing : 1:grub2-efi-x64-modules-2.02-0.65.el7.centos.2.noarch 147/147 ]: 10:42:20 | bootstrap:
[ INFO ]: Processing: [########################################] 100%
[ ERROR ]: 10:42:20 | KiwiBootStrapPhaseFailed: Bootstrap package installation failed: Main config did not have a requires_policy attr. before setopt
Traceback (most recent call last):
File "/usr/bin/yum", line 29, in <module>
yummain.user_main(sys.argv[1:], exit_code=True)
File "/usr/share/yum-cli/yummain.py", line 288, in user_main
errcode = main(args)
File "/usr/share/yum-cli/yummain.py", line 223, in main
return_code = base.doTransaction()
File "/usr/share/yum-cli/cli.py", line 599, in doTransaction
resultobject = self.runTransaction(cb=cb)
File "/usr/lib64/python2.7/site-packages/yum/__init__.py", line 1565, in runTransaction
self.rpmdb.dropCachedDataPostTransaction(list(self.tsInfo))
File "/usr/lib64/python2.7/site-packages/yum/rpmsack.py", line 329, in dropCachedDataPostTransaction
self._cached_rpmdb_mtime = os.path.getmtime(rpmdbfname)
File "/usr/lib64/python2.7/genericpath.py", line 62, in getmtime
return os.stat(filename).st_mtime
OSError: [Errno 2] No such file or directory: '/tmp/74-oem/var/lib/rpm/Packages'
[ INFO ]: 10:42:20 | Cleaning up SystemPrepare instance
[ DEBUG ]: 10:42:20 | EXEC: [mountpoint /tmp/74-oem/var/cache/kiwi]
[ DEBUG ]: 10:42:20 | EXEC: [mountpoint /tmp/74-oem/var/cache/kiwi]
[ DEBUG ]: 10:42:20 | EXEC: [umount -l /tmp/74-oem/var/cache/kiwi]
[ DEBUG ]: 10:42:20 | EXEC: [mountpoint /tmp/74-oem/sys]
--------------------------------------------------
In same time config.xml works ok on centos. Probably it could because of 'yum' difference or
or docker specifics.
I create docker via cmd
`docker run --privileged=true -d -it --name kiwi_ready -v /mnt:/mnt:rw,shared -v /root/src:/root/src:rw,shared -v /local:/local:rw,shared kiwi-installed_all`
`docker run --privileged=true -d -it --name kiwi_ready -v /mnt:/mnt:rw,shared -v /root/src:/root/src:rw,shared -v /local:/local:rw,shared kiwi-installed_all`
--
Thanks,
Roman
Marcus Schäfer
May 28, 2018, 11:53:52 AM5/28/18
to 'Roman Grigoryev' via kiwi
from yum. Have you checked the files in /etc/yum to be valid and compatible
with the version invoked here ?
Regards,
Marcus
--
Public Key available via: https://keybase.io/marcus_schaefer/key.asc
keybase search marcus_schaefer
-------------------------------------------------------
Marcus Schäfer (Res. & Dev.) SUSE Linux GmbH
Tel: 0911-740 53 0 Maxfeldstrasse 5
FAX: 0911-740 53 479 D-90409 Nürnberg
HRB: 21284 (AG Nürnberg) Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton
http://www.suse.de
-------------------------------------------------------
Roman Grigoryev
May 28, 2018, 1:02:35 PM5/28/18
to kiwi
I suppose it should be ok, I installed it via zypper and it is not failed for simple operation.
How could I check it in details?
Sorry, I don't know, how deep yum integrated to suse ecosystem.
Could I use any test from kiwi suite for checking it?
--
Thanks,
Roman
Marcus Schäfer
May 29, 2018, 3:28:41 AM5/29/18
to 'Roman Grigoryev' via kiwi
Hi,
> I suppose it should be ok, I installed it via zypper and it is not
> failed for simple operation.
ok, I'm using:
yum-3.4.3-21.5.x86_64
which version have you installed ?
> How could I check it in details?
The error said "Main config did not have a requires_policy attr" So I would
look up the yum docs for requires_policy and check what was setup on
the system
> Sorry, I don't know, how deep yum integrated to suse ecosystem.
Me neither, suse is not a user of yum. I'd say it was packaged but how
far it was tested is unknown to me too
> Could I use any test from kiwi suite for checking it?
Yes we have a centos example description here:
git clone https://github.com/SUSE/kiwi-descriptions.git
kiwi-ng-3 --type vmx system build \
--description kiwi-descriptions/centos/x86_64/centos-07.0-JeOS \
--target-dir /tmp/mytest/
That build test worked for me so far
Hope that helps
> I suppose it should be ok, I installed it via zypper and it is not
> failed for simple operation.
yum-3.4.3-21.5.x86_64
which version have you installed ?
> How could I check it in details?
look up the yum docs for requires_policy and check what was setup on
the system
> Sorry, I don't know, how deep yum integrated to suse ecosystem.
far it was tested is unknown to me too
> Could I use any test from kiwi suite for checking it?
git clone https://github.com/SUSE/kiwi-descriptions.git
kiwi-ng-3 --type vmx system build \
--description kiwi-descriptions/centos/x86_64/centos-07.0-JeOS \
--target-dir /tmp/mytest/
That build test worked for me so far
Hope that helps
Roman Grigoryev
May 29, 2018, 4:26:49 AM5/29/18
to kiwi-...@googlegroups.com
Hi,
>ok, I'm using:
>
> yum-3.4.3-21.5.x86_64
More or less same, my versions:
------------
# yum --version
3.4.3
Installed: rpm-4.14.1-lp150.8.3.x86_64 at 2018-05-16 09:10
Built : https://bugs.opensuse.org at 2018-05-12 21:17
Committed: m...@suse.de at 2018-05-07
Installed: yum-3.4.3-lp150.3.4.x86_64 at 2018-05-27 19:24
Built : https://bugs.opensuse.org at 2017-11-23 12:00
Committed: rbr...@suse.com at 2017-11-23
----------------
>The error said "Main config did not have a requires_policy attr" So I would
>look up the yum docs for requires_policy and check what was setup on
>the system
I added that option and played with other possible vales - no any changes.
I have seen the same error messages in other cases when kiwi generation failed.
Probably it is not the root case and just pointer which stage failed.
> kiwi-ng-3 --type vmx system build \
> --description kiwi-descriptions/centos/x86_64/centos-07.0-JeOS \
> --target-dir /tmp/mytest/
>That build test worked for me so far
----------------
Trying other mirror.
http://bo.mirror.garr.it/1/slc/centos/7.2/os/x86_64/Packages/yum-3.4.3-154.el7.centos.noarch.rpm:
[Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404
Not Fo
und"
Trying other mirror.
Error Downloading Packages:
tzdata-2017b-1.el7.noarch: failure:
Packages/tzdata-2017b-1.el7.noarch.rpm from centos: [Errno 256] No
more mirrors to try.
bash-4.2.46-28.el7.x86_64: failure:
Packages/bash-4.2.46-28.el7.x86_64.rpm from centos: [Errno 256] No
more mirrors to try.
nss-softokn-freebl-3.28.3-6.el7.x86_64: failure:
Packages/nss-softokn-freebl-3.28.3-6.el7.x86_64.rpm from centos:
[Errno 256] No more mirrors to try.
--------------
--
Thanks,
Roman
On Tue, May 29, 2018 at 10:28 AM, Marcus Schäfer <m...@suse.de> wrote:
> Hi,
>
>> I suppose it should be ok, I installed it via zypper and it is not
>> failed for simple operation.
>
> ok, I'm using:
>
> yum-3.4.3-21.5.x86_64
>
> which version have you installed ?
>
>> How could I check it in details?
>
> The error said "Main config did not have a requires_policy attr" So I would
> look up the yum docs for requires_policy and check what was setup on
> the system
>
>> Sorry, I don't know, how deep yum integrated to suse ecosystem.
>
> Me neither, suse is not a user of yum. I'd say it was packaged but how
> far it was tested is unknown to me too
>
>> Could I use any test from kiwi suite for checking it?
>
> Yes we have a centos example description here:
>
> git clone https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_SUSE_kiwi-2Ddescriptions.git&d=DwIFaQ&c=IGDlg0lD0b-nebmJJ0Kp8A&r=epXg8kRZ6MpUqqxlxtBKw2dkrgtEPIyPW8aTNz7wCCI&m=103mVlnprKk1516sYE2y3-qLGEXitv6Vbno_uPUNass&s=XNmMMjuhe5Y_SjkrPC-zOYGT2KT5gUUv3HTNETAiH10&e=
> -------------------------------------------------------
>
> --
> You received this message because you are subscribed to the Google Groups "kiwi" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kiwi-images...@googlegroups.com.
> For more options, visit https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_d_optout&d=DwIFaQ&c=IGDlg0lD0b-nebmJJ0Kp8A&r=epXg8kRZ6MpUqqxlxtBKw2dkrgtEPIyPW8aTNz7wCCI&m=103mVlnprKk1516sYE2y3-qLGEXitv6Vbno_uPUNass&s=3m4k282Vx8hh0x5PxyHj2nX3avAA7jCwdPP3mxhXtIs&e=.
--
--
Thanks,
Roman
>ok, I'm using:
>
> yum-3.4.3-21.5.x86_64
------------
# yum --version
3.4.3
Installed: rpm-4.14.1-lp150.8.3.x86_64 at 2018-05-16 09:10
Built : https://bugs.opensuse.org at 2018-05-12 21:17
Committed: m...@suse.de at 2018-05-07
Installed: yum-3.4.3-lp150.3.4.x86_64 at 2018-05-27 19:24
Built : https://bugs.opensuse.org at 2017-11-23 12:00
Committed: rbr...@suse.com at 2017-11-23
----------------
>The error said "Main config did not have a requires_policy attr" So I would
>look up the yum docs for requires_policy and check what was setup on
>the system
I have seen the same error messages in other cases when kiwi generation failed.
Probably it is not the root case and just pointer which stage failed.
> kiwi-ng-3 --type vmx system build \
> --description kiwi-descriptions/centos/x86_64/centos-07.0-JeOS \
> --target-dir /tmp/mytest/
>That build test worked for me so far
Trying other mirror.
http://bo.mirror.garr.it/1/slc/centos/7.2/os/x86_64/Packages/yum-3.4.3-154.el7.centos.noarch.rpm:
[Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404
Not Fo
und"
Trying other mirror.
Error Downloading Packages:
tzdata-2017b-1.el7.noarch: failure:
Packages/tzdata-2017b-1.el7.noarch.rpm from centos: [Errno 256] No
more mirrors to try.
bash-4.2.46-28.el7.x86_64: failure:
Packages/bash-4.2.46-28.el7.x86_64.rpm from centos: [Errno 256] No
more mirrors to try.
nss-softokn-freebl-3.28.3-6.el7.x86_64: failure:
Packages/nss-softokn-freebl-3.28.3-6.el7.x86_64.rpm from centos:
[Errno 256] No more mirrors to try.
--------------
--
Thanks,
Roman
On Tue, May 29, 2018 at 10:28 AM, Marcus Schäfer <m...@suse.de> wrote:
> Hi,
>
>> I suppose it should be ok, I installed it via zypper and it is not
>> failed for simple operation.
>
> ok, I'm using:
>
> yum-3.4.3-21.5.x86_64
>
> which version have you installed ?
>
>> How could I check it in details?
>
> The error said "Main config did not have a requires_policy attr" So I would
> look up the yum docs for requires_policy and check what was setup on
> the system
>
>> Sorry, I don't know, how deep yum integrated to suse ecosystem.
>
> Me neither, suse is not a user of yum. I'd say it was packaged but how
> far it was tested is unknown to me too
>
>> Could I use any test from kiwi suite for checking it?
>
> Yes we have a centos example description here:
>
>
> kiwi-ng-3 --type vmx system build \
> --description kiwi-descriptions/centos/x86_64/centos-07.0-JeOS \
> --target-dir /tmp/mytest/
>
> That build test worked for me so far
>
> Hope that helps
>
> Regards,
> Marcus
> --
> Public Key available via: https://urldefense.proofpoint.com/v2/url?u=https-3A__keybase.io_marcus-5Fschaefer_key.asc&d=DwIFaQ&c=IGDlg0lD0b-nebmJJ0Kp8A&r=epXg8kRZ6MpUqqxlxtBKw2dkrgtEPIyPW8aTNz7wCCI&m=103mVlnprKk1516sYE2y3-qLGEXitv6Vbno_uPUNass&s=jV3UXGu3dq0f1dRTeyV_zX6mLEWJGGwHuXl6oVnN-Gs&e=
> kiwi-ng-3 --type vmx system build \
> --description kiwi-descriptions/centos/x86_64/centos-07.0-JeOS \
> --target-dir /tmp/mytest/
>
> That build test worked for me so far
>
> Hope that helps
>
> Regards,
> Marcus
> --
> keybase search marcus_schaefer
> -------------------------------------------------------
> Marcus Schäfer (Res. & Dev.) SUSE Linux GmbH
> Tel: 0911-740 53 0 Maxfeldstrasse 5
> FAX: 0911-740 53 479 D-90409 Nürnberg
> HRB: 21284 (AG Nürnberg) Germany
> GF: Felix Imendörffer, Jane Smithard, Graham Norton
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.suse.de&d=DwIFaQ&c=IGDlg0lD0b-nebmJJ0Kp8A&r=epXg8kRZ6MpUqqxlxtBKw2dkrgtEPIyPW8aTNz7wCCI&m=103mVlnprKk1516sYE2y3-qLGEXitv6Vbno_uPUNass&s=vaCWdi8xWtQvLEajyecmKhKUxXtznMld-XG0vOjEkto&e=
> -------------------------------------------------------
> Marcus Schäfer (Res. & Dev.) SUSE Linux GmbH
> Tel: 0911-740 53 0 Maxfeldstrasse 5
> FAX: 0911-740 53 479 D-90409 Nürnberg
> HRB: 21284 (AG Nürnberg) Germany
> GF: Felix Imendörffer, Jane Smithard, Graham Norton
> -------------------------------------------------------
>
> --
> You received this message because you are subscribed to the Google Groups "kiwi" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kiwi-images...@googlegroups.com.
> For more options, visit https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_d_optout&d=DwIFaQ&c=IGDlg0lD0b-nebmJJ0Kp8A&r=epXg8kRZ6MpUqqxlxtBKw2dkrgtEPIyPW8aTNz7wCCI&m=103mVlnprKk1516sYE2y3-qLGEXitv6Vbno_uPUNass&s=3m4k282Vx8hh0x5PxyHj2nX3avAA7jCwdPP3mxhXtIs&e=.
--
--
Thanks,
Roman
Marcus Schäfer
May 29, 2018, 4:48:12 AM5/29/18
to 'Roman Grigoryev' via kiwi
Hi,
> > kiwi-ng-3 --type vmx system build \
> > --description kiwi-descriptions/centos/x86_64/centos-07.0-JeOS \
> > --target-dir /tmp/mytest/
> >That build test worked for me so far
>
> ----------------
> Trying other mirror.
> http://bo.mirror.garr.it/1/slc/centos/7.2/os/x86_64/Packages/yum-3.4.3-154.el7.centos.noarch.rpm:
> [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404
> Not Fo
> und"
> Trying other mirror.
>
>
> Error Downloading Packages:
> tzdata-2017b-1.el7.noarch: failure:
> Packages/tzdata-2017b-1.el7.noarch.rpm from centos: [Errno 256] No
> more mirrors to try.
Just builds here with the original repo setup:
http://bo.mirror.garr.it/1/slc/centos/7.2/os/x86_64
http://dl.fedoraproject.org/pub/epel/7/x86_64
Can you reach those from your network ?
Regards,
Marcus
--
Public Key available via: https://keybase.io/marcus_schaefer/key.asc
-------------------------------------------------------
> > kiwi-ng-3 --type vmx system build \
> > --description kiwi-descriptions/centos/x86_64/centos-07.0-JeOS \
> > --target-dir /tmp/mytest/
> >That build test worked for me so far
>
> ----------------
> Trying other mirror.
> http://bo.mirror.garr.it/1/slc/centos/7.2/os/x86_64/Packages/yum-3.4.3-154.el7.centos.noarch.rpm:
> [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404
> Not Fo
> und"
> Trying other mirror.
>
>
> Error Downloading Packages:
> tzdata-2017b-1.el7.noarch: failure:
> Packages/tzdata-2017b-1.el7.noarch.rpm from centos: [Errno 256] No
> more mirrors to try.
http://bo.mirror.garr.it/1/slc/centos/7.2/os/x86_64
http://dl.fedoraproject.org/pub/epel/7/x86_64
Can you reach those from your network ?
Regards,
Marcus
--
Public Key available via: https://keybase.io/marcus_schaefer/key.asc
keybase search marcus_schaefer
-------------------------------------------------------
Marcus Schäfer (Res. & Dev.) SUSE Linux GmbH
Tel: 0911-740 53 0 Maxfeldstrasse 5
FAX: 0911-740 53 479 D-90409 Nürnberg
HRB: 21284 (AG Nürnberg) Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton
http://www.suse.de
-------------------------------------------------------
Marcus Schäfer (Res. & Dev.) SUSE Linux GmbH
Tel: 0911-740 53 0 Maxfeldstrasse 5
FAX: 0911-740 53 479 D-90409 Nürnberg
HRB: 21284 (AG Nürnberg) Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton
-------------------------------------------------------
David Cassany
Jul 23, 2018, 9:19:45 AM7/23/18
to kiwi-...@googlegroups.com
Hi Grigoryev,
I am facing the same issue here and finally understood what is going on.
Hopefully it won't be too late.
I am pretty sure the issue is related to the RPM database location. I am also
having this issue in a Leap 15 host trying to build a CentOS 7 image. In
recent versions openSUSE updated the RPM database location in line with
upstream changes, the database location is defined in /usr/lib/rpm/macros thus
I assume this is system wide for both, yum and zypper. I believe that the yum
package from opensuse is also parsing this rpm config file and thus placing
database location in a different place of what yum inside CentOS expect
(<imageroot>/usr/lib/rpm/macros defines a different rpm database location).
See yum complaints about this:
> No such file or directory:
> '/tmp/74-oem/var/lib/rpm/Packages'
This is old (in still current in case of CentOS) default package database
location. The odd part is that it fails in bootstrap phase which could also
point to a YUM issue or bug having this path hardcoded somewhere else beyond /
usr/lib/rpm/macros, otherwise it would not have failed at this point.
The only ugly workaround I have for now is just copying the database the have
been initialized in a different location to the expected one and then rerun
the build with the flag --allow-existing-root to let KIWI reuse the buildroot
created during the first attempt.
To successfully build I did:
$ kiwi system build --description <pathtocentosimage> --target-dir
<destinationdir>
$ cp -rvp <destinationdir>/build/image-root/usr/lib/sysimage/rpm/*
<destinationdir>/build/image-root/var/lib/rpm
$ kiwi system build --description <pathtocentosimage> --target-dir
<destinationdir> --allow-existing-root
I'll dig a bit more to see if there is something that needs to be done from
KIWI side to fix it, but it looks like the issue might go beyond the KIWI
scope.
Regards,
David
I am facing the same issue here and finally understood what is going on.
Hopefully it won't be too late.
I am pretty sure the issue is related to the RPM database location. I am also
having this issue in a Leap 15 host trying to build a CentOS 7 image. In
recent versions openSUSE updated the RPM database location in line with
upstream changes, the database location is defined in /usr/lib/rpm/macros thus
I assume this is system wide for both, yum and zypper. I believe that the yum
package from opensuse is also parsing this rpm config file and thus placing
database location in a different place of what yum inside CentOS expect
(<imageroot>/usr/lib/rpm/macros defines a different rpm database location).
See yum complaints about this:
> No such file or directory:
> '/tmp/74-oem/var/lib/rpm/Packages'
location. The odd part is that it fails in bootstrap phase which could also
point to a YUM issue or bug having this path hardcoded somewhere else beyond /
usr/lib/rpm/macros, otherwise it would not have failed at this point.
The only ugly workaround I have for now is just copying the database the have
been initialized in a different location to the expected one and then rerun
the build with the flag --allow-existing-root to let KIWI reuse the buildroot
created during the first attempt.
To successfully build I did:
$ kiwi system build --description <pathtocentosimage> --target-dir
<destinationdir>
$ cp -rvp <destinationdir>/build/image-root/usr/lib/sysimage/rpm/*
<destinationdir>/build/image-root/var/lib/rpm
$ kiwi system build --description <pathtocentosimage> --target-dir
<destinationdir> --allow-existing-root
I'll dig a bit more to see if there is something that needs to be done from
KIWI side to fix it, but it looks like the issue might go beyond the KIWI
scope.
Regards,
David
Roman Grigoryev
Jul 23, 2018, 9:29:22 AM7/23/18
to kiwi-...@googlegroups.com
Thank you!
It's it time :)
--
--
Thanks,
Roman
It's it time :)
> --
> You received this message because you are subscribed to the Google Groups "kiwi" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kiwi-images...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
> You received this message because you are subscribed to the Google Groups "kiwi" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kiwi-images...@googlegroups.com.
--
--
Thanks,
Roman
Reply all
Reply to author
Forward
0 new messages