Re: Error with sync via ftp removing smoke files?
Bram
> Bram wrote:
>> Ran into a problem setting up smoking on one machine...
>>
>> Configured the smoke to download sources via ftp.
>> The machine is behind a firewall which doesn't seem to be
>> configured properly for ftp.
>> It was able to connect but all other operations timed out.
>>
>> When I ran:
>>
>> cd /tmp
>> /opt/perl/libs/local/bin/smokecurrent.sh -Dcc="ccache /opt/gcc/bin/gcc430"
>>
>> everything in /opt/perl/libs/local/bin got removed.
>
> That is not good. I am so sorry!
>
> I am looking into this matter, but cannot reproduce.
What is happening (I believe):
Test::Smoke::Syncer::new is called with sync_type forest.
Test::Smoke::Syncer::Forest::sync checks the fsync type which is ftp
and calls:
Test::Smoker::Syncer::new,
Then it calls: $syncer->sync (line 1436).
(Note that it does not check if the sync succeeds or fails.)
Then it sets the type to hardlink and calls ->sync to sync the inter
dir from the master dir (assuming I'm reading the code correctly):
Then it sets the type to hardlink and calls ->sync to sync the build
dir from the inter dir.
Test::Smoke::Syncer::Hardlink::sync calls:
$self->clean_from_directory( $source_dir ); (line 1269)
which is supposed to remove some files.
What might fix it (will test and report back):
a) check if the sync succeeded in forest before continuing,
b) error on the ftp/forest combination,
Something else I noticed:
Test::Smoke::Syncer::Rsync::sync does a chdir to the ddir
Test::Smoke::Syncer::Snapsh:sync creates the ddir if it doesn't exists
and does a chdir in _extract_snapshot (assuming it got that far)
Test::Smoke::Syncer::Copy::sync does a chdir to the ddir
Test::Smoke::Syncer::Hardlink::sync creates the ddir if it doesn't
exists (but doesn't do a chdir)
Test::Smoke::Syncer::FTP::sync doesn't do a chdir
Test::Smoke::Syncer::Forest::sync doesn't do a chdir
If the sync type is Forest in combination with rsync then a chdir to
the ddir happend.
If the sync type is Forest in combination with rsync then a chdir to
the ddir did not happen and that might leave it in a wrong directory.
> Can you try with snapshots from APC (I believe that works out of the
> box), that might be safer and is a lot faster.
It's set to Forest + rsync now (used a server in another network to
forward everything on port 80 to the rsync server of activestate)
Kind regards,
Bram
Abe Timmerman
> Ran into a problem setting up smoking on one machine...
>
> Configured the smoke to download sources via ftp.
> The machine is behind a firewall which doesn't seem to be configured
> properly for ftp.
> It was able to connect but all other operations timed out.
>
> When I ran:
>
> cd /tmp
> /opt/perl/libs/local/bin/smokecurrent.sh -Dcc="ccache
> /opt/gcc/bin/gcc430"
>
> everything in /opt/perl/libs/local/bin got removed.
That is not good. I am so sorry!
I am looking into this matter, but cannot reproduce.
Can you try with snapshots from APC (I believe that works out of the box),
that might be safer and is a lot faster.
Thanks for your interest and report!
Good luck,
Abe.
--
_ | "Pinky, are you pondering what I'm pondering?"
( ) | I think so, Brain, but can the Gummi Worms really live in peace with
X | the Marshmallow Chicks? (22 Sep 97 - The Pinky Protocol)
/ \ |
Bram
> Quoting Abe Timmerman <abe...@test-smoke.org>:
>
>> Bram wrote:
>>> Ran into a problem setting up smoking on one machine...
>>>
>>> Configured the smoke to download sources via ftp.
>>> The machine is behind a firewall which doesn't seem to be
>>> configured properly for ftp.
>>> It was able to connect but all other operations timed out.
>>>
>>> When I ran:
>>>
>>> cd /tmp
>>> /opt/perl/libs/local/bin/smokecurrent.sh -Dcc="ccache /opt/gcc/bin/gcc430"
>>>
>>> everything in /opt/perl/libs/local/bin got removed.
>>
>> That is not good. I am so sorry!
>>
>> I am looking into this matter, but cannot reproduce.
>
Patch that fixes this attached.
(No tests for it are added - can't come up with a good test case)
The problem was the cleanup part in FTPClient.pm.
The cleanup routine checks what files are in . and deletes them.
This works fine if at least one file is transfered (since that code
does a chdir) but if no files were transfered things gi boom, since .
is not the same as the ddir/localdir.
If you want to reproduce this then you need to establish a control
channel to an FTP server (port 21) but refuse a data channel (port 20).
I used:
iptables -A INPUT --proto tcp --sport ! 21 --source 204.244.102.6 -j REJECT
iptables -A OUTPUT --proto tcp --dport ! 21 --destination
204.244.102.6 -j REJECT
Where 204.244.102.6 is the address of the Activestate ftp server that
I used to sync with.
Kind regards,
Bram