NEWBIE: how exclude files in ftp
Atropo
windows server, but with the condition that the files do not have the
extension ".par" (partial or being processed). help mget is not very
helpful.
Atropo
> In article
> <ade391fd-f5d3-420d-90f0-22dbcd500...@k39g2000yqd.googlegroups.com>,
> files, and mget only the ones you want. 'Fraid the standard
> command-line ftp client doesn't have any sort of exclude option. From a
> GUI front-end like Interarchie on Macintosh, I can selected files in a
> directory, then unselect ones I don't want graphically and drag the
> selected files to my desktop to transfer them.
>
> But a specific exclude feature does not exist.
>
> --
> DeeDee, don't press that button! DeeDee! NO! Dee...
> [I filter all Goggle Groups posts, so any reply may be automatically ignored]
Thanks Michael, but this intended to be a script to run via crontab
on ciclycal basis.
but maybe you can help me to achieve other idea like.
create a list of files on remote (maybe excluding)
get this list ( don't know if can do it from local)
parse the list (as a here document)
but don't know the steps to accomplish this
John Kelly
wrote:
>> > Hi all, I'm on solaris 5.10 and trying to get several files from
>> > windows server, but with the condition that the files do not have the
>> > extension ".par" (partial or being processed). help mget is not very
>> > helpful.
>Thanks Michael, but this intended to be a script to run via crontab
>on ciclycal basis.
>but maybe you can help me to achieve other idea like.
>
>create a list of files on remote (maybe excluding)
>get this list ( don't know if can do it from local)
>parse the list (as a here document)
>
>but don't know the steps to accomplish this
The tool you need is wget. Its ftp globbing feature may not be good
enough for your purpose, but you can download a list from the server,
process it with grep to produce a new, restricted list, and use that
with wget.
--
Web mail, POP3, and SMTP
http://www.beewyz.com/freeaccounts.php
Radoulov, Dimitre
You can use Perl:
perl -MNet::FTP -e'
( $host, $user, $pass ) = @ARGV;
$ftp = Net::FTP->new($host) or die "$@\n";
$ftp->login( $user, $pass ) or die $ftp->message;
$ftp->binary;
@files = grep !/\.par/, $ftp->ls;
$ftp->get($_) for @files;
$ftp->quit or die $ftp->message;
' <host> <user> <pass>
Regards
Dimitre
Radoulov, Dimitre
> On 14/06/2010 15.47, Atropo wrote:
>> Hi all, I'm on solaris 5.10 and trying to get several files from
>> windows server, but with the condition that the files do not have the
>> extension ".par"
[...]
> perl -MNet::FTP -e'
> ( $host, $user, $pass ) = @ARGV;
> $ftp = Net::FTP->new($host) or die "$@\n";
> $ftp->login( $user, $pass ) or die $ftp->message;
> $ftp->binary;
> @files = grep !/\.par/, $ftp->ls;
It should be:
!/\.par$/
^
> $ftp->get($_) for @files;
> $ftp->quit or die $ftp->message;
> ' <host> <user> <pass>
Dimitre
Ben Finney
If installing programs onto the server is an option, you should consider
Rsync <URL:http://rsync.samba.org/download.html> instead of FTP. You'll
get much better control over which entries are transferred, and more
efficient transfers into the bargain.
--
\ “Facts do not cease to exist because they are ignored.” —Aldous |
`\ Huxley |
_o__) |
Ben Finney
Mart Frauenlob
> On Jun 14, 1:05 pm, Michael Vilain<vil...@NOspamcop.net> wrote:
>> Atropo<lxvasq...@gmail.com> wrote:
>>> Hi all, I'm on solaris 5.10 and trying to get several files from
>>> windows server, but with the condition that the files do not have the
>>> extension ".par" (partial or being processed). help mget is not very
>>> helpful.
>>
>> You can "mget *" and answer "N" to the .par files. Or get a list of
>> files, and mget only the ones you want. 'Fraid the standard
>> command-line ftp client doesn't have any sort of exclude option. From a
>> GUI front-end like Interarchie on Macintosh, I can selected files in a
>> directory, then unselect ones I don't want graphically and drag the
>> selected files to my desktop to transfer them.
>>
>> But a specific exclude feature does not exist.
>>
> Thanks Michael, but this intended to be a script to run via crontab
> on ciclycal basis.
> but maybe you can help me to achieve other idea like.
>
> create a list of files on remote (maybe excluding)
> get this list ( don't know if can do it from local)
> parse the list (as a here document)
>
> but don't know the steps to accomplish this
another option could be 'curl'.
It has many features useful for scripting that kind of thing.
1: You may fetch the remote listing with:
-l/--list-only
(FTP) When listing an FTP directory, this switch
forces a name-only view. Especially useful if you want to machine-parse
the contents of an FTP
directory since the normal directory view doesn't use a
standard look or format.
2: generate the transfer list with some like:
grep -v ".*.par" fetched_list >new_list
(don't know if -v is portable)
3: feed that new_list to curl.
'man curl' for details.
Hope it helps.
Best regards
Mart
Atropo
Thanks a lot Dimitre, althoug i don't know any perl i'm trying your
suggestion, I'm rtfm but it's overwhelming.
I tried $ftp->cd DIR/OTHERDIR/ANOTHER; and it's failing
Bareword found where operator expected at -e line 6, near "->cd DIR"
(Missing operator before DIR?)
beside this i have a question. in ksh i can get rid off extension
using ${file%.*}. how can i use this or something like in perl to
avoid getting the files associates to '.par' files.
in this list i only want to get the last four files
LOC755225
LOC755225.par
LOC705529
LOC705529.par
LOC776480
LOC776480.par
LOC786253
LOC786219
LOC786210
LOC786202
Radoulov, Dimitre
You could try something like this:
perl -MNet::FTP -le'
( $host, $user, $pass, $dir ) = @ARGV;
$ftp = Net::FTP->new($host) or die "$@\n";
$ftp->login( $user, $pass ) or die $ftp->message;
$ftp->cwd($dir) or die $ftp->message;
$ftp->binary;
/(.*)\.par$/ and $files_ko{$1} = 1
or push @files, $_
for $ftp->ls;
$files_ko{$_} or $ftp->get($_) for @files;
$ftp->quit or die $ftp->message;
' <host> <user> <user> <dir>
Regards
Dimitre
Atropo
I really appreciate your help Dimitre. it runs but it brought
everything. i run as it, 'coz i don't understand any after binary
command
Radoulov, Dimitre
> On Jun 15, 10:27 am, "Radoulov, Dimitre"<cichomit...@gmail.com>
> wrote:
>> On 15/06/2010 15.52, Atropo wrote:
>>
>>
>>
>>> On Jun 14, 4:51 pm, "Radoulov, Dimitre"<cichomit...@gmail.com> wrote:
>>>> On 14/06/2010 22.47, Radoulov, Dimitre wrote:
>>
>>>>> On 14/06/2010 15.47, Atropo wrote:
>>>>>> Hi all, I'm on solaris 5.10 and trying to get several files from
>>>>>> windows server, but with the condition that the files do not have the
>>>>>> extension ".par"
>>
>>>> [...]
>>
>>>>> perl -MNet::FTP -e'
>>>>> ( $host, $user, $pass ) = @ARGV;
>>>>> $ftp = Net::FTP->new($host) or die "$@\n";
>>>>> $ftp->login( $user, $pass ) or die $ftp->message;
>>>>> $ftp->binary;
>>>>> @files = grep !/\.par/, $ftp->ls;
[...]
[...]
> it runs but it brought
> everything. i run as it, 'coz i don't understand any after binary
> command
Hm,
could you post the output of the following script:
perl -MNet::FTP -e'
( $host, $user, $pass, $dir ) = @ARGV;
$ftp = Net::FTP->new($host) or die "$@\n";
$ftp->login( $user, $pass ) or die $ftp->message;
$ftp->cwd($dir) or die $ftp->message;
$ftp->binary;
for ($ftp->ls) {
printf "fname ->%s<-\n", $_;
/(.*)\.par$/ and printf "\$1 -->%s<--\n", $1;
}
$ftp->quit or die $ftp->message;
' <host> <user> <pass> <dir>
Regards
Dimitre
Atropo
changed par with tag
fname ->LOC758523948755225<-
fname ->LOC758523948755225.tag<-
$1 -->LOC758523948755225<--
fname ->LOC759375948805529<-
fname ->LOC759375948805529.tag<-
$1 -->LOC759375948805529<--
fname ->LOC760305248876480<-
fname ->LOC760305248876480.tag<-
$1 -->LOC760305248876480<--
fname ->LOC762093448986253<-
fname ->LOC762093848986219<-
fname ->LOC762093948986210<-
Atropo
Sorry dimitre, it works fine, was a typo of mine. but i still
don't understand
/(.*)\.tag$/ and $files_ko{$1} = 1
Atropo
if i want to delete the remote files i have transfer . could I use
@files or do all again like
/(.*)\.tag$/ and $files_ko{$1} = 1
or push @files, $_
for $ftp->ls;
$files_ko{$_} or $ftp->delete($_) for @files; <= only changed
the "get" for "delete"
Radoulov, Dimitre
[...]
>> it works fine, was a typo of mine. but i still
>> don't understand
>>
>> /(.*)\.tag$/ and $files_ko{$1} = 1
>> or push @files, $_
>> for $ftp->ls;
>> $files_ko{$_} or $ftp->get($_) for @files;
>
>
> if i want to delete the remote files i have transfer . could I use
> @files or do all again like
>
>
> /(.*)\.tag$/ and $files_ko{$1} = 1
> or push @files, $_
> for $ftp->ls;
> $files_ko{$_} or $ftp->delete($_) for @files;<= only changed
> the "get" for "delete"
You could use something like this:
perl -MNet::FTP -le'
( $host, $user, $pass, $dir ) = @ARGV;
$ftp = Net::FTP->new($host) or die "$@\n";
$ftp->login( $user, $pass ) or die $ftp->message;
$ftp->cwd($dir) or die $ftp->message;
$ftp->binary;
/(.*)\.tag$/ and $files_ko{$1} = 1
or push @files, $_
for $ftp->ls;
for (@files) {
$ftp->get($_) and $ftp->delete($_)
unless exists $files_ko{$_};
}
$ftp->quit or die $ftp->message;
' <host> <user> <pass> <dir>
Here is the explanation:
/(.*)\.tag$/ and $files_ko{$1} = 1 # If a filename matches the
# the pattern (.*)\.tag$,
# store the string before .tag
# in the hash table
# (an associative array)
# %files_ko as a key, the value
# (1) in this case[1]
# is irrelevant,
# but in Perl it is mandatory (it
# could be even undefined,
# but it should be explicitly
# mentioned (unlike awk).
or push @files, $_ # Otherwise, store the filename
# in the array @files
for $ftp->ls; # The ls method returns the list
# with the filenames.
for (@files) {
$ftp->get($_) and $ftp->delete($_) # Get and delete the file,
unless exists $files_ko{$_}; # unless such a key
# exists in the hash
# %files_ko.
HTH
Dimitre
[1] In my previous code both key and value was used and meaningful.
Atropo
Really thanks Dimitre, the script do exactly what i want. thanks again
for the explanation on how it works.
Atropo
One very last thing. how can i create a log file with the names of the
files i had fetched. and append to this file everytime this script
runs.
Radoulov, Dimitre
> On Jun 16, 8:52 am, Atropo<lxvasq...@gmail.com> wrote:
>> On Jun 16, 5:07 am, "Radoulov, Dimitre"<cichomit...@gmail.com> wrote:
>>
>>
>>
>>> On 15/06/2010 20.31, Atropo wrote:
>>
>>>> On Jun 15, 2:08 pm, Atropo<lxvasq...@gmail.com> wrote:
>>>>> On Jun 15, 11:03 am, "Radoulov, Dimitre"<cichomit...@gmail.com>
>>>>> wrote:
>>
>>>>>> On 15/06/2010 16.52, Atropo wrote:
>>
>>>>>>> On Jun 15, 10:27 am, "Radoulov, Dimitre"<cichomit...@gmail.com>
>>>>>>> wrote:
>>>>>>>> On 15/06/2010 15.52, Atropo wrote:
>>
>>>>>>>>> On Jun 14, 4:51 pm, "Radoulov, Dimitre"<cichomit...@gmail.com> wrote:
>>>>>>>>>> On 14/06/2010 22.47, Radoulov, Dimitre wrote:
>>
>>>>>>>>>>> On 14/06/2010 15.47, Atropo wrote:
>>>>>>>>>>>> Hi all, I'm on solaris 5.10 and trying to get several files from
>>>>>>>>>>>> windows server, but with the condition that the files do not have the
>>>>>>>>>>>> extension ".par"
[...]
> One very last thing. how can i create a log file with the names of the
> files i had fetched. and append to this file everytime this script
> runs.
perl >> ftp.log -MNet::FTP -le'
( $host, $user, $pass, $dir ) = @ARGV;
$ftp = Net::FTP->new($host) or die "$@\n";
$ftp->login( $user, $pass ) or die $ftp->message;
$ftp->cwd($dir) or die $ftp->message;
$ftp->binary;
/(.*)\.tag$/ and $files_ko{$1} = 1
or push @files, $_
for $ftp->ls;
for (@files) {
$ftp->get($_) and $ftp->delete($_) and
print unless exists $files_ko{$_};
}
$ftp->quit or die $ftp->message;
' <host> <user> <pass> <dir>
Regards
Dimitre
Atropo
before asking you i was trying this, according to tutorials. why it
did not work?
perl -MNet::FTP -le'
( $host, $user, $pass, $dir ) = @ARGV;
$my_file = "recibidos.txt";
OPEN(PLOT,">>$my_file") or die("The file cannot be opened!");
$ftp = Net::FTP->new($host) or die "$@\n";
$ftp->login( $user, $pass ) or die $ftp->message;
$ftp->cwd($dir) or die $ftp->message;
$ftp->binary;
/(.*)\.tag$/ and $files_ko{$1} = 1
or push @files, $_
for $ftp->ls;
for (@files) {
$ftp->get($_) and $ftp->delete($_)
unless exists $files_ko{$_};
print PLOT "$_\n";
}
$ftp->quit or die $ftp->message;
CLOSE(PLOT)
' <host> <user> <pass> <dir>
Radoulov, Dimitre
> On Jun 17, 2:40 am, "Radoulov, Dimitre"<cichomit...@gmail.com> wrote:
>> On 16/06/2010 18.33, Atropo wrote:
[...]
> before asking you i was trying this, according to tutorials. why it
> did not work?
>
> perl -MNet::FTP -le'
> ( $host, $user, $pass, $dir ) = @ARGV;
> $my_file = "recibidos.txt";
> OPEN(PLOT,">>$my_file") or die("The file cannot be opened!");
> $ftp = Net::FTP->new($host) or die "$@\n";
> $ftp->login( $user, $pass ) or die $ftp->message;
> $ftp->cwd($dir) or die $ftp->message;
> $ftp->binary;
> /(.*)\.tag$/ and $files_ko{$1} = 1
> or push @files, $_
> for $ftp->ls;
> for (@files) {
> $ftp->get($_) and $ftp->delete($_)
> unless exists $files_ko{$_};
> print PLOT "$_\n";
> }
> $ftp->quit or die $ftp->message;
> CLOSE(PLOT)
> '<host> <user> <pass> <dir>
open and close should be in lowercase.
Regards
Dimitre
Atropo
Sorry Dimitre, everytime i touch this script it stop working, and i
don't know why. i tried to print in the log file, the date the file
was transfered. i added a few lines. the ones that refers to date.
perl >> ftp.log -MNet::FTP -le'
@months = qw(Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec);
@weekDays = qw(Sun Mon Tue Wed Thu Fri Sat Sun);
( $host, $user, $pass, $dir ) = @ARGV;
$ftp = Net::FTP->new($host) or die "$@\n";
$ftp->login( $user, $pass ) or die $ftp->message;
$ftp->cwd($dir) or die $ftp->message;
$ftp->binary;
/(.*)\.tag$/ and $files_ko{$1} = 1
or push @files, $_
for $ftp->ls;
for (@files) {
($second, $minute, $hour, $dayOfMonth, $month, $yearOffset,
$dayOfWeek, $dayOfYear, $daylightSavings) = gmtime();
$year = 1900 + $yearOffset;
$theGMTime = "$hour:$minute:$second, $weekDays[$dayOfWeek]
$months[$month] $dayOfMonth, $year";
$ftp->get($_) and $ftp->delete($_) and
print unless exists $files_ko{$_}, $theGMTime;
pk
> Sorry Dimitre, everytime i touch this script it stop working, and i
> don't know why. i tried to print in the log file, the date the file
> was transfered. i added a few lines. the ones that refers to date.
I'm not going to touch on the actual issue here, but why don't you just list
*ALL* your requirements from the start, so you can have a fully working
script, rather than only telling part of the story and thinking you can
figure out the rest for yourself?
It would save your time, and that of whoever wants to help you.
Also, though not strictly related, this page could probably be useful:
Atropo
You're right PK, i though that RTFM would help me to learn enough to
complete the work. beside this, i have users that keep asking for
more. sorry i can't read your suggestion at http://mywiki.wooledge.org/XyProblem
it's filter as a social networking and avoided on my work place.
Radoulov, Dimitre
> On 17 jun, 02:40, "Radoulov, Dimitre"<cichomit...@gmail.com> wrote:
>> On 16/06/2010 18.33, Atropo wrote:
>>
>>
>>
>>> On Jun 16, 8:52 am, Atropo<lxvasq...@gmail.com> wrote:
>>>> On Jun 16, 5:07 am, "Radoulov, Dimitre"<cichomit...@gmail.com> wrote:
>>
>>>>> On 15/06/2010 20.31, Atropo wrote:
>>
>>>>>> On Jun 15, 2:08 pm, Atropo<lxvasq...@gmail.com> wrote:
>>>>>>> On Jun 15, 11:03 am, "Radoulov, Dimitre"<cichomit...@gmail.com>
>>>>>>> wrote:
>>
>>>>>>>> On 15/06/2010 16.52, Atropo wrote:
>>
>>>>>>>>> On Jun 15, 10:27 am, "Radoulov, Dimitre"<cichomit...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>> On 15/06/2010 15.52, Atropo wrote:
>>
>>>>>>>>>>> On Jun 14, 4:51 pm, "Radoulov, Dimitre"<cichomit...@gmail.com> wrote:
>>>>>>>>>>>> On 14/06/2010 22.47, Radoulov, Dimitre wrote:
>>
>>>>>>>>>>>>> On 14/06/2010 15.47, Atropo wrote:
>>>>>>>>>>>>>> Hi all, I'm on solaris 5.10 and trying to get several files from
>>>>>>>>>>>>>> windows server, but with the condition that the files do not have the
>>>>>>>>>>>>>> extension ".par"
>>
>> [...]
> Sorry Dimitre, everytime i touch this script it stop working, and i
> don't know why. i tried to print in the log file, the date the file
> was transfered. i added a few lines. the ones that refers to date.
[...]
perl >> ftp.log -MNet::FTP -le'
( $host, $user, $pass, $dir ) = @ARGV;
$ftp = Net::FTP->new($host) or die "$@\n";
$ftp->login( $user, $pass ) or die $ftp->message;
$ftp->cwd($dir) or die $ftp->message;
$ftp->binary;
/(.*)\.tag$/ and $files_ko{$1} = 1
or push @files, $_
for $ftp->ls;
for (@files) {
$ftp->get($_) and $ftp->delete($_) and
print scalar gmtime, ": ", $_ unless exists $files_ko{$_};
Kenny McCormack
>Atropo wrote:
>
>> Sorry Dimitre, everytime i touch this script it stop working, and i
>> don't know why. i tried to print in the log file, the date the file
>> was transfered. i added a few lines. the ones that refers to date.
>
>I'm not going to touch on the actual issue here, but why don't you just list
>*ALL* your requirements from the start, so you can have a fully working
>script, rather than only telling part of the story and thinking you can
>figure out the rest for yourself?
Perl is an opaque language.
>Also, though not strictly related, this page could probably be useful:
>
>http://mywiki.wooledge.org/XyProblem
A somewhat better treatment (better if for no other reason than that 3
letters are used, not just 2) can be found (in the sidebar) at:
http://homepage.ntlworld.com./jonathan.deboynepollard/FGA/put-down-the-chocolate-covered-banana.html
But my view is that this "Just tell me what you really want" attitude to
giving help is short-sighted in a non-professional, not-for-pay
environment like Usenet. I.e., although I sympathize with the sentiment
expressed at both of these URLs (and elsewhere - such as ESR's famous
"smart questions" text), the fact is: it's not *quite* that simple.
--
Faced with the choice between changing one's mind and proving that there is
no need to do so, almost everyone gets busy on the proof.
- John Kenneth Galbraith -