[Rocks-Discuss] Transfer files between frontend and compute node

452 views
Skip to first unread message

Funlola Ogunleye

unread,
Mar 26, 2012, 11:21:03 AM3/26/12
to npaci-rocks...@sdsc.edu
How can i transfer a folder between the frontend and compute node?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.sdsc.edu/pipermail/npaci-rocks-discussion/attachments/20120326/dbf972c7/attachment.html

Philip Papadopoulos

unread,
Mar 26, 2012, 11:37:39 AM3/26/12
to Discussion of Rocks Clusters
How many ways are their to skin a cat?
Here's one.

tar cf - <foldername> | ssh <compute node> "cd <desired top level dir>; tar
xvfBp -"

that will create <foldername> underneath your <desired top level dir> on
<compute node>

There are many variations including using rsync.

-P


Nothing rocks-specific about the above pipeline.


--
Philip Papadopoulos, PhD
University of California, San Diego
858-822-3628 (Ofc)
619-331-2990 (Fax)


-------------- next part --------------
An HTML attachment was scrubbed...

URL: https://lists.sdsc.edu/pipermail/npaci-rocks-discussion/attachments/20120326/5d47ca48/attachment.html

Funlola Ogunleye

unread,
Mar 26, 2012, 12:48:55 PM3/26/12
to npaci-rocks...@sdsc.edu
I meant to write how to i copy a folder from frontend to compute node?
Sorry for the novice question but Im new to linux

URL: https://lists.sdsc.edu/pipermail/npaci-rocks-discussion/attachments/20120326/d0b97102/attachment.html

"Hung-Sheng Tsao (Lao Tsao 老曹) Ph.D."

unread,
Mar 26, 2012, 2:06:11 PM3/26/12
to Discussion of Rocks Clusters

On 3/26/2012 1:36 PM, Vlad Manea wrote:
> on the frontend and in the location where you want to copy the compute-0-0 directory:
>
> scp -r compute-0-0:/home/funlola/your_directory .
FE
as root
scp -r <directory path> compute-0-0:<directory path>
>
> V.

--
Hung-Sheng Tsao Ph D.
Founder& Principal
HopBit GridComputing LLC
cell: 9734950840

http://laotsao.blogspot.com/
http://laotsao.wordpress.com/
http://blogs.oracle.com/hstsao/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: laotsao.vcf
Type: text/x-vcard
Size: 608 bytes
Desc: not available
Url : https://lists.sdsc.edu/pipermail/npaci-rocks-discussion/attachments/20120326/df94708d/laotsao.vcf

Philip Papadopoulos

unread,
Mar 26, 2012, 3:31:02 PM3/26/12
to Discussion of Rocks Clusters
need a space between - and eucalyptus. and xvfBp -

for tar, "-" is the name of file (stdin for extracting, stdout for creating)

-P


On Mon, Mar 26, 2012 at 12:15 PM, Funlola Ogunleye <
funlola...@gmail.com> wrote:

> I am trying to transfer an eucalyptus folder from root of frontend
> (cluster) to root of compute node (compute-0-0). This is the command I
> entered but its clearly not working. Any suggestions.
>
> [root@cluster ~]# tar cf -eucalyptus-2.0.3-centos-i386 | ssh compute-0-0
> cd; tar xvfBp-

> https://lists.sdsc.edu/pipermail/npaci-rocks-discussion/attachments/20120326/91ed1ecd/attachment.html
>
>


--
Philip Papadopoulos, PhD
University of California, San Diego
858-822-3628 (Ofc)
619-331-2990 (Fax)

-------------- next part --------------
An HTML attachment was scrubbed...

URL: https://lists.sdsc.edu/pipermail/npaci-rocks-discussion/attachments/20120326/4be09e23/attachment.html

Funlola Ogunleye

unread,
Mar 26, 2012, 3:15:25 PM3/26/12
to npaci-rocks...@sdsc.edu
I am trying to transfer an eucalyptus folder from root of frontend
(cluster) to root of compute node (compute-0-0). This is the command I
entered but its clearly not working. Any suggestions.

[root@cluster ~]# tar cf -eucalyptus-2.0.3-centos-i386 | ssh compute-0-0
cd; tar xvfBp-

On Mon, Mar 26, 2012 at 12:48 PM, Funlola Ogunleye <
funlola...@gmail.com> wrote:

URL: https://lists.sdsc.edu/pipermail/npaci-rocks-discussion/attachments/20120326/91ed1ecd/attachment.html

"Hung-Sheng Tsao (Lao Tsao 老曹) Ph.D."

unread,
Mar 26, 2012, 3:21:27 PM3/26/12
to Discussion of Rocks Clusters, Funlola Ogunleye
# tar cf - eucalyptus-2.0.3-centos-i386 | ssh compute-0-0 "(cd / ; tar xvfBp - )"
look there were space
regrds

--

Hung-Sheng Tsao Ph D.
Founder& Principal
HopBit GridComputing LLC
cell: 9734950840

-------------- next part --------------
A non-text attachment was scrubbed...
Name: laotsao.vcf
Type: text/x-vcard
Size: 608 bytes
Desc: not available

Url : https://lists.sdsc.edu/pipermail/npaci-rocks-discussion/attachments/20120326/0cb2fd09/laotsao.vcf

Bart Brashers

unread,
Mar 28, 2012, 11:57:41 AM3/28/12
to Discussion of Rocks Clusters
This is a basic Linux question, not a rocks-specific question. This list is for Rocks help, even if many of the people are really nice and helpful and will answer when they have time.

You might get better/faster results asking for help on a more general Linux forum. Do you have a local Linux Users Group in your town?

This was the command you gave:

# tar cf -eucalyptus-2.0.3-centos-i386 | ssh compute-0-0 cd; tar xvfBp-

The directory (or file) named "-eucalyptus-2.0.3-centos-i386" must exist on the frontend, in a location relative to the directory in which you typed the command. I seldom see directories that start with a dash ("-"), so that's pretty suspect. In Linux/Unix, command-line options often begin with a dash. So most people avoid using a dash as the first character in a file name or directory name -- because it's ambiguous, the computer can't tell if you meant to give an option or refer to a file.

If you can do "ls -- -eucalyptus-2.0.3-centos-i386" and get some valid output, then that file/directory exists. If it responds with "ls: -eucalyptus-2.0.3-centos-i386: No such file or directory" then it doesn't exist. Note the crafty use us two dashes ("--") to tell the command to stop interpreting a dash as an option, but instead assume it's a filename.

On the compute-0-0 side, you first gave a command to change to the user's home directory ("cd"). With no arguments, "cd" just changed to /root. So you should find the directory at /root/-eucalyptus-2.0.3-centos-i386 on the compute node.

An alternative method, with my comments on the right side (don't type them):

# su - (become root)
# cd /path/to/eucalyptus-dir (substitute the real path)
# tar zcvf eucalyptus.tar.gz euc[TAB] (press the Tab key to auto-complete the dir name)
# scp eucalyptus.tar.gz compute-0-0:/path/where/you/want/it
(the path must exist)
# ssh compute-0-0 (log into a compute node)
# cd /path/where/you/want/it (move to where you put the file)
# tar zxvf eucalyptus.tar.gz (extract the directory)

Bart

> -----Original Message-----
> From: npaci-rocks-dis...@sdsc.edu [mailto:npaci-rocks-
> discussio...@sdsc.edu] On Behalf Of Funlola Ogunleye
> Sent: Wednesday, March 28, 2012 6:52 AM
> To: npaci-rocks...@sdsc.edu
> Subject: Re: [Rocks-Discuss] Transfer files between frontend and compute
> node
>
> Thank you for the recent command. When I logged in as compute-0-0, i
> entered "ls" to check if the directory is listed but alas, it was not.
>
> On Mon, Mar 26, 2012 at 3:15 PM, Funlola Ogunleye

> discussion/attachments/20120328/c7c7bcee/attachment.html

________________________________
This message contains information that may be confidential, privileged or otherwise protected by law from disclosure. It is intended for the exclusive use of the Addressee(s). Unless you are the addressee or authorized agent of the addressee, you may not review, copy, distribute or disclose to anyone the message or any information contained within. If you have received this message in error, please contact the sender by electronic reply to em...@environcorp.com and immediately delete all copies of the message.

Reply all
Reply to author
Forward
0 new messages