"Could not determine the type of file "sftp://root:***@192.168.10.xxx/root/gs-files".".

[Yassine Maachi]

I am attempting to bootstrap a private OpenStack cloud using Cloudify 2.7.1. It boots up the Linux instance correctly but fails "Uploading files to 192.168.10.XXX." due to an SFTP problem : "Could not determine the type of file "sftp://root:***@192.168.10.xxx/root/gs-files".".
I can access to the Instance using ssh (there is no probleme in the connection).
I tried with other images (CentOS, Ubuntu, Cerros, ...) but always the same error !!
can anyone help me please ?
I attached a screenshot (cloudify-sftp-error.png) of the network topology created by Cloudify, and the stack trace.

Full stack trace:

2015-04-30 10:26:27,470  INFO [org.cloudifysource.shell.commands.AbstractGSCommand] - Setting security profile to "nonsecure".
2015-04-30
 10:26:27,589  INFO
[org.cloudifysource.shell.commands.AbstractGSCommand] - Bootstrapping
cloud openstack-havana. This may take a few minutes.
2015-04-30
10:26:27,677  INFO
[org.cloudifysource.esc.driver.provisioning.BaseProvisioningDriver] -
Setup network configuration for managers
2015-04-30 10:26:27,677  
INFO [org.cloudifysource.esc.driver.provisioning.BaseProvisioningDriver]
 - Using management network : Cloudify-Management-Network
2015-04-30
10:26:51,536  INFO
[org.cloudifysource.esc.shell.listener.CliAgentlessInstallerListener] -
Attempting to access Management VM 192.168.10.241.
2015-04-30
10:27:10,551  INFO
[org.cloudifysource.esc.shell.listener.CliAgentlessInstallerListener] -
Uploading files to 192.168.10.241.
2015-04-30 10:27:15,708  WARNING [com.jcraft.jsch] - Permanently added '192.168.10.241' (RSA) to the list of known hosts.
2015-04-30
 10:27:25,998  INFO
[org.cloudifysource.esc.shell.installer.CloudGridAgentBootstrapper] -
Failed accessing management VM 192.168.10.241 Reason: Failed to set up
file transfer: Unknown message with code "Could not determine the type
of file "sftp://cir...@192.168.10.241/cirros/gs-files".".; Caused by:
org.cloudifysource.esc.installer.InstallerException: Failed to set up
file transfer: Unknown message with code "Could not determine the type
of file "sftp://cir...@192.168.10.241/cirros/gs-files".".
2015-04-30
10:27:26,210  INFO
[org.cloudifysource.esc.driver.provisioning.openstack.OpenStackCloudifyDriver]
 - Deleting Floating ip:
FloatingIp[floatingNetworkId=15578898-5e6b-44d9-a73a-1328ca6ea140,floatingIpAddress=192.168.10.241,portId=4b8dc211-12e8-4383-8799-f783d2786e98,id=593d8424-cfec-41ed-8204-ed8609366416]
2015-04-30
 10:27:29,607  SEVERE
[org.cloudifysource.shell.commands.AbstractGSCommand] - Failed to set up
 file transfer: Unknown message with code "Could not determine the type
of file "sftp://cir...@192.168.10.241/cirros/gs-files".". :
org.cloudifysource.esc.installer.InstallerException: Failed to set up
file transfer: Unknown message with code "Could not determine the type
of file "sftp://cir...@192.168.10.241/cirros/gs-files".".
    at org.cloudifysource.esc.installer.filetransfer.VfsFileTransfer.initialize(VfsFileTransfer.java:206)
    at org.cloudifysource.esc.installer.AgentlessInstaller.uploadFilesToServer(AgentlessInstaller.java:306)
    at org.cloudifysource.esc.installer.AgentlessInstaller.installOnMachineWithIP(AgentlessInstaller.java:210)
    at org.cloudifysource.esc.shell.installer.CloudGridAgentBootstrapper$1.call(CloudGridAgentBootstrapper.java:865)
    at org.cloudifysource.esc.shell.installer.CloudGridAgentBootstrapper$1.call(CloudGridAgentBootstrapper.java:860)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)
Caused
 by: org.apache.commons.vfs2.FileSystemException: Unknown message with
code "Could not determine the type of file
"sftp://cir...@192.168.10.241/cirros/gs-files".".
    at org.apache.commons.vfs2.provider.sftp.SftpFileObject.refresh(SftpFileObject.java:95)
    at org.apache.commons.vfs2.provider.AbstractFileSystem.resolveFile(AbstractFileSystem.java:366)
    at org.apache.commons.vfs2.provider.AbstractFileSystem.resolveFile(AbstractFileSystem.java:317)
    at org.apache.commons.vfs2.provider.AbstractOriginatingFileProvider.findFile(AbstractOriginatingFileProvider.java:85)
    at org.apache.commons.vfs2.provider.AbstractOriginatingFileProvider.findFile(AbstractOriginatingFileProvider.java:65)
    at org.apache.commons.vfs2.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:693)
    at org.apache.commons.vfs2.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:621)
    at org.cloudifysource.esc.installer.filetransfer.VfsFileTransfer.resolveTargetDirectory(VfsFileTransfer.java:218)
    at org.cloudifysource.esc.installer.filetransfer.VfsFileTransfer.initialize(VfsFileTransfer.java:203)
    ... 8 more
Caused
 by: org.apache.commons.vfs2.FileSystemException: Could not determine
the type of file "sftp://cir...@192.168.10.241/cirros/gs-files".
    at org.apache.commons.vfs2.provider.AbstractFileObject.getType(AbstractFileObject.java:505)
    at org.apache.commons.vfs2.provider.sftp.SftpFileObject.refresh(SftpFileObject.java:91)
    ... 16 more
Caused by: org.apache.commons.vfs2.FileSystemException: Could not connect to SFTP server at "sftp://cir...@192.168.10.241/".
    at org.apache.commons.vfs2.provider.sftp.SftpFileSystem.getChannel(SftpFileSystem.java:153)
    at org.apache.commons.vfs2.provider.sftp.SftpFileObject.statSelf(SftpFileObject.java:151)
    at org.apache.commons.vfs2.provider.sftp.SftpFileObject.doGetType(SftpFileObject.java:114)
    at org.apache.commons.vfs2.provider.AbstractFileObject.getType(AbstractFileObject.java:496)
    ... 17 more
Caused by: com.jcraft.jsch.JSchException: java.io.IOException: Pipe closed
    at com.jcraft.jsch.ChannelSftp.start(ChannelSftp.java:288)
    at com.jcraft.jsch.Channel.connect(Channel.java:152)
    at com.jcraft.jsch.Channel.connect(Channel.java:145)
    at org.apache.commons.vfs2.provider.sftp.SftpFileSystem.getChannel(SftpFileSystem.java:130)
    ... 20 more
Caused by: java.io.IOException: Pipe closed
    at java.io.PipedInputStream.read(PipedInputStream.java:308)
    at java.io.PipedInputStream.read(PipedInputStream.java:378)
    at com.jcraft.jsch.ChannelSftp.fill(ChannelSftp.java:2665)
    at com.jcraft.jsch.ChannelSftp.header(ChannelSftp.java:2691)
    at com.jcraft.jsch.ChannelSftp.start(ChannelSftp.java:257)
    ... 23 more

[Barak Merimovich]

As mentioned here: http://stackoverflow.com/questions/29963316/cloudify-could-not-determine-the-type-of-file-sftp-root192-168-10-xxx/29964122#29964122

Using cirros is not supported - and not a good idea.

Please send the error you get when using CentOS/Ubuntu, and make sure to try both SFTP and SCP file transfer modes.

...

[Yassine Maachi]

Yes i tried them both (SCP & SFTP modes).
this is the error using Ubuntu :

2015-05-04 15:50:21,144  INFO [org.cloudifysource.shell.commands.AbstractGSCommand] - Setting security profile to "nonsecure".
2015-05-04 15:50:21,236  INFO [org.cloudifysource.shell.commands.AbstractGSCommand] - Bootstrapping cloud openstack-havana. This may take a few minutes.
2015-05-04 15:50:21,708  INFO [org.cloudifysource.esc.driver.provisioning.BaseProvisioningDriver] - Setup network configuration for managers
2015-05-04 15:50:21,709  INFO [org.cloudifysource.esc.driver.provisioning.BaseProvisioningDriver] - Using management network : Cloudify-Management-Network
2015-05-04 15:50:49,832  INFO [org.cloudifysource.esc.shell.listener.CliAgentlessInstallerListener] - Attempting to access Management VM 192.168.10.230.
2015-05-04 15:54:48,268  INFO [org.cloudifysource.esc.shell.installer.CloudGridAgentBootstrapper] - Failed accessing management VM 192.168.10.249 Reason: Failed connecting to 192.168.10.249:22; Caused by: java.util.concurrent.TimeoutException: Failed connecting to 192.168.10.249:22
2015-05-04 15:56:58,993  INFO [org.cloudifysource.esc.shell.listener.CliAgentlessInstallerListener] - Uploading files to 192.168.10.230.
2015-05-04 15:58:36,839  INFO [org.cloudifysource.esc.shell.installer.CloudGridAgentBootstrapper] - Failed accessing management VM 192.168.10.230 Reason: Failed to set up file transfer: Unknown message with code "Could not determine the type of file "sftp://ro...@192.168.10.230/root/gs-files".".; Caused by: org.cloudifysource.esc.installer.InstallerException: Failed to set up file transfer: Unknown message with code "Could not determine the type of file "sftp://ro...@192.168.10.230/root/gs-files".".
2015-05-04 15:58:37,228  INFO [org.cloudifysource.esc.driver.provisioning.openstack.OpenStackCloudifyDriver] - Deleting Floating ip: FloatingIp[floatingNetworkId=15578898-5e6b-44d9-a73a-1328ca6ea140,floatingIpAddress=192.168.10.230,portId=fa748e92-3731-4199-bcc8-c94394f0002f,id=5d8649eb-c034-4b7e-8f32-3aecbe8ee1f9]
2015-05-04 15:58:41,251  SEVERE [org.cloudifysource.shell.commands.AbstractGSCommand] - Failed to set up file transfer: Unknown message with code "Could not determine the type of file "sftp://ro...@192.168.10.230/root/gs-files".". : org.cloudifysource.esc.installer.InstallerException: Failed to set up file transfer: Unknown message with code "Could not determine the type of file "sftp://ro...@192.168.10.230/root/gs-files".".

    at org.cloudifysource.esc.installer.filetransfer.VfsFileTransfer.initialize(VfsFileTransfer.java:206)
    at org.cloudifysource.esc.installer.AgentlessInstaller.uploadFilesToServer(AgentlessInstaller.java:306)
    at org.cloudifysource.esc.installer.AgentlessInstaller.installOnMachineWithIP(AgentlessInstaller.java:210)
    at org.cloudifysource.esc.shell.installer.CloudGridAgentBootstrapper$1.call(CloudGridAgentBootstrapper.java:865)
    at org.cloudifysource.esc.shell.installer.CloudGridAgentBootstrapper$1.call(CloudGridAgentBootstrapper.java:860)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)

Caused by: org.apache.commons.vfs2.FileSystemException: Unknown message with code "Could not determine the type of file "sftp://ro...@192.168.10.230/root/gs-files".".

    at org.apache.commons.vfs2.provider.sftp.SftpFileObject.refresh(SftpFileObject.java:95)
    at org.apache.commons.vfs2.provider.AbstractFileSystem.resolveFile(AbstractFileSystem.java:366)
    at org.apache.commons.vfs2.provider.AbstractFileSystem.resolveFile(AbstractFileSystem.java:317)
    at org.apache.commons.vfs2.provider.AbstractOriginatingFileProvider.findFile(AbstractOriginatingFileProvider.java:85)
    at org.apache.commons.vfs2.provider.AbstractOriginatingFileProvider.findFile(AbstractOriginatingFileProvider.java:65)
    at org.apache.commons.vfs2.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:693)
    at org.apache.commons.vfs2.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:621)
    at org.cloudifysource.esc.installer.filetransfer.VfsFileTransfer.resolveTargetDirectory(VfsFileTransfer.java:218)
    at org.cloudifysource.esc.installer.filetransfer.VfsFileTransfer.initialize(VfsFileTransfer.java:203)
    ... 8 more

Caused by: org.apache.commons.vfs2.FileSystemException: Could not determine the type of file "sftp://ro...@192.168.10.230/root/gs-files".

    at org.apache.commons.vfs2.provider.AbstractFileObject.getType(AbstractFileObject.java:505)
    at org.apache.commons.vfs2.provider.sftp.SftpFileObject.refresh(SftpFileObject.java:91)
    ... 16 more

Caused by: org.apache.commons.vfs2.FileSystemException: Could not connect to SFTP server at "sftp://ro...@192.168.10.230/".

    at org.apache.commons.vfs2.provider.sftp.SftpFileSystem.getChannel(SftpFileSystem.java:153)
    at org.apache.commons.vfs2.provider.sftp.SftpFileObject.statSelf(SftpFileObject.java:151)
    at org.apache.commons.vfs2.provider.sftp.SftpFileObject.doGetType(SftpFileObject.java:114)
    at org.apache.commons.vfs2.provider.AbstractFileObject.getType(AbstractFileObject.java:496)
    ... 17 more

Caused by: com.jcraft.jsch.JSchException: session is down
    at com.jcraft.jsch.Channel.sendChannelOpen(Channel.java:667)
    at com.jcraft.jsch.Channel.connect(Channel.java:151)

    at com.jcraft.jsch.Channel.connect(Channel.java:145)
    at org.apache.commons.vfs2.provider.sftp.SftpFileSystem.getChannel(SftpFileSystem.java:130)
    ... 20 more

2015-05-04 16:03:55,455  INFO [org.cloudifysource.shell.commands.AbstractGSCommand] - Setting security profile to "nonsecure".
2015-05-04 16:03:55,539  INFO [org.cloudifysource.shell.commands.AbstractGSCommand] - Bootstrapping cloud openstack-havana. This may take a few minutes.
2015-05-04 16:03:55,917  INFO [org.cloudifysource.esc.driver.provisioning.BaseProvisioningDriver] - Setup network configuration for managers
2015-05-04 16:03:55,918  INFO [org.cloudifysource.esc.driver.provisioning.BaseProvisioningDriver] - Using management network : Cloudify-Management-Network
2015-05-04 16:04:23,121  INFO [org.cloudifysource.esc.shell.listener.CliAgentlessInstallerListener] - Attempting to access Management VM 192.168.10.232.
2015-05-04 16:10:20,769  INFO [org.cloudifysource.esc.shell.listener.CliAgentlessInstallerListener] - Uploading files to 192.168.10.232.
2015-05-04 16:11:58,051  INFO [org.cloudifysource.esc.shell.installer.CloudGridAgentBootstrapper] - Failed accessing management VM 192.168.10.232 Reason: Failed to set up file transfer: Unknown message with code "Could not determine the type of file "sftp://ubu...@192.168.10.232/ubuntu/gs-files".".; Caused by: org.cloudifysource.esc.installer.InstallerException: Failed to set up file transfer: Unknown message with code "Could not determine the type of file "sftp://ubu...@192.168.10.232/ubuntu/gs-files".".
2015-05-04 16:11:58,376  INFO [org.cloudifysource.esc.driver.provisioning.openstack.OpenStackCloudifyDriver] - Deleting Floating ip: FloatingIp[floatingNetworkId=15578898-5e6b-44d9-a73a-1328ca6ea140,floatingIpAddress=192.168.10.232,portId=bf4b9e8b-cd3d-4fc0-864b-0beed5fd9e4c,id=b2e08a66-9a82-444a-9e21-9b53e6737e50]
2015-05-04 16:12:04,048  SEVERE [org.cloudifysource.shell.commands.AbstractGSCommand] - Failed to set up file transfer: Unknown message with code "Could not determine the type of file "sftp://ubu...@192.168.10.232/ubuntu/gs-files".".

...

[Barak Merimovich]

The errors indicate that SFTP is not available on the image. Did you try running the linux sftp command line utility (not a GUI based utility!) to connect to this instance?

Also, what was the error you got when using SCP?

---

From: cloudif...@googlegroups.com <cloudif...@googlegroups.com> on behalf of Yassine Maachi <yassine...@gmail.com>
Sent: Monday, May 04, 2015 5:21 PM
To: cloudif...@googlegroups.com
Cc: clemen...@fastconnect.fr
Subject: Re: "Could not determine the type of file "sftp://root:***@192.168.10.xxx/root/gs-files".".

 

--
You received this message because you are subscribed to the Google Groups "cloudify-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cloudify-user...@googlegroups.com.
To post to this group, send email to cloudif...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cloudify-users/14f6de22-4936-4ba2-b70b-29c4d16fce9c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Yassine Maachi]

I tried to connect to the instance created by cloudify but due to the error this instance is deleted automaticly after a few secondes so i don't have enough time to investigate if SFTP is installed. There is any way in cloudify to do a pause or skip the step of deleting the instances, so i can acces to it and verify ?
In order to be sure that SFTP is installed on the image, i created my own image (ubuntu) implementing this functionality, so i have an other error :

2015-05-05 10:05:13,609  INFO [org.cloudifysource.shell.commands.AbstractGSCommand] - Setting security profile to "nonsecure".
2015-05-05 10:05:13,729  INFO [org.cloudifysource.shell.commands.AbstractGSCommand] - Bootstrapping cloud openstack-havana. This may take a few minutes.
2015-05-05 10:05:14,008  INFO [org.cloudifysource.esc.driver.provisioning.BaseProvisioningDriver] - Setup network configuration for managers
2015-05-05 10:05:14,009  INFO [org.cloudifysource.esc.driver.provisioning.BaseProvisioningDriver] - Using management network : Cloudify-Management-Network
2015-05-05 10:05:40,457  INFO [org.cloudifysource.esc.shell.listener.CliAgentlessInstallerListener] - Attempting to access Management VM 192.168.10.247.
2015-05-05 10:10:55,393  INFO [org.cloudifysource.esc.shell.listener.CliAgentlessInstallerListener] - Uploading files to 192.168.10.247.
2015-05-05 10:11:56,440  INFO [org.cloudifysource.esc.shell.listener.CliAgentlessInstallerListener] - Launching agent on 192.168.10.247.
2015-05-05 10:11:57,018  WARNING [com.jcraft.jsch] - Permanently added '192.168.10.247' (RSA) to the list of known hosts.
2015-05-05 10:12:11,864  INFO [org.cloudifysource.esc.shell.installer.CloudGridAgentBootstrapper] - Failed accessing management VM 192.168.10.247 Reason: SSH execution failed with exit code: 101, message: Failed downloading Java installation; Caused by: org.cloudifysource.esc.installer.InstallerException: SSH execution failed with exit code: 101, message: Failed downloading Java installation
2015-05-05 10:12:12,179  INFO [org.cloudifysource.esc.driver.provisioning.openstack.OpenStackCloudifyDriver] - Deleting Floating ip: FloatingIp[floatingNetworkId=15578898-5e6b-44d9-a73a-1328ca6ea140,floatingIpAddress=192.168.10.247,portId=7674f752-a179-46fa-a8fc-1469899a1f61,id=d21d118e-e953-455b-9f37-bdfde073ebee]
2015-05-05 10:12:17,570  SEVERE [org.cloudifysource.shell.commands.AbstractGSCommand] - SSH execution failed with exit code: 101, message: Failed downloading Java installation

And when using SCP, i got this :

​

2015-05-05 10:14:58,317  INFO [org.cloudifysource.shell.commands.AbstractGSCommand] - Setting security profile to "nonsecure".
2015-05-05 10:14:58,452  INFO [org.cloudifysource.shell.commands.AbstractGSCommand] - Bootstrapping cloud openstack-havana. This may take a few minutes.
2015-05-05 10:14:58,525  INFO [org.cloudifysource.esc.driver.provisioning.BaseProvisioningDriver] - Setup network configuration for managers
2015-05-05 10:14:58,526  INFO [org.cloudifysource.esc.driver.provisioning.BaseProvisioningDriver] - Using management network : Cloudify-Management-Network
2015-05-05 10:15:25,307  INFO [org.cloudifysource.esc.shell.listener.CliAgentlessInstallerListener] - Attempting to access Management VM 192.168.10.226.
2015-05-05 10:21:10,747  INFO [org.cloudifysource.esc.shell.listener.CliAgentlessInstallerListener] - Uploading files to 192.168.10.226.
2015-05-05 10:21:43,984  INFO [org.cloudifysource.esc.shell.installer.CloudGridAgentBootstrapper] - Failed accessing management VM 192.168.10.226 Reason: Failed to create remote directory: /home/ubuntu/gs-files. Error was: Timeout expired; Caused by: org.cloudifysource.esc.installer.InstallerException: Failed to create remote directory: /home/ubuntu/gs-files. Error was: Timeout expired
2015-05-05 10:21:44,328  INFO [org.cloudifysource.esc.driver.provisioning.openstack.OpenStackCloudifyDriver] - Deleting Floating ip: FloatingIp[floatingNetworkId=15578898-5e6b-44d9-a73a-1328ca6ea140,floatingIpAddress=192.168.10.226,portId=327bc9d0-aad7-45e8-bcc7-2960b61411f1,id=5a598287-712d-4af5-8507-da07445e48b4]
2015-05-05 10:21:47,643  SEVERE [org.cloudifysource.shell.commands.AbstractGSCommand] - Failed to create remote directory: /home/ubuntu/gs-files. Error was: Timeout expired

--
You received this message because you are subscribed to a topic in the Google Groups "cloudify-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/cloudify-users/Rp0U_8Su4Kw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to cloudify-user...@googlegroups.com.

To post to this group, send email to cloudif...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cloudify-users/1430811684450.61395%40gigaspaces.com.

For more options, visit https://groups.google.com/d/optout.

--

_____________________________________________________

Yassine MAACHI 

Stagiaire SOA et Cloud ​​chez FastConnect - Atos.

Étudiant en Master 2 R IT RT à l'INSA de Toulouse.

Ingénieur  d'état en Réseaux & Télécommunications.

_________________________________________________________________________

[Barak Merimovich]

Don't use cloudify to start the VM - use the openstack console and set it to the same security group and keypair.

---

From: cloudif...@googlegroups.com <cloudif...@googlegroups.com> on behalf of Yassine Maachi <yassine...@gmail.com>

Sent: Tuesday, May 05, 2015 12:19 PM
To: Barak Merimovich
Cc: cloudif...@googlegroups.com; clemen...@fastconnect.fr

To view this discussion on the web visit https://groups.google.com/d/msgid/cloudify-users/CAEGkW%3DmhsOOfX7R1G_6O_pwYjFHf67NAKW%3Dq%2B1JMLW_DVPw9wQ%40mail.gmail.com.