2017-07-05 10:30:35 EDT INFO [main] org.apache.hadoop.conf.Configuration 1174 - mapreduce.user.classpath.first is deprecated. Instead, use mapreduce.job.user.classpath.first 2017-07-05 10:30:37 EDT WARN [main] gobblin.runtime.JobContext 275 - Property task.data.root.dir is missing. 2017-07-05 10:30:38 EDT INFO [main] gobblin.util.ClustersNames 69 - no default cluster mapping found 2017-07-05 10:30:38 EDT INFO [main] gobblin.runtime.app.ServiceBasedAppLauncher 158 - Starting the Gobblin application and all its associated Services 2017-07-05 10:30:38 EDT INFO [main] gobblin.util.ExecutorsUtils 125 - Attempting to shutdown ExecutorService: java.util.concurrent.ThreadPoolExecutor@1e7f2e0f[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 1] 2017-07-05 10:30:38 EDT INFO [main] gobblin.util.ExecutorsUtils 144 - Successfully shutdown ExecutorService: java.util.concurrent.ThreadPoolExecutor@1e7f2e0f[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 1] 2017-07-05 10:30:38 EDT WARN [main] gobblin.password.PasswordManager 190 - Property encrypt.key.loc not set. Cannot decrypt any encrypted password. 2017-07-05 10:30:38 EDT WARN [main] gobblin.password.PasswordManager 190 - Property encrypt.key.loc not set. Cannot decrypt any encrypted password. 2017-07-05 10:30:38 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper 146 - Attempting to connect to source via SFTP with privateKey: hdfs://localhost:8020/tmp/sftp_staging/user2/id_rsa_2048_a knownHosts: /tmp/sftp_staging/hosts userName: user1 hostName: sftp_source port: 22 proxyHost: null proxyPort: -1 2017-07-05 10:30:38 EDT WARN [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$LocalFileIdentityStrategy 436 - Failed to set identity using local file. Will attempt next strategy. java.io.FileNotFoundException: hdfs:/localhost:8020/tmp/sftp_staging/user2/id_rsa_2048_a (No such file or directory) 2017-07-05 10:30:38 EDT WARN [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$LocalFileIdentityStrategy 436 - Failed to set identity using local file. Will attempt next strategy. java.io.FileNotFoundException: id_rsa_2048_a (No such file or directory) 2017-07-05 10:30:38 EDT WARN [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$HDFSIdentityStrategy 419 - Failed to set identity using HDFS file. Will attempt next strategy. invalid privatekey: [B@77bb0ab5 2017-07-05 10:30:38 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - Connecting to sftp_source port 22 2017-07-05 10:30:38 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - Connection established 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - Remote version string: SSH-2.0-6.3.11.121 SSH Tectia Server 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - Local version string: SSH-2.0-JSCH-0.1.53 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - CheckCiphers: aes256-ctr,aes192-ctr,aes128-ctr,aes256-cbc,aes192-cbc,aes128-cbc,3des-ctr,arcfour,arcfour128,arcfour256 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - CheckKexes: diffie-hellman-group14-sha1,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - CheckSignatures: ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - SSH_MSG_KEXINIT sent 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - SSH_MSG_KEXINIT received 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - kex: server: diffie-hellman-group1-sha1,diffie-hellman-group14-sha1,diffie-hellman-group14-sha256@ssh.com,diffie-hellman-group-exchange-sha1,diffie-hellman-group-exchange-sha256 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - kex: server: ssh-rsa,ssh-rsa-sha256@ssh.com 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - kex: server: aes128-cbc,aes192-cbc,aes256-cbc,3des-cbc 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - kex: server: aes128-cbc,aes192-cbc,aes256-cbc,3des-cbc 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - kex: server: hmac-sha1,hmac-sha256@ssh.com,hmac-sha384@ssh.com,hmac-sha512@ssh.com 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - kex: server: hmac-sha1,hmac-sha256@ssh.com,hmac-sha384@ssh.com,hmac-sha512@ssh.com 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - kex: server: none,zlib 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - kex: server: none,zlib 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - kex: server: 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - kex: server: 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - kex: client: ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - kex: client: ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - kex: client: aes128-ctr,aes128-cbc,3des-ctr,3des-cbc,blowfish-cbc,aes192-ctr,aes192-cbc,aes256-ctr,aes256-cbc 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - kex: client: aes128-ctr,aes128-cbc,3des-ctr,3des-cbc,blowfish-cbc,aes192-ctr,aes192-cbc,aes256-ctr,aes256-cbc 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - kex: client: hmac-md5,hmac-sha1,hmac-sha2-256,hmac-sha1-96,hmac-md5-96 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - kex: client: hmac-md5,hmac-sha1,hmac-sha2-256,hmac-sha1-96,hmac-md5-96 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - kex: client: none 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - kex: client: none 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - kex: client: 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - kex: client: 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - kex: server->client aes128-cbc hmac-sha1 none 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - kex: client->server aes128-cbc hmac-sha1 none 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - SSH_MSG_KEXDH_INIT sent 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - expecting SSH_MSG_KEXDH_REPLY 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - ssh_rsa_verify: signature true 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - Host 'sftp_source' is known and matches the RSA host key 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - SSH_MSG_NEWKEYS sent 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - SSH_MSG_NEWKEYS received 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - SSH_MSG_SERVICE_REQUEST sent 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - SSH_MSG_SERVICE_ACCEPT received 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$MyUserInfo 386 - You are authorized to use this System for approved business purposes only. Use for any other purpose is prohibited. All transactional records, reports, email, software and other data generated by or residing upon this System, to the extent permitted by local law, are the property of Citigroup Inc. or one of its subsidiaries or their affiliates (individually or collectively ' Citigroup ') and may be used by Citigroup for any purpose authorized and permissible in your country of work. Activities on this System are monitored to the extent permitted by local law. 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.extract.sftp.SftpFsHelper$JSchLogger 333 - Disconnecting from sftp_source port 22 2017-07-05 10:30:39 EDT ERROR [main] gobblin.source.extractor.extract.sftp.SftpFsHelper 191 - Auth fail com.jcraft.jsch.JSchException: Auth fail at com.jcraft.jsch.Session.connect(Session.java:512) at com.jcraft.jsch.Session.connect(Session.java:183) at gobblin.source.extractor.extract.sftp.SftpFsHelper.connect(SftpFsHelper.java:184) at gobblin.source.extractor.extract.sftp.SftpSource.initFileSystemHelper(SftpSource.java:33) at gobblin.source.extractor.filebased.FileBasedSource.getWorkunits(FileBasedSource.java:77) at gobblin.runtime.SourceDecorator.getWorkunits(SourceDecorator.java:54) at gobblin.runtime.AbstractJobLauncher.launchJob(AbstractJobLauncher.java:242) at gobblin.runtime.mapreduce.CliMRJobLauncher.launchJob(CliMRJobLauncher.java:84) at gobblin.runtime.mapreduce.CliMRJobLauncher.run(CliMRJobLauncher.java:61) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at gobblin.runtime.mapreduce.CliMRJobLauncher.main(CliMRJobLauncher.java:106) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) 2017-07-05 10:30:39 EDT ERROR [main] gobblin.runtime.SourceDecorator 61 - Failed to get work units for job job_SFTP_example_job_1499265036207 java.lang.RuntimeException: gobblin.source.extractor.filebased.FileBasedHelperException: Cannot connect to SFTP source at com.google.common.base.Throwables.propagate(Throwables.java:160) at gobblin.source.extractor.filebased.FileBasedSource.getWorkunits(FileBasedSource.java:79) at gobblin.runtime.SourceDecorator.getWorkunits(SourceDecorator.java:54) at gobblin.runtime.AbstractJobLauncher.launchJob(AbstractJobLauncher.java:242) at gobblin.runtime.mapreduce.CliMRJobLauncher.launchJob(CliMRJobLauncher.java:84) at gobblin.runtime.mapreduce.CliMRJobLauncher.run(CliMRJobLauncher.java:61) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at gobblin.runtime.mapreduce.CliMRJobLauncher.main(CliMRJobLauncher.java:106) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: gobblin.source.extractor.filebased.FileBasedHelperException: Cannot connect to SFTP source at gobblin.source.extractor.extract.sftp.SftpFsHelper.connect(SftpFsHelper.java:192) at gobblin.source.extractor.extract.sftp.SftpSource.initFileSystemHelper(SftpSource.java:33) at gobblin.source.extractor.filebased.FileBasedSource.getWorkunits(FileBasedSource.java:77) ... 13 more Caused by: com.jcraft.jsch.JSchException: Auth fail at com.jcraft.jsch.Session.connect(Session.java:512) at com.jcraft.jsch.Session.connect(Session.java:183) at gobblin.source.extractor.extract.sftp.SftpFsHelper.connect(SftpFsHelper.java:184) ... 15 more 2017-07-05 10:30:39 EDT ERROR [main] gobblin.runtime.AbstractJobLauncher 321 - Failed to launch and run job job_SFTP_example_job_1499265036207: gobblin.runtime.JobException: Failed to get work units for job job_SFTP_example_job_1499265036207 gobblin.runtime.JobException: Failed to get work units for job job_SFTP_example_job_1499265036207 at gobblin.runtime.AbstractJobLauncher.launchJob(AbstractJobLauncher.java:249) at gobblin.runtime.mapreduce.CliMRJobLauncher.launchJob(CliMRJobLauncher.java:84) at gobblin.runtime.mapreduce.CliMRJobLauncher.run(CliMRJobLauncher.java:61) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at gobblin.runtime.mapreduce.CliMRJobLauncher.main(CliMRJobLauncher.java:106) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) 2017-07-05 10:30:39 EDT INFO [main] gobblin.util.ExecutorsUtils 125 - Attempting to shutdown ExecutorService: java.util.concurrent.ThreadPoolExecutor@5432050b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 1] 2017-07-05 10:30:39 EDT INFO [main] gobblin.util.ExecutorsUtils 144 - Successfully shutdown ExecutorService: java.util.concurrent.ThreadPoolExecutor@5432050b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 1] 2017-07-05 10:30:39 EDT INFO [main] gobblin.util.ExecutorsUtils 125 - Attempting to shutdown ExecutorService: java.util.concurrent.ThreadPoolExecutor@5cbb84b1[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 1] 2017-07-05 10:30:39 EDT INFO [main] gobblin.util.ExecutorsUtils 144 - Successfully shutdown ExecutorService: java.util.concurrent.ThreadPoolExecutor@5cbb84b1[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 1] 2017-07-05 10:30:39 EDT INFO [main] gobblin.runtime.app.ServiceBasedAppLauncher 179 - Shutting down the application 2017-07-05 10:30:39 EDT INFO [main] gobblin.runtime.mapreduce.MRJobLauncher 468 - Deleted working directory /user2/gobblin-jobs/gobblin-work-dir/working/SFTP_example_job/job_SFTP_example_job_1499265036207 2017-07-05 10:30:39 EDT INFO [main] gobblin.source.extractor.filebased.FileBasedSource 199 - Shutting down the FileSystemHelper connection 2017-07-05 10:30:39 EDT WARN [Thread-8] gobblin.runtime.app.ServiceBasedAppLauncher 174 - ApplicationLauncher has already stopped