Excon::Errors::SocketError (getaddrinfo: name or service not known (SocketError))

Abhishek Kumar

unread,

Feb 23, 2016, 8:07:26 AM2/23/16

to Snowplow

Dear All,

Already change the region: ap-southeast-1

strange thing is that if I do not mentioned the region( left blank)..I am getting the same error.

[ec2-user@ip-10-0-0-67 ~]$ ./snowplow-emr-etl-runner --config config.yml --resolver resolver.json --enrichments enrichments

D, [2016-02-23T13:02:05.879000 #1373] DEBUG -- : Staging...

F, [2016-02-23T13:02:06.970000 #1373] FATAL -- :

Excon::Errors::SocketError (getaddrinfo: name or service not known (SocketError)):

org/jruby/ext/socket/RubySocket.java:292:in `getaddrinfo'

/home/ec2-user/snowplow-emr-etl-runner!/gems/excon-0.45.3/lib/excon/socket.rb:108:in `connect'

/home/ec2-user/snowplow-emr-etl-runner!/gems/excon-0.45.3/lib/excon/ssl_socket.rb:148:in `connect'

/home/ec2-user/snowplow-emr-etl-runner!/gems/excon-0.45.3/lib/excon/socket.rb:28:in `initialize'

/home/ec2-user/snowplow-emr-etl-runner!/gems/excon-0.45.3/lib/excon/ssl_socket.rb:8:in `initialize'

/home/ec2-user/snowplow-emr-etl-runner!/gems/excon-0.45.3/lib/excon/connection.rb:387:in `socket'

/home/ec2-user/snowplow-emr-etl-runner!/gems/excon-0.45.3/lib/excon/connection.rb:106:in `request_call'

/home/ec2-user/snowplow-emr-etl-runner!/gems/excon-0.45.3/lib/excon/middlewares/mock.rb:47:in `request_call'

/home/ec2-user/snowplow-emr-etl-runner!/gems/excon-0.45.3/lib/excon/middlewares/instrumentor.rb:22:in `request_call'

/home/ec2-user/snowplow-emr-etl-runner!/gems/excon-0.45.3/lib/excon/middlewares/base.rb:15:in `request_call'

/home/ec2-user/snowplow-emr-etl-runner!/gems/excon-0.45.3/lib/excon/connection.rb:233:in `request'

/home/ec2-user/snowplow-emr-etl-runner!/gems/excon-0.45.3/lib/excon/middlewares/idempotent.rb:26:in `error_call'

/home/ec2-user/snowplow-emr-etl-runner!/gems/excon-0.45.3/lib/excon/middlewares/base.rb:10:in `error_call'

/home/ec2-user/snowplow-emr-etl-runner!/gems/excon-0.45.3/lib/excon/connection.rb:256:in `request'

/home/ec2-user/snowplow-emr-etl-runner!/gems/excon-0.45.3/lib/excon/middlewares/idempotent.rb:26:in `error_call'

/home/ec2-user/snowplow-emr-etl-runner!/gems/excon-0.45.3/lib/excon/middlewares/base.rb:10:in `error_call'

/home/ec2-user/snowplow-emr-etl-runner!/gems/excon-0.45.3/lib/excon/connection.rb:256:in `request'

/home/ec2-user/snowplow-emr-etl-runner!/gems/excon-0.45.3/lib/excon/middlewares/idempotent.rb:26:in `error_call'

/home/ec2-user/snowplow-emr-etl-runner!/gems/excon-0.45.3/lib/excon/middlewares/base.rb:10:in `error_call'

/home/ec2-user/snowplow-emr-etl-runner!/gems/excon-0.45.3/lib/excon/connection.rb:256:in `request'

/home/ec2-user/snowplow-emr-etl-runner!/gems/fog-1.24.0/lib/fog/xml/sax_parser_connection.rb:35:in `request'

/home/ec2-user/snowplow-emr-etl-runner!/gems/fog-1.24.0/lib/fog/xml/connection.rb:17:in `request'

/home/ec2-user/snowplow-emr-etl-runner!/gems/fog-1.24.0/lib/fog/aws/storage.rb:547:in `_request'

/home/ec2-user/snowplow-emr-etl-runner!/gems/fog-1.24.0/lib/fog/aws/storage.rb:542:in `request'

/home/ec2-user/snowplow-emr-etl-runner!/gems/fog-1.24.0/lib/fog/aws/requests/storage/get_service.rb:21:in `get_service'

/home/ec2-user/snowplow-emr-etl-runner!/gems/fog-1.24.0/lib/fog/aws/requests/storage/sync_clock.rb:9:in `sync_clock'

/home/ec2-user/snowplow-emr-etl-runner!/gems/sluice-0.2.2/lib/sluice/storage/s3/s3.rb:52:in `new_fog_s3_from'

/home/ec2-user/snowplow-emr-etl-runner!/emr-etl-runner/lib/snowplow-emr-etl-runner/s3_tasks.rb:110:in `stage_logs_for_emr'

/home/ec2-user/snowplow-emr-etl-runner!/emr-etl-runner/lib/snowplow-emr-etl-runner/runner.rb:51:in `run'

/home/ec2-user/snowplow-emr-etl-runner!/gems/contracts-0.7/lib/contracts/method_reference.rb:46:in `send_to'

/home/ec2-user/snowplow-emr-etl-runner!/gems/contracts-0.7/lib/contracts.rb:305:in `call_with'

/home/ec2-user/snowplow-emr-etl-runner!/gems/contracts-0.7/lib/contracts/decorators.rb:159:in `common_method_added'

file:/home/ec2-user/snowplow-emr-etl-runner!/emr-etl-runner/bin/snowplow-emr-etl-runner:39:in `(root)'

org/jruby/RubyKernel.java:1091:in `load'

file:/home/ec2-user/snowplow-emr-etl-runner!/META-INF/main.rb:1:in `(root)'

org/jruby/RubyKernel.java:1072:in `require'

file:/home/ec2-user/snowplow-emr-etl-runner!/META-INF/main.rb:1:in `(root)'

/tmp/jruby2512018509752283391extract/jruby-stdlib-1.7.20.1.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:1:in `(root)'

Thanks

Abhishek

Ihor Tomilenko

unread,

Feb 23, 2016, 11:32:36 AM2/23/16

to Snowplow

Hi Abhishek,

Do you still experience this error? I'm asking as your other today's post "Bad S3 Location" indicates that you've made some progress.

If the issue is still ongoing could you, please, provide us with your configuration file?

Regards,

Ihor

Abhishek Kumar

unread,

Feb 23, 2016, 12:35:50 PM2/23/16

to Snowplow

Thanks for quick reply I am still getting a error.

D, [2016-02-23T16:52:08.710000 #6090] DEBUG -- : Staging...

F, [2016-02-23T16:52:09.721000 #6090] FATAL -- :

Excon::Errors::SocketError (getaddrinfo: name or service not known (SocketError)):

org/jruby/ext/socket/RubySocket.java:292:in `getaddrinfo'

I am attaching my config.yml file

config.yml

Ihor Tomilenko

unread,

Feb 23, 2016, 12:53:12 PM2/23/16

to Snowplow

Abhishek,

Could you, please, remove the region zone indicator (last letter "b") from the value "ap-southeast-1b" against the region parameter and try again?

That is instead of

region: ap-southeast-1b

place

region: ap-southeast-1

Mind you it is stated in a few places in the configuration file (s3 and emr sections).

Regards,

Ihor

Abhishek Kumar

unread,

Feb 23, 2016, 1:06:56 PM2/23/16

to Snowplow

Hi Ihor,

Did the same but getting the same error..

Thanks &Regards

Abhishek

Ihor Tomilenko

unread,

Feb 23, 2016, 1:30:48 PM2/23/16

to Snowplow

Abhishek,

From the very beginning you mentioned "Already change the region: ap-southeast-1".

Could you verify that your subnet subnet-xxxxxxxx as well as your key name are in your current region ap-southeast-1, please?

--Ihor

abhishek kumar

unread,

Feb 23, 2016, 2:15:54 PM2/23/16

to snowpl...@googlegroups.com

Yes My subnet and key name both are in current region ap-southeast-1

--
You received this message because you are subscribed to a topic in the Google Groups "Snowplow" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/snowplow-user/EK8L6vIl_8Y/unsubscribe.
To unsubscribe from this group and all its topics, send an email to snowplow-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Ihor Tomilenko

unread,

Feb 23, 2016, 2:30:34 PM2/23/16

to Snowplow, abhishe...@careerlauncher.com

Abhishek,

Could I ask you to make the following modification to you config file, please?

1. Your log bucket is still missing "s3;//" schema.

2. Could you replace your raw in bucket reference with the bucket path as it's explained in this article: https://github.com/snowplow/snowplow/wiki/Enable-logging-to-S3

If it's still not working, could you, please again send us the latest config file for a review, please?

--Ihor

Abhishek Kumar

unread,

Feb 24, 2016, 5:23:19 AM2/24/16

to Snowplow, abhishe...@careerlauncher.com

Thanks Ihor.. solve the problem by the help of article provided by you..Now I am getting a new error after Initializing EMR jobflow

[ec2-user@ip-10-0-0-67 ~]$ ./snowplow-emr-etl-runner --config config.yml --resolver resolver.json --enrichments enrichments --skip staging

D, [2016-02-24T10:10:55.229000 #30267] DEBUG -- : Initializing EMR jobflow

D, [2016-02-24T10:11:06.576000 #30267] DEBUG -- : EMR jobflow j-4CQM83UOAF00 started, waiting for jobflow to complete...

I, [2016-02-24T10:11:06.692000 #30267] INFO -- : SnowplowTracker::Emitter initialized with endpoint http://snowplowcc-env.ap-southeast-1.elasticbeanstalk.com:80/i

I, [2016-02-24T10:11:07.534000 #30267] INFO -- : Attempting to send 1 request

I, [2016-02-24T10:11:07.536000 #30267] INFO -- : Sending GET request to http://snowplowcc-env.ap-southeast-1.elasticbeanstalk.com:80/i...

I, [2016-02-24T10:11:07.550000 #30267] INFO -- : GET request to http://snowplowcc-env.ap-southeast-1.elasticbeanstalk.com:80/i finished with status code 200

I, [2016-02-24T10:19:11.741000 #30267] INFO -- : Attempting to send 1 request

I, [2016-02-24T10:19:11.784000 #30267] INFO -- : Sending GET request to http://snowplowcc-env.ap-southeast-1.elasticbeanstalk.com:80/i...

I, [2016-02-24T10:19:11.834000 #30267] INFO -- : GET request to http://snowplowcc-env.ap-southeast-1.elasticbeanstalk.com:80/i finished with status code 200

F, [2016-02-24T10:19:12.505000 #30267] FATAL -- :

Snowplow::EmrEtlRunner::EmrExecutionError (EMR jobflow j-4CQM83UOAF00 failed, check Amazon EMR console and Hadoop logs for details (help: https://github.com/snowplow/snowplow/wiki/Troubleshooting-jobs-on-Elastic-MapReduce). Data files not archived.

SnowplowETL: TERMINATING [STEP_FAILURE] ~ elapsed time n/a [2016-02-24 10:18:16 UTC - ]

- 1. Start HBase 0.92.0: FAILED ~ 00:00:00 [2016-02-24 10:18:16 UTC - 2016-02-24 10:18:16 UTC]

- 2. Elasticity S3DistCp Step: Shredded HDFS -> S3: CANCELLED ~ elapsed time n/a [ - ]

- 3. Elasticity Scalding Step: Shred Enriched Events: CANCELLED ~ elapsed time n/a [ - ]

- 4. Elasticity S3DistCp Step: Enriched HDFS _SUCCESS -> S3: CANCELLED ~ elapsed time n/a [ - ]

- 5. Elasticity S3DistCp Step: Enriched HDFS -> S3: CANCELLED ~ elapsed time n/a [ - ]

- 6. Elasticity Scalding Step: Enrich Raw Events: CANCELLED ~ elapsed time n/a [ - ]):

/home/ec2-user/snowplow-emr-etl-runner!/emr-etl-runner/lib/snowplow-emr-etl-runner/emr_job.rb:426:in `run'

/home/ec2-user/snowplow-emr-etl-runner!/gems/contracts-0.7/lib/contracts/method_reference.rb:46:in `send_to'

/home/ec2-user/snowplow-emr-etl-runner!/gems/contracts-0.7/lib/contracts.rb:305:in `call_with'

/home/ec2-user/snowplow-emr-etl-runner!/gems/contracts-0.7/lib/contracts/decorators.rb:159:in `common_method_added'

/home/ec2-user/snowplow-emr-etl-runner!/emr-etl-runner/lib/snowplow-emr-etl-runner/runner.rb:68:in `run'

/home/ec2-user/snowplow-emr-etl-runner!/gems/contracts-0.7/lib/contracts/method_reference.rb:46:in `send_to'

/home/ec2-user/snowplow-emr-etl-runner!/gems/contracts-0.7/lib/contracts.rb:305:in `call_with'

/home/ec2-user/snowplow-emr-etl-runner!/gems/contracts-0.7/lib/contracts/decorators.rb:159:in `common_method_added'

file:/home/ec2-user/snowplow-emr-etl-runner!/emr-etl-runner/bin/snowplow-emr-etl-runner:39:in `(root)'

org/jruby/RubyKernel.java:1091:in `load'

file:/home/ec2-user/snowplow-emr-etl-runner!/META-INF/main.rb:1:in `(root)'

org/jruby/RubyKernel.java:1072:in `require'

file:/home/ec2-user/snowplow-emr-etl-runner!/META-INF/main.rb:1:in `(root)'

/tmp/jruby1060519287963425749extract/jruby-stdlib-1.7.20.1.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:1:in `(root)'

Ihor Tomilenko

unread,

Feb 24, 2016, 10:42:44 AM2/24/16

to Snowplow, abhishe...@careerlauncher.com

Hi Abhishek,

HBase is optional. You can leave it blank (in config.yml). Please, see the following discussions:

https://groups.google.com/forum/#!searchin/snowplow-user/EmrExecutionError/snowplow-user/GJJv11VbN2M/z3khXjsBBwAJ

https://groups.google.com/forum/#!searchin/snowplow-user/EmrExecutionError/snowplow-user/cziCLpjhv5I/IkRo_GLjESEJ

--Ihor

abhishek kumar

unread,

Feb 25, 2016, 8:25:07 AM2/25/16

to snowpl...@googlegroups.com

Thanks a Lot .. for your great help, now I have complete snowplow-emr-etl-runner setup successfully.When I go to create table in my postgre database

/home/ec2-user/4-storage/postgres-storage/sql/ there folder recipes but folder cube is missing where I can get this folder.?

Step I did:

$ cd 4-storage/postgres-storage/sql
$ psql -h <HOSTNAME> -U power_user -d snowplow -p <PORT> -f atomic-def.sql

$ psql -h <HOSTNAME> -U power_user -d snowplow -p <PORT> -f recipes/recipes-basic.sql
$ psql -h <HOSTNAME> -U power_user -d snowplow -p <PORT> -f recipes/recipes-catalog.sql
$ psql -h <HOSTNAME> -U power_user -d snowplow -p <PORT> -f recipes/recipes-customers.sql------complete

$ psql -h <HOSTNAME> -U power_user -d snowplow -p <PORT> -f cubes/cube-pages.sql
$ psql -h <HOSTNAME> -U power_user -d snowplow -p <PORT> -f cubes/cube-visits.sql
$ psql -h <HOSTNAME> -U power_user -d snowplow -p <PORT> -f cubes/cube-transactions.sql-----uncomplete because cube folder is not there(/home/ec2-user/4-storage/postgres-storage/sql)

Jimit Modi

unread,

Feb 25, 2016, 8:47:57 AM2/25/16

to snowpl...@googlegroups.com

On Thu, Feb 25, 2016 at 6:55 PM, abhishek kumar <abhishe...@careerlauncher.com> wrote:

cubes

Hey Abhishek,

I think the documentation you are referring is not updated. cubes has been removed in Release 64 - https://github.com/snowplow/snowplow/blob/2001aef473215d3422b5a9ae61537b1c4b94465e/CHANGELOG#L497

Thanks

--

Jim(y || it)

Ihor Tomilenko

unread,

Feb 25, 2016, 12:36:01 PM2/25/16

to Snowplow, abhishe...@careerlauncher.com

Hi Abhishek,

Jimit is right. The wiki page you are following (presumably this one) is a bit outdated. Sorry for confusion. We will modify it to reflect the current trend. In short, you can ignore cubes views (as well as recipes). Soon, we'll create new data models and recipes that are more specific than we now have.

On the side note, could I ask you to rather open a new topic (if you still have an issue with Posgres) so that it could be helpful to all community members. Right now the issue with configuration, ETL runner and storage are all mixed in this single topic with the subject "SocketError", which is confusing.

Regards,

Ihor

Abhishek Kumar

unread,

Feb 25, 2016, 12:39:04 PM2/25/16

to Snowplow

Thanks Jimit,

When I run ./snowplow-storage-loader --config config.yml --skip analyze

Downloading Snowplow events...

Everything goes right but when

Loading Snowplow events into snowplow (PostgreSQL database)...

Opening database connection ...

Unexpected error: FATAL: password authentication failed for user "power_user"

abhishek kumar

unread,

Feb 25, 2016, 12:50:47 PM2/25/16

to snowpl...@googlegroups.com

Thanks Ihor for again help me.. now I have issue with posgres when I run snowplow-storage-loader.I am going to details on new post.

Reply all

Reply to author

Forward