Edda setup for multiple regions from a single AWs account ERROR - [DynamoDB.scala:89] [initState] Error getting item from dynamodb: name

Kaushal

unread,

Sep 13, 2019, 3:21:28 PM9/13/19

to Edda Users

hello Everyone,

I am trying to get Edda to work with one region first then add more regions if it works but single is not working for so far

here is my config my, as soon i specify edda.accounts=inf.<region>

i see an exceptions for Dynamo as given below.

please can someone take a look and let me know what should be done to get this working .

i am using the 2.1 container mentioned in the zerotodocker wiki https://github.com/Netflix-Skunkworks/zerotodocker/wiki/Edda

docker run -d --name eddatest -p 8080:8080 -v `pwd`/edda.properties.mine:/tomcat/webapps/ROOT/WEB-INF/classes/edda.properties netflixoss/edda:2.1

#-- change these properties ---------------------------------------------

edda.accounts=inf.eu-central-1

edda.inf.eu-central-1.region=eu-central-1

edda.inf.aws.accessKey=

edda.inf.aws.secretKey=

edda.s3current.bucket=edda-state-inf

#------------------------------------------------------------------------

edda.collection.aws.stacks.refresh=3600000

edda.collection.inf.cache.refresh=30000

edda.collection.jitter.enabled=false

edda.collection.inf.refresh=120000

edda.bean.argPattern=[^a-zA-Z0-9_]

edda.datastore.class=

edda.elector.class=com.netflix.edda.aws.DynamoDBElector

edda.elector.dynamodb.account=inf

edda.elector.dynamodb.leaderTimeout=60000

edda.elector.dynamodb.tableName=edda-leader

edda.elector.dynamodb.readCapacity=5

edda.elector.dynamodb.writeCapacity=1

edda.datastore.current.class=com.netflix.edda.aws.S3CurrentDatastore

edda.s3current.account=inf

edda.s3current.table=edda-s3current-collection-index-inf

edda.s3current.readCapacity=10

edda.s3current.writeCapacity=1

edda.s3current.locationPrefix=edda/s3current/inf

~

2019-09-13 19:15:36.592 - ERROR - [StateMachine.scala:221] [Elector DynamoDB] caught exception

java.lang.NullPointerException

at com.amazonaws.util.AwsHostNameUtils.parseRegionName(AwsHostNameUtils.java:24)

at com.amazonaws.auth.AWS4Signer.extractRegionName(AWS4Signer.java:147)

at com.amazonaws.auth.AWS4Signer.getScope(AWS4Signer.java:267)

at com.amazonaws.auth.AWS4Signer.sign(AWS4Signer.java:80)

at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:274)

at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:199)

at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:1245)

at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.describeTable(AmazonDynamoDBClient.java:600)

at com.netflix.edda.aws.DynamoDB$.init(DynamoDB.scala:33)

at com.netflix.edda.aws.DynamoDBElector.init(DynamoDBElector.scala:58)

at com.netflix.edda.StateMachine.act(StateMachine.scala:176)

at scala.actors.Reactor$$anonfun$dostart$1.apply(Reactor.scala:224)

at scala.actors.ReactorTask.run(ReactorTask.scala:33)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

2019-09-13 19:15:36.593 - INFO - [Collection.scala:612] [start] Starting [Collection inf.eu-central-1.view.simpleQueues]

2019-09-13 19:15:36.608 - INFO - [MergedCollection.scala:91] Starting [MergedCollection aws.iamVirtualMFADevices]

2019-09-13 19:15:36.609 - INFO - [Collection.scala:612] [start] Starting [Collection inf.eu-central-1.aws.iamVirtualMFADevices]

2019-09-13 19:15:36.633 - INFO - [Collection.scala:612] [start] Starting [Collection inf.eu-central-1.aws.iamUsers]

2019-09-13 19:15:36.664 - INFO - [MergedCollection.scala:91] Starting [MergedCollection aws.iamRoles]

2019-09-13 19:15:36.666 - INFO - [Collection.scala:612] [start] Starting [Collection inf.eu-central-1.aws.iamRoles]

2019-09-13 19:15:36.671 - ERROR - [DynamoDB.scala:89] [initState] Error getting item from dynamodb: name

java.lang.NullPointerException

at com.amazonaws.util.AwsHostNameUtils.parseRegionName(AwsHostNameUtils.java:24)

at com.amazonaws.auth.AWS4Signer.extractRegionName(AWS4Signer.java:147)

at com.amazonaws.auth.AWS4Signer.getScope(AWS4Signer.java:267)

at com.amazonaws.auth.AWS4Signer.sign(AWS4Signer.java:80)

at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:274)

at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:199)

at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.invoke(AmazonDynamoDBClient.java:1245)

at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.getItem(AmazonDynamoDBClient.java:652)

at com.netflix.edda.aws.DynamoDB$.get(DynamoDB.scala:77)

at com.netflix.edda.aws.S3CurrentDatastore.loadImpl(S3CurrentDatastore.scala:110)

at com.netflix.edda.aws.S3CurrentDatastore.load(S3CurrentDatastore.scala:94)

at com.netflix.edda.Collection.load(Collection.scala:287)

at com.netflix.edda.Collection.doLoad(Collection.scala:488)

at com.netflix.edda.Collection.initState(Collection.scala:405)

at com.netflix.edda.CollectionProcessor.initState(CollectionProcessor.scala:56)

at com.netflix.edda.StateMachine$$anonfun$act$1$$anonfun$2.apply(StateMachine.scala:180)

at com.netflix.edda.Utils$RETRY$.apply(Utils.scala:112)

at com.netflix.edda.StateMachine$$anonfun$act$1.applyOrElse(StateMachine.scala:179)

at scala.runtime.AbstractPartialFunction$mcVL$sp.apply$mcVL$sp(AbstractPartialFunction.scala:33)

at scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:33)

at scala.runtime.AbstractPartialFunction$mcVL$sp.apply(AbstractPartialFunction.scala:25)

at scala.actors.ReactorTask.run(ReactorTask.scala:31)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:745)

2019-09-13 19:15:36.672 - INFO - [DynamoDB.scala:95] [initState] dynamo read lapse: 3ms

2019-09-13 19:15:36.672 - ERROR - [Collection.scala:492] [initState] [Collection inf.eu-central-1.aws.hostedZones] failed to load

java.lang.NullPointerException

at com.amazonaws.util.AwsHostNameUtils.parseRegionName(AwsHostNameUtils.java:24)

Thanks,

Kaushal.

Matthew Johnson

unread,

Sep 13, 2019, 4:10:58 PM9/13/19

to Kaushal, Edda Users

At Netflix, we deploy one cluster per account and region we wish Edda to crawl.

If you follow this convention, then it should work as expected.

The codebase had the capability of crawling multiple accounts and regions, but since we do not run it this way internally, these code paths do not get much attention.

--
You received this message because you are subscribed to the Google Groups "Edda Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to edda-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/edda-users/ebdd7a22-3ae3-4500-a420-b6b752b4ee62%40googlegroups.com.

Kaushal

unread,

Sep 13, 2019, 4:23:21 PM9/13/19

to Edda Users

Hey Matthew,

Thanks for your response.

If you see my configuration file in the earlier email. i am following the convention mentioned in the docs https://github.com/Netflix/edda/wiki/Configuration .

How should i change it to make it work for one region one account.

here is what i do in config

edda.accounts=inf.eu-central-1

edda.inf.eu-central-1.region=eu-central-1

where inf is account and region is eu-central-1

Thanks,

Kaushal.

To unsubscribe from this group and stop receiving emails from it, send an email to edda-...@googlegroups.com.

Matthew Johnson

unread,

Sep 16, 2019, 12:12:58 PM9/16/19

to Kaushal, Edda Users

In your first attempt, it looked like you were trying to use the ZeroToDocker containers as a starting point. These are no longer being maintained.

I've started cleanup on the Edda project and should have a branch with a number of updates landing in a few weeks.

I'll be able to provide better guidance on running Edda locally and in a Tomcat container when that lands.

To unsubscribe from this group and stop receiving emails from it, send an email to edda-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/edda-users/21de5ecb-7ca0-4ec3-aae9-43dcf90ae30a%40googlegroups.com.

Reply all

Reply to author

Forward