AWS S3 buckets in US Standard region

[wallac...@sentient.ai]

I am new to fluentd and trying to set up an AWS S3 output plugin.  Consistently I get the following error:

2015-07-06 16:25:30 +0000 [error]: unexpected error error_class=RuntimeError error=#<RuntimeError: can't call S3 API. Please check your aws_key_id / aws_sec_key or s3_region configuration. error = #<AWS::S3::Errors::PermanentRedirect: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.>>

The AWS error about the specified endpoint suggests the problem is related to how AWS addresses US Standard buckets vs. region specific path-style buckets.  My bucket is in the US Standard region and none of the required region names apply (e.g., us-west-1, etc.).  For US Standard buckets, one address them by s3.amazonaws.com.  For region specific buckets, one address them by s3-<region>.amazonaws.com.  I don't know how to make this work with Fluentd.

Does anyone have experience with using Fluentd with buckets that are US Standard?

Thanks,
Wallace

[wallac...@sentient.ai]

It would appear that one solution is to use 

  s3_enpoint s3.amazonaws.com

but the documentation says that is deprecated.  Is there a non-deprecated solution?

[Mr. Fiber]

> but the documentation says that is deprecated.  Is there a non-deprecated solution?

s3_region paramter is non-deprecated solution.

http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region

s3_endpoint is removed in the next AWS SDK: https://ruby.awsblog.com/post/TxVOTODBPHAEP9/Working-with-Regions

So after updated aws-sdk to v2, fluent-plugin-s3 also removes s3_endpoint support.

> Does anyone have experience with using Fluentd with buckets that are US Standard?

There are meny users who use s3 plugin with US standard but I first see this error.

http://stackoverflow.com/questions/3835681/permanentredirect-s3-for-unspecified-endpoint

Hmm... very weird.

Masahiro

--
You received this message because you are subscribed to the Google Groups "Fluentd Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fluentd+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[wallac...@sentient.ai]

Ah!  Thanks.  My problem was born of ignorance.

I failed to notice that "US Standard" is just another name for us-east-1, which is the region to use.

Thanks,

Wallace