Configuring timeouts for waiting on states

929 views
Skip to first unread message

Joe Barnes

unread,
Feb 25, 2015, 4:29:29 PM2/25/15
to terrafo...@googlegroups.com
I have some terraform configurations which provision RDS instances in AWS.  Up until today, they have worked really well.  However I now typically get a timeout while terraform is waiting for the RDS instance to become available.  

The errors look like this:
aws_db_instance.user_db: Error: timeout while waiting for state to become 'available'

When I encounter the error, I have noted that the RDS instance is in a state of "Backing Up".  A minute or so later the RDS indeed moves to the "Available" state.

Does terraform support configuring the timeout it uses for waiting on these transitions?  I've not found any in the documentation.  I have the code up and it appears that WaitForState() in helper\resource\state.go is responsible for this waiting and is perhaps configurable via this conf *StateChangeConf parameter as of 0.3.7.  I'm digging around to see if there is a way for me to configure the .Timeout field. (This is literally my first ever look at Go so forgive any ignorance if I'm completely off here)

I'll continue to dig around but if anyone has any pointers on how to configure the timeout, that will be greatly appreciated!

Thanks,
Joe

Rahul Menon

unread,
Feb 25, 2015, 9:31:23 PM2/25/15
to Joe Barnes, terrafo...@googlegroups.com
Joe,

The refresh is actually hardcoded. I dont believe its configurable. Here are the lines of code which does this.
https://github.com/hashicorp/terraform/blob/master/builtin/providers/aws/resource_aws_db_instance.go#L272-L278

Thanks
Rahul

--
You received this message because you are subscribed to the Google Groups "Terraform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to terraform-too...@googlegroups.com.
To post to this group, send email to terrafo...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/terraform-tool/14b5b55f-17ae-4b21-808f-51d2936c665c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Joe Barnes

unread,
Feb 25, 2015, 10:31:01 PM2/25/15
to Rahul Menon, <terraform-tool@googlegroups.com>
Thanks Rahul,

I had stumbled upon that code and arrived at the same conclusion.  I'll probably build it with more timeout so I can continue on for now.  

Joe

David Cunningham

unread,
Feb 26, 2015, 12:03:49 AM2/26/15
to Joe Barnes, Rahul Menon, <terraform-tool@googlegroups.com>
This seems like something that could be configured for all providers with a single commandline parameter / env var.

Joe Barnes

unread,
Feb 26, 2015, 6:18:58 PM2/26/15
to David Cunningham, Rahul Menon, <terraform-tool@googlegroups.com>
I could see this being ideal as a parameter for the resource itself.  I would want a timeout of 30 minutes for an RDS as it can take over 20 minutes when you have applied a DB parameter group etc.  But the timeout for the creation of say a VPC should be much lower.

Joe

David Cunningham

unread,
Feb 26, 2015, 6:26:04 PM2/26/15
to Joe Barnes, Rahul Menon, <terraform-tool@googlegroups.com>
To add to the complexity: the Google APIs actually have two time outs, as requests that take a long time asynchronously return an operation object which can then be polled for completion (with further http requests).  In that case there is a timeout inherent to making an http connection, and also an application-level timeout beyond which you give up calling the poll API.  I don't know if other providers have such a distinction.

In this case what you mention would affect the 'number of polls' timeout but there would also need to be a global control since http timeouts would be due to local connectivity issues / partial outages of the platform.
Reply all
Reply to author
Forward
0 new messages