Ongoing Issue : DEADLINE_EXCEEDED

[Pete Lavetsky (AdWords API Guru)]

Originally reported on an old thread here : [DEADLINE_EXCEEDED](https://github.com/googleads/google-ads-java/issues/671#issuecomment-1597506702)

We are on v14 of the Java Client Library and have successfully been on it since June 9th.

Example request-id xCSCtyZt4MvNZlGEVgdfVw

For over 24 hours we've been receiving the DEADLINE_EXCEEDED response on some of our normal queries. Queries that have been functioning properly for years. We have a threshold of 10 minutes on our Grpc call:

```
final ServerStream<SearchGoogleAdsStreamResponse> stream = googleAdsServiceClient
                        .searchStreamCallable()
                        .call( request, GrpcCallContext.createDefault().withTimeout( Duration.of( MAX_TIMEOUT_LENGTH, MAX_TIMEOUT_UNITS ) ) );
```

```
./deadline-exceeded-test.sh
google-requests-summary.2023-06-19.log 26382
google-requests-summary.2023-06-18.log 10
google-requests-summary.2023-06-17.log 349
google-requests-summary.2023-06-16.log 91
```

Example request:

```
Request
-------
MethodName: google.ads.googleads.v14.services.GoogleAdsService/SearchStream
Endpoint: googleads.googleapis.com:443
Headers: {developer-token=REDACTED, login-customer-id=REDACTED, x-goog-api-client=gl-java/11.0.12 gccl/26.0.0 gapic/26.0.0 gax/2.29.0 grpc/1.55.1}
Body: customer_id: "REDACTED"
query: "select customer.auto_tagging_enabled, customer.call_reporting_setting.call_conversion_action, customer.call_reporting_setting.call_conversion_reporting_enabled, customer.call_reporting_setting.call_reporting_enabled, customer.conversion_tracking_setting.accepted_customer_data_terms, customer.conversion_tracking_setting.conversion_tracking_id, customer.conversion_tracking_setting.conversion_tracking_status, customer.conversion_tracking_setting.cross_account_conversion_tracking_id, customer.conversion_tracking_setting.enhanced_conversions_for_leads_enabled, customer.conversion_tracking_setting.google_ads_conversion_customer, customer.currency_code, customer.descriptive_name, customer.final_url_suffix, customer.has_partners_badge, customer.id, customer.image_asset_auto_migration_done, customer.image_asset_auto_migration_done_date_time, customer.location_asset_auto_migration_done, customer.location_asset_auto_migration_done_date_time, customer.manager, customer.offline_conversion_client_summaries, customer.optimization_score, customer.optimization_score_weight, customer.pay_per_conversion_eligibility_failure_reasons, customer.remarketing_setting.google_global_site_tag, customer.resource_name, customer.status, customer.test_account, customer.time_zone, customer.tracking_url_template from customer "


Response
--------
Headers: Metadata(content-type=application/grpc,request-id=xCSCtyZt4MvNZlGEVgdfVw,date=Tue, 20 Jun 2023 09:19:14 GMT,alt-svc=h3=":443"; ma=2592000,h3-29=":443"; ma=2592000)
Body: null
Failure message: errors {
  error_code {
    internal_error: DEADLINE_EXCEEDED
  }
  message: "The request took too long to respond."
}
request_id: "xCSCtyZt4MvNZlGEVgdfVw"
```

```
traceroute to googleads.googleapis.com (172.217.4.74), 50 hops max, 60 byte packets
 1  ip-172-31-130-116.us-east-2.compute.internal (172.31.130.116)  0.114 ms  0.128 ms  0.173 ms
 2  ec2-52-15-0-161.us-east-2.compute.amazonaws.com (52.15.0.161)  4.899 ms ec2-52-15-0-189.us-east-2.compute.amazonaws.com (52.15.0.189)  59.276 ms ec2-52-15-0-187.us-east-2.compute.amazonaws.com (52.15.0.187)  11.513 ms
 3  100.65.28.48 (100.65.28.48)  1.102 ms 100.65.30.160 (100.65.30.160)  8.960 ms 100.65.27.32 (100.65.27.32)  1.199 ms
 4  100.66.13.80 (100.66.13.80)  2.334 ms 100.66.13.252 (100.66.13.252)  3.461 ms 100.66.13.68 (100.66.13.68)  21.252 ms
 5  100.66.15.234 (100.66.15.234)  55.528 ms 100.66.14.4 (100.66.14.4)  15.191 ms 100.66.14.132 (100.66.14.132)  74.032 ms
 6  241.0.12.195 (241.0.12.195)  0.649 ms  0.620 ms 241.0.12.193 (241.0.12.193)  0.631 ms
 7  242.0.103.197 (242.0.103.197)  1.195 ms 242.0.103.71 (242.0.103.71)  1.249 ms  1.071 ms
 8  240.0.184.14 (240.0.184.14)  11.119 ms 240.0.184.35 (240.0.184.35)  11.043 ms 240.0.184.32 (240.0.184.32)  11.043 ms
 9  100.100.36.96 (100.100.36.96)  11.047 ms 100.100.2.126 (100.100.2.126)  12.206 ms  12.100 ms
10  99.83.65.3 (99.83.65.3)  11.211 ms 99.82.180.135 (99.82.180.135)  11.952 ms  12.061 ms
11  * * *
12  * 209.85.252.18 (209.85.252.18)  11.659 ms 108.170.240.97 (108.170.240.97)  12.870 ms
13  142.251.69.20 (142.251.69.20)  18.118 ms 108.170.246.49 (108.170.246.49)  11.687 ms 142.251.227.121 (142.251.227.121)  18.658 ms
14  142.251.69.9 (142.251.69.9)  22.605 ms  22.578 ms 142.251.49.75 (142.251.49.75)  12.751 ms
15  142.251.69.20 (142.251.69.20)  19.079 ms 216.239.59.150 (216.239.59.150)  22.378 ms 142.251.233.222 (142.251.233.222)  21.951 ms
16  142.251.69.9 (142.251.69.9)  23.227 ms 216.239.57.196 (216.239.57.196)  21.471 ms 108.170.243.225 (108.170.243.225)  22.538 ms
17  216.239.63.174 (216.239.63.174)  20.782 ms 142.251.234.70 (142.251.234.70)  21.910 ms 216.239.63.174 (216.239.63.174)  20.964 ms
18  108.170.243.225 (108.170.243.225)  22.593 ms 108.170.244.1 (108.170.244.1)  20.831 ms  20.255 ms
19  ord37s18-in-f10.1e100.net (172.217.4.74)  21.416 ms 216.239.51.117 (216.239.51.117)  21.465 ms  21.302 ms
```

We've seen this in the past but this has been persistent for over 24 hours

```
https://github.com/googleads/google-ads-java/issues/682
https://github.com/googleads/google-ads-java/issues/671
https://github.com/googleads/google-ads-java/issues/407
```

Pete