NodeJS gRPC server in k8s QUEUE_TIMEOUT
293 views
Skip to first unread message
Chris Tyler
May 1, 2021, 4:50:59 AM5/1/21
to grpc.io
I have created a gRPC client/server for processing Unary requests. I have had it working fine on my laptop. Now, I am working to deploy it to a Kubernetes cluster in IBM Cloud. The Deployment succeeds and the Service is running. I've tried this with both LoadBalancer and NodePort service types.
I have enabled GRPC_TRACE=all and GRPC_VERBOSITY=DEBUG. When the pod starts, I see the logs below. I get about 20 repetitions of RETURN_EVENT...QUEUE_TIMEOUT. When I make a request using grpcurl, I get the message transport is closing.
Currently, I'm using an insecure credentials connection.
I have poured over a ton of posts around K8s and gRPC. It seemed pretty straight forward, but I'm struggling to get this to work. Any help would be greatly appreciated.
Log file
D0501 07:53:17.904046507 20 dns_resolver.cc:294] Using native dns resolver
I0501 07:53:17.904189226 20 timer_manager.cc:88] Spawn timer thread
I0501 07:53:17.904274322 20 init.cc:164] grpc_init(void)
I0501 07:53:17.904358489 35 timer_manager.cc:250] timers not checked: expect another thread to
I0501 07:53:17.904396913 35 timer_manager.cc:197] sleep until kicked
I0501 07:53:17.904765647 20 completion_queue.cc:501] grpc_completion_queue_create_internal(completion_type=0, polling_type=0)
I0501 07:53:18.310402717 20 server.cc:1021] grpc_server_create(0x55a6d0a0ad40, 0)
I0501 07:53:18.310629797 20 server.cc:1002] grpc_server_register_completion_queue(server=0x55a6d093a980, cq=0x55a6d0bce8a0, reserved=0)
I0501 07:53:18.330949779 20 server_chttp2.cc:33] grpc_server_add_insecure_http2_port(server=0x55a6d093a980, addr=0.0.0.0:50052)
I0501 07:53:18.331257146 20 tcp_server_custom.cc:382] SERVER 0x55a6d0b09cc0 add_port [::]:50052 error="No Error"
I0501 07:53:18.331726761 20 server.cc:1111] grpc_server_start(server=0x55a6d093a980)
I0501 07:53:18.331884195 20 tcp_server_custom.cc:424] SERVER_START 0x55a6d0b09cc0
I0501 07:53:18.332028119 20 call_details.cc:31] grpc_call_details_init(cd=0x55a6d0ad8170)
I0501 07:53:18.332155662 20 metadata_array.cc:29] grpc_metadata_array_init(array=0x55a6d0ad81d0)
I0501 07:53:18.332369156 20 server.cc:1470] grpc_server_request_call(server=0x55a6d093a980, call=0x55a6d0ad8168, details=0x55a6d0ad8170, initial_metadata=0x55a6d0ad81d0, cq_bound_to_call=0x55a6d0bce8a0, cq_for_notification=0x55a6d0bce8a0, tag=0x55a6d0b73d60)
I0501 07:53:18.334480815 20 credentials.cc:183] grpc_server_credentials_release(creds=0)
I0501 07:53:18.334693475 20 completion_queue.cc:952] grpc_completion_queue_next(cq=0x55a6d0bce8a0, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0)
I0501 07:53:18.334835887 20 completion_queue.cc:1060] RETURN_EVENT[0x55a6d0bce8a0]: QUEUE_TIMEOUT
I0501 07:53:18.335157812 20 completion_queue.cc:952] grpc_completion_queue_next(cq=0x55a6d0bce8a0, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0)
I0501 07:53:18.335308566 20 completion_queue.cc:1060] RETURN_EVENT[0x55a6d0bce8a0]: QUEUE_TIMEOUT
I0501 07:53:25.872128754 20 completion_queue.cc:952] grpc_completion_queue_next(cq=0x55a6d0bce8a0, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0)
I0501 07:53:25.872209655 20 completion_queue.cc:1060] RETURN_EVENT[0x55a6d0bce8a0]: QUEUE_TIMEOUT
I0501 07:53:25.872281900 20 completion_queue.cc:952] grpc_completion_queue_next(cq=0x55a6d0bce8a0, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0)
Michael Lumish
May 3, 2021, 2:10:35 PM5/3/21
to Chris Tyler, grpc.io
Those QUEUE_TIMEOUT events don't really mean anything. They just show the library polling for new events and not finding any at that time. If there's nothing else in the log, that probably implies that the client can't establish a connection to the server at all. If you enable tracing on the client, you may get a better idea of what is going wrong.
--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/2d7f941e-9148-4d6c-833b-1242f41a684dn%40googlegroups.com.
Chris Tyler
May 3, 2021, 3:27:28 PM5/3/21
to Michael Lumish, grpc.io
Michael,
Have a blessed day,
Chris Tyler
214.673.0483
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee, you should not disseminate, distribute or copy this email. Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.
Thanks for the response. I did run client tracing as well. Again, after getting several QUEUE_TIMEOUTs, I am getting a Deadline Exceeded. I even bumped it up to 10 seconds, but continue to get that. I’m now running this in minikube and still getting the same errors.
I0503 14:19:15.964035000 4431523328 completion_queue.cc:960] grpc_completion_queue_next(cq=0x10540bd90, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 14:19:15.964050000 4431523328 completion_queue.cc:1060] RETURN_EVENT[0x10540bd90]: QUEUE_TIMEOUT
I0503 14:19:15.964205000 4431523328 completion_queue.cc:960] grpc_completion_queue_next(cq=0x10540bd90, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 14:19:15.964213000 4431523328 completion_queue.cc:1060] RETURN_EVENT[0x10540bd90]: QUEUE_TIMEOUT
I0503 14:19:15.964331000 4431523328 completion_queue.cc:960] grpc_completion_queue_next(cq=0x10540bd90, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 14:19:15.964337000 4431523328 completion_queue.cc:1060] RETURN_EVENT[0x10540bd90]: QUEUE_TIMEOUT
I0503 14:19:15.964476000 4431523328 completion_queue.cc:960] grpc_completion_queue_next(cq=0x10540bd90, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 14:19:15.964482000 4431523328 completion_queue.cc:1060] RETURN_EVENT[0x10540bd90]: QUEUE_TIMEOUT
I0503 14:19:15.964848000 4431523328 completion_queue.cc:960] grpc_completion_queue_next(cq=0x10540bd90, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 14:19:15.964860000 4431523328 completion_queue.cc:1060] RETURN_EVENT[0x10540bd90]: QUEUE_TIMEOUT
I0503 14:19:15.965061000 4431523328 completion_queue.cc:960] grpc_completion_queue_next(cq=0x10540bd90, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 14:19:15.965072000 4431523328 completion_queue.cc:1060] RETURN_EVENT[0x10540bd90]: QUEUE_TIMEOUT
I0503 14:19:15.965181000 4431523328 completion_queue.cc:960] grpc_completion_queue_next(cq=0x10540bd90, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 14:19:15.965188000 4431523328 completion_queue.cc:1060] RETURN_EVENT[0x10540bd90]: QUEUE_TIMEOUT
I0503 14:19:15.965355000 4431523328 completion_queue.cc:960] grpc_completion_queue_next(cq=0x10540bd90, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 14:19:15.965367000 4431523328 completion_queue.cc:1060] RETURN_EVENT[0x10540bd90]: QUEUE_TIMEOUT
I0503 14:19:15.965429000 4431523328 completion_queue.cc:960] grpc_completion_queue_next(cq=0x10540bd90, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 14:19:15.965435000 4431523328 completion_queue.cc:1060] RETURN_EVENT[0x10540bd90]: QUEUE_TIMEOUT
I0503 14:19:15.968316000 4431523328 completion_queue.cc:960] grpc_completion_queue_next(cq=0x10540bd90, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 14:19:15.968334000 4431523328 completion_queue.cc:1060] RETURN_EVENT[0x10540bd90]: QUEUE_TIMEOUT
I0503 14:19:15.968527000 4431523328 completion_queue.cc:960] grpc_completion_queue_next(cq=0x10540bd90, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 14:19:15.968540000 4431523328 completion_queue.cc:1060] RETURN_EVENT[0x10540bd90]: QUEUE_TIMEOUT
I0503 14:19:15.968594000 4431523328 completion_queue.cc:960] grpc_completion_queue_next(cq=0x10540bd90, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 14:19:15.968602000 4431523328 completion_queue.cc:1060] RETURN_EVENT[0x10540bd90]: QUEUE_TIMEOUT
I0503 14:19:17.191481000 4431523328 completion_queue.cc:960] grpc_completion_queue_next(cq=0x10540bd90, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 14:19:17.191517000 4431523328 completion_queue.cc:1060] RETURN_EVENT[0x10540bd90]: QUEUE_TIMEOUT
I0503 14:19:17.191549000 4431523328 completion_queue.cc:960] grpc_completion_queue_next(cq=0x10540bd90, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 14:19:17.191557000 4431523328 completion_queue.cc:1060] RETURN_EVENT[0x10540bd90]: QUEUE_TIMEOUT
I0503 14:19:18.012466000 4431523328 client_channel.cc:3663] chand=0x10587d860 calld=0x10587c630: cancelling queued pick: error={"created":"@1620069558.012433000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4} self=0x105437f80 calld->pick_canceller=0x105437f80
I0503 14:19:18.012498000 4431523328 client_channel.cc:3687] chand=0x10587d860 calld=0x10587c630: removing from queued picks list
I0503 14:19:18.012511000 4431523328 client_channel.cc:2359] chand=0x10587d860 calld=0x10587c630: failing 1 pending batches: {"created":"@1620069558.012433000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}
I0503 14:19:18.012527000 4431523328 completion_queue.cc:682] cq_end_op_for_next(cq=0x10540bd90, tag=0x105062dc0, error={"created":"@1620069558.012433000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}, done=0x1082b5be0, done_arg=0x10587cb10, storage=0x10587cb60)
E0503 14:19:18.012535000 4431523328 completion_queue.cc:685] Operation failed: tag=0x105062dc0, error={"created":"@1620069558.012433000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}
I0503 14:19:18.012546000 4431523328 call.cc:626] OP[client_idle:0x10587c5f0]: SEND_MESSAGE:flags=0x00000000:len=22
I0503 14:19:18.012554000 4431523328 channel_stack.cc:226] OP[client-channel:0x10587c608]: SEND_MESSAGE:flags=0x00000000:len=22
I0503 14:19:18.012561000 4431523328 client_channel.cc:2249] chand=0x10587d860 calld=0x10587c630: adding pending batch at index 1
I0503 14:19:18.012604000 4431523328 client_channel.cc:2083] chand=0x10587d860 calld=0x10587c630: saved batch, yielding call combiner
I0503 14:19:18.012615000 4431523328 call.cc:626] OP[client_idle:0x10587c5f0]: SEND_TRAILING_METADATA{} RECV_INITIAL_METADATA RECV_MESSAGE RECV_TRAILING_METADATA
I0503 14:19:18.012624000 4431523328 channel_stack.cc:226] OP[client-channel:0x10587c608]: SEND_TRAILING_METADATA{} RECV_INITIAL_METADATA RECV_MESSAGE RECV_TRAILING_METADATA
I0503 14:19:18.012630000 4431523328 client_channel.cc:2249] chand=0x10587d860 calld=0x10587c630: adding pending batch at index 2
I0503 14:19:18.012636000 4431523328 client_channel.cc:2083] chand=0x10587d860 calld=0x10587c630: saved batch, yielding call combiner
I0503 14:19:18.012643000 4431523328 client_channel.cc:2035] chand=0x10587d860 calld=0x10587c630: recording cancel_error={"created":"@1620069558.012433000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}
I0503 14:19:18.012667000 4431523328 client_channel.cc:2359] chand=0x10587d860 calld=0x10587c630: failing 2 pending batches: {"created":"@1620069558.012433000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}
I0503 14:19:18.012678000 4431523328 call.cc:626] OP[client_idle:0x10587c5f0]: CANCEL:{"created":"@1620069558.012433000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}
I0503 14:19:18.012686000 4431523328 channel_stack.cc:226] OP[client-channel:0x10587c608]: CANCEL:{"created":"@1620069558.012433000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}
I0503 14:19:18.012969000 4431523328 client_channel.cc:2016] chand=0x10587d860 calld=0x10587c630: failing batch with error: {"created":"@1620069558.012433000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}
I0503 14:19:18.012998000 4431523328 completion_queue.cc:682] cq_end_op_for_next(cq=0x10540bd90, tag=0x10510f120, error={"created":"@1620069558.012433000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}, done=0x1082b5be0, done_arg=0x10587cbf0, storage=0x10587cc40)
E0503 14:19:18.013008000 4431523328 completion_queue.cc:685] Operation failed: tag=0x10510f120, error={"created":"@1620069558.012433000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}
D0503 14:19:18.013018000 4431523328 call.cc:733] set_final_status CLI
D0503 14:19:18.013024000 4431523328 call.cc:734] {"created":"@1620069558.012433000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}
I0503 14:19:18.013034000 4431523328 completion_queue.cc:682] cq_end_op_for_next(cq=0x10540bd90, tag=0x1051100a0, error="No Error", done=0x1082b5be0, done_arg=0x10587ccd0, storage=0x10587cd20)
I0503 14:19:18.013046000 4431523328 completion_queue.cc:960] grpc_completion_queue_next(cq=0x10540bd90, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 14:19:18.013118000 4431523328 completion_queue.cc:1060] RETURN_EVENT[0x10540bd90]: OP_COMPLETE: tag:0x105062dc0 ERROR
I0503 14:19:18.013290000 4431523328 metadata_array.cc:34] grpc_metadata_array_destroy(array=0x1050619d8)
I0503 14:19:18.013306000 4431523328 completion_queue.cc:960] grpc_completion_queue_next(cq=0x10540bd90, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 14:19:18.013316000 4431523328 completion_queue.cc:1060] RETURN_EVENT[0x10540bd90]: OP_COMPLETE: tag:0x10510f120 ERROR
I0503 14:19:18.013449000 4431523328 completion_queue.cc:960] grpc_completion_queue_next(cq=0x10540bd90, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 14:19:18.013469000 4431523328 completion_queue.cc:1060] RETURN_EVENT[0x10540bd90]: OP_COMPLETE: tag:0x1051100a0 OK
Error: 4 DEADLINE_EXCEEDED: Deadline Exceeded
at Object.exports.createStatusError (/Users/canyonman/Development/BlockSpaces/connect/test/node_modules/grpc/src/common.js:91:15)
at Object.onReceiveStatus (/Users/canyonman/Development/BlockSpaces/connect/test/node_modules/grpc/src/client_interceptors.js:1209:28)
at InterceptingListener._callNext (/Users/canyonman/Development/BlockSpaces/connect/test/node_modules/grpc/src/client_interceptors.js:568:42)
at InterceptingListener.onReceiveStatus (/Users/canyonman/Development/BlockSpaces/connect/test/node_modules/grpc/src/client_interceptors.js:618:8)
at callback (/Users/canyonman/Development/BlockSpaces/connect/test/node_modules/grpc/src/client_interceptors.js:847:24) {
code: 4,
metadata: Metadata { _internal_repr: {}, flags: 0 },
details: 'Deadline Exceeded'
}
Have a blessed day,
Chris Tyler
CTO | BlockSpaces, Inc.
214.673.0483
chris...@blockspaces.io
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee, you should not disseminate, distribute or copy this email. Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.
Michael Lumish
May 3, 2021, 4:26:40 PM5/3/21
to Chris Tyler, grpc.io
OK, what's in the rest of the client log? The earlier part probably shows what happened when it tried to establish a connection to the server.
Chris Tyler
May 3, 2021, 5:57:34 PM5/3/21
to Michael Lumish, grpc.io
I’ve seen a lot of examples using the nginx-ingress. Is that required. Currently, I just have my gRPC server service with a NodePort or LoadBalancer type exposing the port. I try and connect directly from my client to my <worker node ip address>:<exposed gRPC service port>. I’m attempting now to go through the full nginx-ingress config to test it, but below is the log with no nginx-ingress in the mix.
Here is the complete log.
canyonman@CanyonMans-MacBook-Pro test % export GRPC_TRACE=all
canyonman@CanyonMans-MacBook-Pro test % export GRPC_VERBOSITY=DEBUG
canyonman@CanyonMans-MacBook-Pro test % node test.js '{id: "1234-5678", data:{id:21}}'
D0503 16:53:20.904622000 4521967104 dns_resolver.cc:294] Using native dns resolver
I0503 16:53:20.906078000 4521967104 timer_manager.cc:88] Spawn timer thread
I0503 16:53:20.906101000 4521967104 init.cc:164] grpc_init(void)
I0503 16:53:20.906130000 123145361285120 timer_manager.cc:250] timers not checked: expect another thread to
I0503 16:53:20.906142000 123145361285120 timer_manager.cc:197] sleep until kicked
I0503 16:53:20.906396000 4521967104 completion_queue.cc:504] grpc_completion_queue_create_internal(completion_type=0, polling_type=0)
[2021-05-03T16:53:21.002] [INFO] appid-sdk - Initialized
I0503 16:53:21.200044000 4521967104 channel_create.cc:97] grpc_insecure_channel_create(target=169.61.238.230:30052, args=0x104720a40, reserved=0x0)
I0503 16:53:21.200097000 4521967104 init.cc:164] grpc_init(void)
I0503 16:53:21.200138000 4521967104 client_idle_filter.cc:294] (client idle filter) created with max_leisure_time = 1800000 ms
I0503 16:53:21.200154000 4521967104 client_channel.cc:1367] chand=0x104873e60: creating client_channel for channel stack 0x104873c80
client ServiceClient {
'$interceptors': [],
'$interceptor_providers': [],
'$callInvocationTransformer': undefined,
'$channel': Channel {}
}
[2021-05-03T16:53:21.201] [INFO] appid-TokenManager-config - clientId 57c6796c-f5d7-4f2a-b71e-0207881ea26d
[2021-05-03T16:53:21.201] [INFO] appid-TokenManager-config - secret [CANNOT LOG SECRET]
[2021-05-03T16:53:21.201] [INFO] appid-TokenManager-config - oauthServerUrl https://us-south.appid.cloud.ibm.com/oauth/v4/8da4b5c5-db05-4b38-b527-8fa17bebea33
I0503 16:53:21.228635000 4521967104 credentials.cc:43] grpc_channel_credentials_release(creds=0x0)
[2021-05-03T16:53:21.703] [INFO] appid-public-key-util - Public keys updated
[2021-05-03T16:53:21.707] [INFO] appid-token-util - token ver: 4
[2021-05-03T16:53:21.707] [INFO] appid-token-util - token version is valid
[2021-05-03T16:53:21.707] [INFO] appid-token-util - token aud: 57c6796c-f5d7-4f2a-b71e-0207881ea26d
[2021-05-03T16:53:21.857] [INFO] appid-token-util - token iss: https://us-south.appid.cloud.ibm.com/oauth/v4/8da4b5c5-db05-4b38-b527-8fa17bebea33
rawJSON {id: "1234-5678", data:{id:21}}
fixedJSON { id: '1234-5678', data: { id: 21 } }
sending message { id: '1234-5678', data: <Buffer 7b 22 69 64 22 3a 32 31 7d> } Metadata {
_internal_repr: {
authorization: [
'Bearer <removed>
]
},
flags: 0
}
I0503 16:53:21.862270000 4521967104 client_idle_filter.cc:206] (client idle filter) call counter has increased to 1
I0503 16:53:21.862315000 4521967104 channel.cc:322] grpc_channel_get_target(channel=0x104873c00)
I0503 16:53:21.863008000 4521967104 metadata_array.cc:29] grpc_metadata_array_init(array=0x105175368)
I0503 16:53:21.863066000 4521967104 call.cc:1967] grpc_call_start_batch(call=0x106065660, ops=0x105169540, nops=1, tag=0x1051753f0, reserved=0x0)
I0503 16:53:21.863089000 4521967104 call.cc:1566] ops[0]: SEND_INITIAL_METADATA
key=authorization value=42 65 61 72 65 72 20 65 79 4a 68 62 47 63 69 4f 69 4a 53 55 7a 49 31 4e 69 49 73 49 6e 52 35 63 43 49 36 49 6b 70 58 56 43 49 73 49 6d 74 70 5a 43 49 36 49 6d 46 77 63 45 6c 6b 4c 54 68 6b 59 54 52 69 4e 57 4d 31 4c 57 52 69 4d 44 55 74 4e 47 49 7a 4f 43 31 69 4e 54 49 33 4c 54 68 6d 59 54 45 33 59 6d 56 69 5a 57 45 7a 4d 79 30 79 4d 44 49 78 4c 54 41 30 4c 54 41 34 56 44 41 7a 4f 6a 41 33 4f 6a 41 33 4c 6a 51 7a 4e 43 49 73 49 6e 5a 6c 63 69 49 36 4e 48 30 2e 65 79 4a 70 63 33 4d 69 4f 69 4a 6f 64 48 52 77 63 7a 6f 76 4c 33 56 7a 4c 58 4e 76 64 58 52 6f 4c 6d 46 77 63 47 6c 6b 4c 6d 4e 73 62 33 56 6b 4c 6d 6c 69 62 53 35 6a 62 32 30 76 62 32 46 31 64 47 67 76 64 6a 51 76 4f 47 52 68 4e 47 49 31 59 7a 55 74 5a 47 49 77 4e 53 30 30 59 6a 4d 34 4c 57 49 31 4d 6a 63 74 4f 47 5a 68 4d 54 64 69 5a 57 4a 6c 59 54 4d 7a 49 69 77 69 5a 58 68 77 49 6a 6f 78 4e 6a 49 77 4d 44 67 79 4e 44 41 78 4c 43 4a 68 64 57 51 69 4f 6c 73 69 4e 54 64 6a 4e 6a 63 35 4e 6d 4d 74 5a 6a 56 6b 4e 79 30 30 5a 6a 4a 68 4c 57 49 33 4d 57 55 74 4d 44 49 77 4e 7a 67 34 4d 57 56 68 4d 6a 5a 6b 49 6c 30 73 49 6e 4e 31 59 69 49 36 49 6a 55 33 59 7a 59 33 4f 54 5a 6a 4c 57 59 31 5a 44 63 74 4e 47 59 79 59 53 31 69 4e 7a 46 6c 4c 54 41 79 4d 44 63 34 4f 44 46 6c 59 54 49 32 5a 43 49 73 49 6d 46 74 63 69 49 36 57 79 4a 68 63 48 42 70 5a 46 39 6a 62 47 6c 6c 62 6e 52 66 59 33 4a 6c 5a 47 56 75 64 47 6c 68 62 48 4d 69 58 53 77 69 61 57 46 30 49 6a 6f 78 4e 6a 49 77 4d 44 63 34 4f 44 41 78 4c 43 4a 30 5a 57 35 68 62 6e 51 69 4f 69 49 34 5a 47 45 30 59 6a 56 6a 4e 53 31 6b 59 6a 41 31 4c 54 52 69 4d 7a 67 74 59 6a 55 79 4e 79 30 34 5a 6d 45 78 4e 32 4a 6c 59 6d 56 68 4d 7a 4d 69 4c 43 4a 7a 59 32 39 77 5a 53 49 36 49 6d 46 77 63 47 6c 6b 58 32 52 6c 5a 6d 46 31 62 48 51 69 66 51 2e 54 63 64 4b 65 2d 71 4d 45 6f 4f 49 36 69 4a 69 58 57 6e 6f 4e 38 65 58 58 45 30 78 66 6a 73 32 79 50 30 6b 31 45 52 5f 56 59 63 67 42 5a 39 78 65 57 32 38 6b 69 62 41 54 77 78 62 36 55 4b 58 42 74 4b 79 4e 79 67 51 49 46 36 74 41 36 30 6f 47 33 70 6c 64 35 66 5a 33 55 38 37 41 4a 48 73 59 4a 4d 77 6b 6f 51 73 34 38 43 63 4d 2d 4d 74 4f 42 70 58 34 76 6b 62 32 42 4e 73 32 70 47 6f 5a 76 38 6b 77 4e 57 57 36 42 78 6b 5a 54 48 41 66 61 63 7a 6a 33 50 2d 70 6f 7a 72 76 45 71 34 68 36 73 7a 68 50 51 4f 37 45 51 67 4e 32 67 4d 68 49 33 6a 51 36 72 75 59 46 61 68 50 74 74 70 6b 51 73 52 63 75 55 44 75 6c 64 38 73 58 69 53 42 50 34 52 5a 35 49 35 73 34 53 54 6e 61 6a 4b 7a 32 78 43 59 59 46 32 34 77 6b 44 45 63 74 50 52 61 44 4c 54 5f 57 6b 68 30 7a 65 53 53 2d 62 43 65 39 4d 31 75 57 4a 39 30 39 37 65 39 74 4d 41 44 6e 76 48 64 4c 64 4d 5f 47 33 5a 6b 76 79 6d 6a 73 79 58 6e 34 78 32 35 55 74 46 4b 67 52 74 5f 39 74 64 4b 47 36 5f 68 64 75 55 55 6e 70 79 43 72 72 6b 5a 53 33 30 57 6d 65 6d 36 67 5f 7a 41 50 53 69 77 'Bearer <removed>'
I0503 16:53:21.865061000 4521967104 call.cc:626] OP[client_idle:0x106065ff0]: SEND_INITIAL_METADATA{key=3a 70 61 74 68 ':path' value=2f 62 6c 6f 63 6b 73 70 61 63 65 73 2e 43 6f 6e 6e 65 63 74 2f 53 65 6e 64 '/blockspaces.Connect/Send', key=61 75 74 68 6f 72 69 7a 61 74 69 6f 6e 'authorization' value=42 65 61 72 65 72 20 65 79 4a 68 62 47 63 69 4f 69 4a 53 55 7a 49 31 4e 69 49 73 49 6e 52 35 63 43 49 36 49 6b 70 58 56 43 49 73 49 6d 74 70 5a 43 49 36 49 6d 46 77 63 45 6c 6b 4c 54 68 6b 59 54 52 69 4e 57 4d 31 4c 57 52 69 4d 44 55 74 4e 47 49 7a 4f 43 31 69 4e 54 49 33 4c 54 68 6d 59 54 45 33 59 6d 56 69 5a 57 45 7a 4d 79 30 79 4d 44 49 78 4c 54 41 30 4c 54 41 34 56 44 41 7a 4f 6a 41 33 4f 6a 41 33 4c 6a 51 7a 4e 43 49 73 49 6e 5a 6c 63 69 49 36 4e 48 30 2e 65 79 4a 70 63 33 4d 69 4f 69 4a 6f 64 48 52 77 63 7a 6f 76 4c 33 56 7a 4c 58 4e 76 64 58 52 6f 4c 6d 46 77 63 47 6c 6b 4c 6d 4e 73 62 33 56 6b 4c 6d 6c 69 62 53 35 6a 62 32 30 76 62 32 46 31 64 47 67 76 64 6a 51 76 4f 47 52 68 4e 47 49 31 59 7a 55 74 5a 47 49 77 4e 53 30 30 59 6a 4d 34 4c 57 49 31 4d 6a 63 74 4f 47 5a 68 4d 54 64 69 5a 57 4a 6c 59 54 4d 7a 49 69 77 69 5a 58 68 77 49 6a 6f 78 4e 6a 49 77 4d 44 67 79 4e 44 41 78 4c 43 4a 68 64 57 51 69 4f 6c 73 69 4e 54 64 6a 4e 6a 63 35 4e 6d 4d 74 5a 6a 56 6b 4e 79 30 30 5a 6a 4a 68 4c 57 49 33 4d 57 55 74 4d 44 49 77 4e 7a 67 34 4d 57 56 68 4d 6a 5a 6b 49 6c 30 73 49 6e 4e 31 59 69 49 36 49 6a 55 33 59 7a 59 33 4f 54 5a 6a 4c 57 59 31 5a 44 63 74 4e 47 59 79 59 53 31 69 4e 7a 46 6c 4c 54 41 79 4d 44 63 34 4f 44 46 6c 59 54 49 32 5a 43 49 73 49 6d 46 74 63 69 49 36 57 79 4a 68 63 48 42 70 5a 46 39 6a 62 47 6c 6c 62 6e 52 66 59 33 4a 6c 5a 47 56 75 64 47 6c 68 62 48 4d 69 58 53 77 69 61 57 46 30 49 6a 6f 78 4e 6a 49 77 4d 44 63 34 4f 44 41 78 4c 43 4a 30 5a 57 35 68 62 6e 51 69 4f 69 49 34 5a 47 45 30 59 6a 56 6a 4e 53 31 6b 59 6a 41 31 4c 54 52 69 4d 7a 67 74 59 6a 55 79 4e 79 30 34 5a 6d 45 78 4e 32 4a 6c 59 6d 56 68 4d 7a 4d 69 4c 43 4a 7a 59 32 39 77 5a 53 49 36 49 6d 46 77 63 47 6c 6b 58 32 52 6c 5a 6d 46 31 62 48 51 69 66 51 2e 54 63 64 4b 65 2d 71 4d 45 6f 4f 49 36 69 4a 69 58 57 6e 6f 4e 38 65 58 58 45 30 78 66 6a 73 32 79 50 30 6b 31 45 52 5f 56 59 63 67 42 5a 39 78 65 57 32 38 6b 69 62 41 54 77 78 62 36 55 4b 58 42 74 4b 79 4e 79 67 51 49 46 36 74 41 36 30 6f 47 33 70 6c 64 35 66 5a 33 55 38 37 41 4a 48 73 59 4a 4d 77 6b 6f 51 73 34 38 43 63 4d 2d 4d 74 4f 42 70 58 34 76 6b 62 32 42 4e 73 32 70 47 6f 5a 76 38 6b 77 4e 57 57 36 42 78 6b 5a 54 48 41 66 61 63 7a 6a 33 50 2d 70 6f 7a 72 76 45 71 34 68 36 73 7a 68 50 51 4f 37 45 51 67 4e 32 67 4d 68 49 33 6a 51 36 72 75 59 46 61 68 50 74 74 70 6b 51 73 52 63 75 55 44 75 6c 64 38 73 58 69 53 42 50 34 52 5a 35 49 35 73 34 53 54 6e 61 6a 4b 7a 32 78 43 59 59 46 32 34 77 6b 44 45 63 74 50 52 61 44 4c 54 5f 57 6b 68 30 7a 65 53 53 2d 62 43 65 39 4d 31 75 57 4a 39 30 39 37 65 39 74 4d 41 44 6e 76 48 64 4c 64 4d 5f 47 33 5a 6b 76 79 6d 6a 73 79 58 6e 34 78 32 35 55 74 46 4b 67 52 74 5f 39 74 64 4b 47 36 5f 68 64 75 55 55 6e 70 79 43 72 72 6b 5a 53 33 30 57 6d 65 6d 36 67 5f 7a 41 50 53 69 77 'Bearer <removed>' deadline=10957}
I0503 16:53:21.866184000 4521967104 channel_stack.cc:226] OP[client-channel:0x106066008]: SEND_INITIAL_METADATA{key=3a 70 61 74 68 ':path' value=2f 62 6c 6f 63 6b 73 70 61 63 65 73 2e 43 6f 6e 6e 65 63 74 2f 53 65 6e 64 '/blockspaces.Connect/Send', key=61 75 74 68 6f 72 69 7a 61 74 69 6f 6e 'authorization' value=42 65 61 72 65 72 20 65 79 4a 68 62 47 63 69 4f 69 4a 53 55 7a 49 31 4e 69 49 73 49 6e 52 35 63 43 49 36 49 6b 70 58 56 43 49 73 49 6d 74 70 5a 43 49 36 49 6d 46 77 63 45 6c 6b 4c 54 68 6b 59 54 52 69 4e 57 4d 31 4c 57 52 69 4d 44 55 74 4e 47 49 7a 4f 43 31 69 4e 54 49 33 4c 54 68 6d 59 54 45 33 59 6d 56 69 5a 57 45 7a 4d 79 30 79 4d 44 49 78 4c 54 41 30 4c 54 41 34 56 44 41 7a 4f 6a 41 33 4f 6a 41 33 4c 6a 51 7a 4e 43 49 73 49 6e 5a 6c 63 69 49 36 4e 48 30 2e 65 79 4a 70 63 33 4d 69 4f 69 4a 6f 64 48 52 77 63 7a 6f 76 4c 33 56 7a 4c 58 4e 76 64 58 52 6f 4c 6d 46 77 63 47 6c 6b 4c 6d 4e 73 62 33 56 6b 4c 6d 6c 69 62 53 35 6a 62 32 30 76 62 32 46 31 64 47 67 76 64 6a 51 76 4f 47 52 68 4e 47 49 31 59 7a 55 74 5a 47 49 77 4e 53 30 30 59 6a 4d 34 4c 57 49 31 4d 6a 63 74 4f 47 5a 68 4d 54 64 69 5a 57 4a 6c 59 54 4d 7a 49 69 77 69 5a 58 68 77 49 6a 6f 78 4e 6a 49 77 4d 44 67 79 4e 44 41 78 4c 43 4a 68 64 57 51 69 4f 6c 73 69 4e 54 64 6a 4e 6a 63 35 4e 6d 4d 74 5a 6a 56 6b 4e 79 30 30 5a 6a 4a 68 4c 57 49 33 4d 57 55 74 4d 44 49 77 4e 7a 67 34 4d 57 56 68 4d 6a 5a 6b 49 6c 30 73 49 6e 4e 31 59 69 49 36 49 6a 55 33 59 7a 59 33 4f 54 5a 6a 4c 57 59 31 5a 44 63 74 4e 47 59 79 59 53 31 69 4e 7a 46 6c 4c 54 41 79 4d 44 63 34 4f 44 46 6c 59 54 49 32 5a 43 49 73 49 6d 46 74 63 69 49 36 57 79 4a 68 63 48 42 70 5a 46 39 6a 62 47 6c 6c 62 6e 52 66 59 33 4a 6c 5a 47 56 75 64 47 6c 68 62 48 4d 69 58 53 77 69 61 57 46 30 49 6a 6f 78 4e 6a 49 77 4d 44 63 34 4f 44 41 78 4c 43 4a 30 5a 57 35 68 62 6e 51 69 4f 69 49 34 5a 47 45 30 59 6a 56 6a 4e 53 31 6b 59 6a 41 31 4c 54 52 69 4d 7a 67 74 59 6a 55 79 4e 79 30 34 5a 6d 45 78 4e 32 4a 6c 59 6d 56 68 4d 7a 4d 69 4c 43 4a 7a 59 32 39 77 5a 53 49 36 49 6d 46 77 63 47 6c 6b 58 32 52 6c 5a 6d 46 31 62 48 51 69 66 51 2e 54 63 64 4b 65 2d 71 4d 45 6f 4f 49 36 69 4a 69 58 57 6e 6f 4e 38 65 58 58 45 30 78 66 6a 73 32 79 50 30 6b 31 45 52 5f 56 59 63 67 42 5a 39 78 65 57 32 38 6b 69 62 41 54 77 78 62 36 55 4b 58 42 74 4b 79 4e 79 67 51 49 46 36 74 41 36 30 6f 47 33 70 6c 64 35 66 5a 33 55 38 37 41 4a 48 73 59 4a 4d 77 6b 6f 51 73 34 38 43 63 4d 2d 4d 74 4f 42 70 58 34 76 6b 62 32 42 4e 73 32 70 47 6f 5a 76 38 6b 77 4e 57 57 36 42 78 6b 5a 54 48 41 66 61 63 7a 6a 33 50 2d 70 6f 7a 72 76 45 71 34 68 36 73 7a 68 50 51 4f 37 45 51 67 4e 32 67 4d 68 49 33 6a 51 36 72 75 59 46 61 68 50 74 74 70 6b 51 73 52 63 75 55 44 75 6c 64 38 73 58 69 53 42 50 34 52 5a 35 49 35 73 34 53 54 6e 61 6a 4b 7a 32 78 43 59 59 46 32 34 77 6b 44 45 63 74 50 52 61 44 4c 54 5f 57 6b 68 30 7a 65 53 53 2d 62 43 65 39 4d 31 75 57 4a 39 30 39 37 65 39 74 4d 41 44 6e 76 48 64 4c 64 4d 5f 47 33 5a 6b 76 79 6d 6a 73 79 58 6e 34 78 32 35 55 74 46 4b 67 52 74 5f 39 74 64 4b 47 36 5f 68 64 75 55 55 6e 70 79 43 72 72 6b 5a 53 33 30 57 6d 65 6d 36 67 5f 7a 41 50 53 69 77 'Bearer <removed>' deadline=10957}
I0503 16:53:21.868183000 4521967104 client_channel.cc:2249] chand=0x104873e60 calld=0x106066030: adding pending batch at index 0
I0503 16:53:21.868194000 4521967104 client_channel.cc:2075] chand=0x104873e60 calld=0x106066030: grabbing data plane mutex to perform pick
I0503 16:53:21.868202000 4521967104 connectivity_state.cc:79] CONWATCH: 0x104873f30 client_channel: get IDLE
I0503 16:53:21.868208000 4521967104 client_channel.cc:3699] chand=0x104873e60 calld=0x106066030: adding to queued picks list
I0503 16:53:21.868228000 4521967104 resolving_lb_policy.cc:196] resolving_lb=0x10476dd70: starting name resolution
I0503 16:53:21.868235000 4521967104 client_channel.cc:1273] chand=0x104873e60: update: state=CONNECTING picker=0x104706420
I0503 16:53:21.868285000 4521967104 connectivity_state.cc:147] SET: 0x104873f30 client_channel: IDLE --> CONNECTING [helper]
I0503 16:53:21.868303000 4521967104 client_channel.cc:3847] chand=0x104873e60 calld=0x106066030: LB pick returned QUEUE (subchannel=0x0, error="No Error")
D0503 16:53:21.868309000 4521967104 dns_resolver.cc:242] Start resolving.
I0503 16:53:21.868376000 4521967104 client_channel.cc:1551] chand=0x104873e60: created resolving_lb_policy=0x10476dd70
I0503 16:53:21.869437000 4521967104 call.cc:1967] grpc_call_start_batch(call=0x106065660, ops=0x105317c70, nops=1, tag=0x105308870, reserved=0x0)
I0503 16:53:21.869454000 4521967104 call.cc:1566] ops[0]: SEND_MESSAGE ptr=0x10536e3e0
I0503 16:53:21.869547000 4521967104 metadata_array.cc:29] grpc_metadata_array_init(array=0x105317da8)
I0503 16:53:21.869558000 4521967104 metadata_array.cc:29] grpc_metadata_array_init(array=0x105317c78)
I0503 16:53:21.869566000 4521967104 call.cc:1967] grpc_call_start_batch(call=0x106065660, ops=0x10536e560, nops=4, tag=0x10536e7a0, reserved=0x0)
I0503 16:53:21.869571000 4521967104 call.cc:1566] ops[0]: SEND_CLOSE_FROM_CLIENT
I0503 16:53:21.869576000 4521967104 call.cc:1566] ops[1]: RECV_INITIAL_METADATA ptr=0x105317da8
I0503 16:53:21.869580000 4521967104 call.cc:1566] ops[2]: RECV_MESSAGE ptr=0x105305d08
I0503 16:53:21.869585000 4521967104 call.cc:1566] ops[3]: RECV_STATUS_ON_CLIENT metadata=0x105317c78 status=0x105317c90 details=0x105317c98
I0503 16:53:21.870083000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:21.870096000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:21.870122000 4521967104 resolving_lb_policy.cc:441] resolving_lb=0x10476dd70: got resolver result
I0503 16:53:21.870128000 4521967104 client_channel.cc:3712] chand=0x104873e60 calld=0x106066030: applying service config to call
I0503 16:53:21.870133000 4521967104 resolving_lb_policy.cc:339] resolving_lb=0x10476dd70: Creating new child policy pick_first
I0503 16:53:21.870142000 4521967104 pick_first.cc:146] Pick First 0x10536e520 created.
I0503 16:53:21.870146000 4521967104 resolving_lb_policy.cc:393] resolving_lb=0x10476dd70: created new LB policy "pick_first" (0x10536e520)
I0503 16:53:21.870151000 4521967104 resolving_lb_policy.cc:356] resolving_lb=0x10476dd70: Updating child policy 0x10536e520
I0503 16:53:21.870155000 4521967104 pick_first.cc:265] Pick First 0x10536e520 received update with 1 addresses
I0503 16:53:21.870165000 4521967104 subchannel_list.h:370] [pick_first 0x10536e520] Creating subchannel list 0x10536e8b0 for 1 subchannels
I0503 16:53:21.870241000 4521967104 client_channel.cc:831] chand=0x104873e60: creating subchannel wrapper 0x105317af0 for subchannel 0x10536f1e0
I0503 16:53:21.870253000 4521967104 subchannel_list.h:420] [pick_first 0x10536e520] subchannel list 0x10536e8b0 index 0: Created subchannel 0x105317af0 for address uri ipv4:169.61.238.230:30052
I0503 16:53:21.870259000 4521967104 subchannel_list.h:316] [pick_first 0x10536e520] subchannel list 0x10536e8b0 index 0 of 1 (subchannel 0x105317af0): starting watch (from IDLE)
I0503 16:53:21.870268000 4521967104 client_channel.cc:992] chand=0x104873e60: connectivity change for subchannel wrapper 0x105317af0 subchannel 0x10536f1e0 (connected_subchannel=0x0 state=CONNECTING); hopping into combiner
I0503 16:53:21.870283000 4521967104 tcp_client_custom.cc:151] CLIENT_CONNECT: 0x10536eb20 ipv4:169.61.238.230:30052: asynchronously connecting
I0503 16:53:21.870395000 4521967104 client_channel.cc:1040] chand=0x104873e60: processing connectivity change in combiner for subchannel wrapper 0x105317af0 subchannel 0x10536f1e0 (connected_subchannel=0x0 state=CONNECTING): watcher=0x105308ff0
I0503 16:53:21.870892000 4521967104 subchannel_list.h:249] [pick_first 0x10536e520] subchannel list 0x10536e8b0 index 0 of 1 (subchannel 0x105317af0): connectivity changed: state=CONNECTING, shutting_down=0, pending_watcher=0x105308ff0
I0503 16:53:21.870903000 4521967104 client_channel.cc:1273] chand=0x104873e60: update: state=CONNECTING picker=0x10500fa80
I0503 16:53:21.870909000 4521967104 connectivity_state.cc:147] SET: 0x104873f30 client_channel: CONNECTING --> CONNECTING [helper]
I0503 16:53:21.870916000 4521967104 client_channel.cc:3847] chand=0x104873e60 calld=0x106066030: LB pick returned QUEUE (subchannel=0x0, error="No Error")
I0503 16:53:21.871107000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:21.874443000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:21.875357000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:21.875375000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:26.143342000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:26.143383000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:26.143444000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:26.143458000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.306813000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.306854000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.307399000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.307423000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.307735000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.307760000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.308002000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.308015000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.308259000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.308270000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.308492000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.308504000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.308748000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.308759000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.308989000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.308998000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.309220000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.310072000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.310261000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.310274000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.310407000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.310417000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.310626000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.310641000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.310841000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.310853000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.311048000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.311057000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.311139000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.311929000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.312090000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.312101000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.312230000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.312240000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.312384000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.313686000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.313870000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.313882000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.314040000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.314048000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.314229000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.314940000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.315049000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.315059000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.315191000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.315199000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.315284000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.317602000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.317701000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.317710000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.317865000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.317873000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.317945000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.317951000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.319693000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.319705000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.325714000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.325735000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.326416000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.326436000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.326521000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.326532000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.929766000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.929805000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.930206000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.930224000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.930486000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.930500000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.930780000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.930793000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.931013000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.931024000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.931244000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.931256000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.931423000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.931433000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.931634000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.931644000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.931851000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.931861000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.932084000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.932096000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.932330000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.932340000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.932565000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.932575000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.932809000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.934897000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.935072000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.935084000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.935221000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.935230000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.935740000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.935757000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.935836000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.935845000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.935964000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.935972000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.936217000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.936794000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.936973000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.936984000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.937278000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.937286000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.937428000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.938672000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.938811000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.938821000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.938955000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.938962000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.939101000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.940858000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.941035000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.941045000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.941066000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.941072000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.941141000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.942918000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.943046000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.943056000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.945305000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.945324000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.945578000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.945595000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:29.945705000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:29.945716000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:31.143515000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:31.143559000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:31.143666000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:31.143695000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:31.857223000 4521967104 client_channel.cc:3663] chand=0x104873e60 calld=0x106066030: cancelling queued pick: error={"created":"@1620078811.857179000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4} self=0x104768950 calld->pick_canceller=0x104768950
I0503 16:53:31.857259000 4521967104 client_channel.cc:3687] chand=0x104873e60 calld=0x106066030: removing from queued picks list
I0503 16:53:31.857274000 4521967104 client_channel.cc:2359] chand=0x104873e60 calld=0x106066030: failing 1 pending batches: {"created":"@1620078811.857179000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}
I0503 16:53:31.857293000 4521967104 completion_queue.cc:682] cq_end_op_for_next(cq=0x104761030, tag=0x1051753f0, error={"created":"@1620078811.857179000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}, done=0x10883abe0, done_arg=0x106066510, storage=0x106066560)
E0503 16:53:31.857374000 4521967104 completion_queue.cc:685] Operation failed: tag=0x1051753f0, error={"created":"@1620078811.857179000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}
I0503 16:53:31.857402000 4521967104 call.cc:626] OP[client_idle:0x106065ff0]: SEND_MESSAGE:flags=0x00000000:len=22
I0503 16:53:31.857414000 4521967104 channel_stack.cc:226] OP[client-channel:0x106066008]: SEND_MESSAGE:flags=0x00000000:len=22
I0503 16:53:31.857423000 4521967104 client_channel.cc:2249] chand=0x104873e60 calld=0x106066030: adding pending batch at index 1
I0503 16:53:31.857431000 4521967104 client_channel.cc:2083] chand=0x104873e60 calld=0x106066030: saved batch, yielding call combiner
I0503 16:53:31.857446000 4521967104 call.cc:626] OP[client_idle:0x106065ff0]: SEND_TRAILING_METADATA{} RECV_INITIAL_METADATA RECV_MESSAGE RECV_TRAILING_METADATA
I0503 16:53:31.857566000 4521967104 channel_stack.cc:226] OP[client-channel:0x106066008]: SEND_TRAILING_METADATA{} RECV_INITIAL_METADATA RECV_MESSAGE RECV_TRAILING_METADATA
I0503 16:53:31.857579000 4521967104 client_channel.cc:2249] chand=0x104873e60 calld=0x106066030: adding pending batch at index 2
I0503 16:53:31.857587000 4521967104 client_channel.cc:2083] chand=0x104873e60 calld=0x106066030: saved batch, yielding call combiner
I0503 16:53:31.857597000 4521967104 client_channel.cc:2035] chand=0x104873e60 calld=0x106066030: recording cancel_error={"created":"@1620078811.857179000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}
I0503 16:53:31.857606000 4521967104 client_channel.cc:2359] chand=0x104873e60 calld=0x106066030: failing 2 pending batches: {"created":"@1620078811.857179000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}
I0503 16:53:31.857619000 4521967104 call.cc:626] OP[client_idle:0x106065ff0]: CANCEL:{"created":"@1620078811.857179000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}
I0503 16:53:31.857629000 4521967104 channel_stack.cc:226] OP[client-channel:0x106066008]: CANCEL:{"created":"@1620078811.857179000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}
I0503 16:53:31.857637000 4521967104 client_channel.cc:2016] chand=0x104873e60 calld=0x106066030: failing batch with error: {"created":"@1620078811.857179000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}
I0503 16:53:31.857715000 4521967104 completion_queue.cc:682] cq_end_op_for_next(cq=0x104761030, tag=0x105308870, error={"created":"@1620078811.857179000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}, done=0x10883abe0, done_arg=0x1060665f0, storage=0x106066640)
E0503 16:53:31.857731000 4521967104 completion_queue.cc:685] Operation failed: tag=0x105308870, error={"created":"@1620078811.857179000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}
D0503 16:53:31.857740000 4521967104 call.cc:733] set_final_status CLI
D0503 16:53:31.857748000 4521967104 call.cc:734] {"created":"@1620078811.857179000","description":"Deadline Exceeded","file":"../deps/grpc/src/core/ext/filters/deadline/deadline_filter.cc","file_line":69,"grpc_status":4}
I0503 16:53:31.857758000 4521967104 completion_queue.cc:682] cq_end_op_for_next(cq=0x104761030, tag=0x10536e7a0, error="No Error", done=0x10883abe0, done_arg=0x1060666d0, storage=0x106066720)
I0503 16:53:31.857772000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:31.857784000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: OP_COMPLETE: tag:0x1051753f0 ERROR
I0503 16:53:31.857973000 4521967104 metadata_array.cc:34] grpc_metadata_array_destroy(array=0x105175368)
I0503 16:53:31.857997000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:31.858009000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: OP_COMPLETE: tag:0x105308870 ERROR
I0503 16:53:31.858057000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:31.858071000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: OP_COMPLETE: tag:0x10536e7a0 OK
Error: 4 DEADLINE_EXCEEDED: Deadline Exceeded
at Object.exports.createStatusError (/Users/canyonman/Development/BlockSpaces/connect/test/node_modules/grpc/src/common.js:91:15)
at Object.onReceiveStatus (/Users/canyonman/Development/BlockSpaces/connect/test/node_modules/grpc/src/client_interceptors.js:1209:28)
at InterceptingListener._callNext (/Users/canyonman/Development/BlockSpaces/connect/test/node_modules/grpc/src/client_interceptors.js:568:42)
at InterceptingListener.onReceiveStatus (/Users/canyonman/Development/BlockSpaces/connect/test/node_modules/grpc/src/client_interceptors.js:618:8)
at callback (/Users/canyonman/Development/BlockSpaces/connect/test/node_modules/grpc/src/client_interceptors.js:847:24) {
code: 4,
metadata: Metadata { _internal_repr: {}, flags: 0 },
details: 'Deadline Exceeded'
}
I0503 16:53:31.860763000 4521967104 call.cc:573] grpc_call_unref(c=0x106065660)
I0503 16:53:31.860789000 4521967104 client_idle_filter.cc:245] (client idle filter) call counter has decreased to 0
I0503 16:53:31.860798000 4521967104 client_idle_filter.cc:361] (client idle filter) timer has started
I0503 16:53:31.860815000 4521967104 metadata_array.cc:34] grpc_metadata_array_destroy(array=0x105317c78)
I0503 16:53:31.860824000 4521967104 metadata_array.cc:34] grpc_metadata_array_destroy(array=0x105317da8)
I0503 16:53:31.860840000 4521967104 completion_queue.cc:960] grpc_completion_queue_next(cq=0x104761030, deadline=gpr_timespec { tv_sec: -9223372036854775808, tv_nsec: 0, clock_type: 0 }, reserved=0x0)
I0503 16:53:31.860850000 4521967104 completion_queue.cc:1060] RETURN_EVENT[0x104761030]: QUEUE_TIMEOUT
I0503 16:53:41.870250000 4521967104 tcp_client_custom.cc:69] CLIENT_CONNECT: ipv4:169.61.238.230:30052: on_alarm: error="No Error"
I0503 16:53:41.870392000 4521967104 subchannel.cc:1034] Connect failed: {"created":"@1620078821.870371000","description":"Timeout occurred","file":"../deps/grpc/src/core/lib/iomgr/tcp_uv.cc","file_line":318}
I0503 16:53:41.870415000 4521967104 client_channel.cc:992] chand=0x104873e60: connectivity change for subchannel wrapper 0x105317af0 subchannel 0x10536f1e0 (connected_subchannel=0x0 state=TRANSIENT_FAILURE); hopping into combiner
I0503 16:53:41.870426000 4521967104 client_channel.cc:1040] chand=0x104873e60: processing connectivity change in combiner for subchannel wrapper 0x105317af0 subchannel 0x10536f1e0 (connected_subchannel=0x0 state=TRANSIENT_FAILURE): watcher=0x105308ff0
I0503 16:53:41.870436000 4521967104 subchannel_list.h:249] [pick_first 0x10536e520] subchannel list 0x10536e8b0 index 0 of 1 (subchannel 0x105317af0): connectivity changed: state=TRANSIENT_FAILURE, shutting_down=0, pending_watcher=0x105308ff0
I0503 16:53:41.870444000 4521967104 subchannel_list.h:337] [pick_first 0x10536e520] subchannel list 0x10536e8b0 index 0 of 1 (subchannel 0x105317af0): canceling connectivity watch (connection attempt failed)
I0503 16:53:41.870480000 4521967104 resolving_lb_policy.cc:148] resolving_lb=0x10476dd70: started name re-resolving
D0503 16:53:41.870497000 4521967104 dns_resolver.cc:223] In cooldown from last resolution (from 20002 ms ago). Will resolve again in 9998 ms
I0503 16:53:41.870509000 4521967104 client_channel.cc:1273] chand=0x104873e60: update: state=TRANSIENT_FAILURE picker=0x10516df60
I0503 16:53:41.870517000 4521967104 connectivity_state.cc:147] SET: 0x104873f30 client_channel: CONNECTING --> TRANSIENT_FAILURE [helper]
I0503 16:53:41.870528000 4521967104 subchannel_list.h:316] [pick_first 0x10536e520] subchannel list 0x10536e8b0 index 0 of 1 (subchannel 0x105317af0): starting watch (from TRANSIENT_FAILURE)
I0503 16:53:41.870536000 4521967104 subchannel.cc:971] Subchannel 0x10536f1e0: Retry immediately
I0503 16:53:41.870543000 4521967104 subchannel.cc:998] Failed to connect to channel, retrying
I0503 16:53:41.870552000 4521967104 client_channel.cc:992] chand=0x104873e60: connectivity change for subchannel wrapper 0x105317af0 subchannel 0x10536f1e0 (connected_subchannel=0x0 state=CONNECTING); hopping into combiner
I0503 16:53:41.870577000 4521967104 tcp_client_custom.cc:151] CLIENT_CONNECT: 0x10516d1e0 ipv4:169.61.238.230:30052: asynchronously connecting
I0503 16:53:41.870719000 4521967104 client_channel.cc:1040] chand=0x104873e60: processing connectivity change in combiner for subchannel wrapper 0x105317af0 subchannel 0x10536f1e0 (connected_subchannel=0x0 state=CONNECTING): watcher=0x10516d110
I0503 16:53:41.870735000 4521967104 subchannel_list.h:249] [pick_first 0x10536e520] subchannel list 0x10536e8b0 index 0 of 1 (subchannel 0x105317af0): connectivity changed: state=CONNECTING, shutting_down=0, pending_watcher=0x10516d110
I0503 16:53:41.870742000 4521967104 client_channel.cc:1273] chand=0x104873e60: update: state=CONNECTING picker=0x105172490
I0503 16:53:41.870749000 4521967104 connectivity_state.cc:147] SET: 0x104873f30 client_channel: TRANSIENT_FAILURE --> CONNECTING [helper]
D0503 16:53:51.871929000 4521967104 dns_resolver.cc:242] Start resolving.
I0503 16:53:51.872250000 4521967104 resolving_lb_policy.cc:441] resolving_lb=0x10476dd70: got resolver result
I0503 16:53:51.872266000 4521967104 resolving_lb_policy.cc:356] resolving_lb=0x10476dd70: Updating child policy 0x10536e520
I0503 16:53:51.872273000 4521967104 pick_first.cc:265] Pick First 0x10536e520 received update with 1 addresses
I0503 16:53:51.872283000 4521967104 subchannel_list.h:370] [pick_first 0x10536e520] Creating subchannel list 0x105317b70 for 1 subchannels
I0503 16:53:51.872323000 4521967104 client_channel.cc:831] chand=0x104873e60: creating subchannel wrapper 0x10530bc80 for subchannel 0x10536f1e0
I0503 16:53:51.872332000 4521967104 subchannel_list.h:420] [pick_first 0x10536e520] subchannel list 0x105317b70 index 0: Created subchannel 0x10530bc80 for address uri ipv4:169.61.238.230:30052
I0503 16:53:51.872338000 4521967104 subchannel_list.h:442] [pick_first 0x10536e520] Shutting down subchannel_list 0x10536e8b0
I0503 16:53:51.872547000 4521967104 subchannel_list.h:337] [pick_first 0x10536e520] subchannel list 0x10536e8b0 index 0 of 1 (subchannel 0x105317af0): canceling connectivity watch (shutdown)
I0503 16:53:51.872559000 4521967104 subchannel_list.h:293] [pick_first 0x10536e520] subchannel list 0x10536e8b0 index 0 of 1 (subchannel 0x105317af0): unreffing subchannel
I0503 16:53:51.872564000 4521967104 client_channel.cc:850] chand=0x104873e60: destroying subchannel wrapper 0x105317af0 for subchannel 0x10536f1e0
I0503 16:53:51.872569000 4521967104 subchannel_list.h:434] [pick_first 0x10536e520] Destroying subchannel_list 0x10536e8b0
I0503 16:53:51.872575000 4521967104 subchannel_list.h:316] [pick_first 0x10536e520] subchannel list 0x105317b70 index 0 of 1 (subchannel 0x10530bc80): starting watch (from CONNECTING)
I0503 16:54:01.874290000 4521967104 tcp_client_custom.cc:69] CLIENT_CONNECT: ipv4:169.61.238.230:30052: on_alarm: error="No Error"
I0503 16:54:01.874451000 4521967104 subchannel.cc:1034] Connect failed: {"created":"@1620078841.874437000","description":"Timeout occurred","file":"../deps/grpc/src/core/lib/iomgr/tcp_uv.cc","file_line":318}
I0503 16:54:01.874465000 4521967104 client_channel.cc:992] chand=0x104873e60: connectivity change for subchannel wrapper 0x10530bc80 subchannel 0x10536f1e0 (connected_subchannel=0x0 state=TRANSIENT_FAILURE); hopping into combiner
I0503 16:54:01.874473000 4521967104 client_channel.cc:1040] chand=0x104873e60: processing connectivity change in combiner for subchannel wrapper 0x10530bc80 subchannel 0x10536f1e0 (connected_subchannel=0x0 state=TRANSIENT_FAILURE): watcher=0x10530bff0
I0503 16:54:01.874481000 4521967104 subchannel_list.h:249] [pick_first 0x10536e520] subchannel list 0x105317b70 index 0 of 1 (subchannel 0x10530bc80): connectivity changed: state=TRANSIENT_FAILURE, shutting_down=0, pending_watcher=0x10530bff0
I0503 16:54:01.874487000 4521967104 subchannel_list.h:337] [pick_first 0x10536e520] subchannel list 0x105317b70 index 0 of 1 (subchannel 0x10530bc80): canceling connectivity watch (connection attempt failed)
I0503 16:54:01.874559000 4521967104 resolving_lb_policy.cc:148] resolving_lb=0x10476dd70: started name re-resolving
D0503 16:54:01.874567000 4521967104 dns_resolver.cc:223] In cooldown from last resolution (from 10003 ms ago). Will resolve again in 19997 ms
I0503 16:54:01.874574000 4521967104 client_channel.cc:1273] chand=0x104873e60: update: state=TRANSIENT_FAILURE picker=0x10536ecd0
I0503 16:54:01.874580000 4521967104 connectivity_state.cc:147] SET: 0x104873f30 client_channel: CONNECTING --> TRANSIENT_FAILURE [helper]
I0503 16:54:01.874587000 4521967104 subchannel_list.h:316] [pick_first 0x10536e520] subchannel list 0x105317b70 index 0 of 1 (subchannel 0x10530bc80): starting watch (from TRANSIENT_FAILURE)
I0503 16:54:01.874592000 4521967104 subchannel.cc:971] Subchannel 0x10536f1e0: Retry immediately
I0503 16:54:01.874686000 4521967104 subchannel.cc:998] Failed to connect to channel, retrying
I0503 16:54:01.874695000 4521967104 client_channel.cc:992] chand=0x104873e60: connectivity change for subchannel wrapper 0x10530bc80 subchannel 0x10536f1e0 (connected_subchannel=0x0 state=CONNECTING); hopping into combiner
I0503 16:54:01.874713000 4521967104 tcp_client_custom.cc:151] CLIENT_CONNECT: 0x10530a5a0 ipv4:169.61.238.230:30052: asynchronously connecting
I0503 16:54:01.874824000 4521967104 client_channel.cc:1040] chand=0x104873e60: processing connectivity change in combiner for subchannel wrapper 0x10530bc80 subchannel 0x10536f1e0 (connected_subchannel=0x0 state=CONNECTING): watcher=0x105308350
I0503 16:54:01.874834000 4521967104 subchannel_list.h:249] [pick_first 0x10536e520] subchannel list 0x105317b70 index 0 of 1 (subchannel 0x10530bc80): connectivity changed: state=CONNECTING, shutting_down=0, pending_watcher=0x105308350
I0503 16:54:01.875619000 4521967104 client_channel.cc:1273] chand=0x104873e60: update: state=CONNECTING picker=0x105172490
I0503 16:54:01.875633000 4521967104 connectivity_state.cc:147] SET: 0x104873f30 client_channel: TRANSIENT_FAILURE --> CONNECTING [helper]
D0503 16:54:21.874935000 4521967104 dns_resolver.cc:242] Start resolving.
I0503 16:54:21.874977000 4521967104 tcp_client_custom.cc:69] CLIENT_CONNECT: ipv4:169.61.238.230:30052: on_alarm: error="No Error"
I0503 16:54:21.875219000 4521967104 subchannel.cc:1034] Connect failed: {"created":"@1620078861.875205000","description":"Timeout occurred","file":"../deps/grpc/src/core/lib/iomgr/tcp_uv.cc","file_line":318}
I0503 16:54:21.875234000 4521967104 client_channel.cc:992] chand=0x104873e60: connectivity change for subchannel wrapper 0x10530bc80 subchannel 0x10536f1e0 (connected_subchannel=0x0 state=TRANSIENT_FAILURE); hopping into combiner
I0503 16:54:21.875241000 4521967104 client_channel.cc:1040] chand=0x104873e60: processing connectivity change in combiner for subchannel wrapper 0x10530bc80 subchannel 0x10536f1e0 (connected_subchannel=0x0 state=TRANSIENT_FAILURE): watcher=0x105308350
I0503 16:54:21.875248000 4521967104 subchannel_list.h:249] [pick_first 0x10536e520] subchannel list 0x105317b70 index 0 of 1 (subchannel 0x10530bc80): connectivity changed: state=TRANSIENT_FAILURE, shutting_down=0, pending_watcher=0x105308350
I0503 16:54:21.875254000 4521967104 subchannel_list.h:337] [pick_first 0x10536e520] subchannel list 0x105317b70 index 0 of 1 (subchannel 0x10530bc80): canceling connectivity watch (connection attempt failed)
I0503 16:54:21.875325000 4521967104 resolving_lb_policy.cc:148] resolving_lb=0x10476dd70: started name re-resolving
I0503 16:54:21.875334000 4521967104 client_channel.cc:1273] chand=0x104873e60: update: state=TRANSIENT_FAILURE picker=0x10530b260
I0503 16:54:21.875339000 4521967104 connectivity_state.cc:147] SET: 0x104873f30 client_channel: CONNECTING --> TRANSIENT_FAILURE [helper]
I0503 16:54:21.875346000 4521967104 subchannel_list.h:316] [pick_first 0x10536e520] subchannel list 0x105317b70 index 0 of 1 (subchannel 0x10530bc80): starting watch (from TRANSIENT_FAILURE)
I0503 16:54:21.875352000 4521967104 subchannel.cc:971] Subchannel 0x10536f1e0: Retry immediately
I0503 16:54:21.875356000 4521967104 subchannel.cc:998] Failed to connect to channel, retrying
I0503 16:54:21.876734000 4521967104 client_channel.cc:992] chand=0x104873e60: connectivity change for subchannel wrapper 0x10530bc80 subchannel 0x10536f1e0 (connected_subchannel=0x0 state=CONNECTING); hopping into combiner
I0503 16:54:21.876763000 4521967104 tcp_client_custom.cc:151] CLIENT_CONNECT: 0x105308ac0 ipv4:169.61.238.230:30052: asynchronously connecting
I0503 16:54:21.876878000 4521967104 client_channel.cc:1040] chand=0x104873e60: processing connectivity change in combiner for subchannel wrapper 0x10530bc80 subchannel 0x10536f1e0 (connected_subchannel=0x0 state=CONNECTING): watcher=0x1053075e0
I0503 16:54:21.876889000 4521967104 subchannel_list.h:249] [pick_first 0x10536e520] subchannel list 0x105317b70 index 0 of 1 (subchannel 0x10530bc80): connectivity changed: state=CONNECTING, shutting_down=0, pending_watcher=0x1053075e0
I0503 16:54:21.876895000 4521967104 client_channel.cc:1273] chand=0x104873e60: update: state=CONNECTING picker=0x10536ecc0
I0503 16:54:21.878584000 4521967104 connectivity_state.cc:147] SET: 0x104873f30 client_channel: TRANSIENT_FAILURE --> CONNECTING [helper]
I0503 16:54:21.878629000 4521967104 resolving_lb_policy.cc:441] resolving_lb=0x10476dd70: got resolver result
I0503 16:54:21.878637000 4521967104 resolving_lb_policy.cc:356] resolving_lb=0x10476dd70: Updating child policy 0x10536e520
I0503 16:54:21.878642000 4521967104 pick_first.cc:265] Pick First 0x10536e520 received update with 1 addresses
I0503 16:54:21.878652000 4521967104 subchannel_list.h:370] [pick_first 0x10536e520] Creating subchannel list 0x1051717f0 for 1 subchannels
I0503 16:54:21.878680000 4521967104 client_channel.cc:831] chand=0x104873e60: creating subchannel wrapper 0x10516d0e0 for subchannel 0x10536f1e0
I0503 16:54:21.878689000 4521967104 subchannel_list.h:420] [pick_first 0x10536e520] subchannel list 0x1051717f0 index 0: Created subchannel 0x10516d0e0 for address uri ipv4:169.61.238.230:30052
I0503 16:54:21.879078000 4521967104 subchannel_list.h:442] [pick_first 0x10536e520] Shutting down subchannel_list 0x105317b70
I0503 16:54:21.879090000 4521967104 subchannel_list.h:337] [pick_first 0x10536e520] subchannel list 0x105317b70 index 0 of 1 (subchannel 0x10530bc80): canceling connectivity watch (shutdown)
I0503 16:54:21.879098000 4521967104 subchannel_list.h:293] [pick_first 0x10536e520] subchannel list 0x105317b70 index 0 of 1 (subchannel 0x10530bc80): unreffing subchannel
I0503 16:54:21.879103000 4521967104 client_channel.cc:850] chand=0x104873e60: destroying subchannel wrapper 0x10530bc80 for subchannel 0x10536f1e0
I0503 16:54:21.879108000 4521967104 subchannel_list.h:434] [pick_first 0x10536e520] Destroying subchannel_list 0x105317b70
I0503 16:54:21.879114000 4521967104 subchannel_list.h:316] [pick_first 0x10536e520] subchannel list 0x1051717f0 index 0 of 1 (subchannel 0x10516d0e0): starting watch (from CONNECTING)
Have a blessed day,
Chris Tyler
CTO | BlockSpaces, Inc.
214.673.0483
chris...@blockspaces.io
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee, you should not disseminate, distribute or copy this email. Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.
<PastedGraphic-3.png>
Michael Lumish
May 4, 2021, 2:23:22 PM5/4/21
to Chris Tyler, grpc.io
As I thought, it never establishes a connection. You can see the IDLE to CONNECTING transitions in a couple of places, but there's no corresponding CONNECTING to READY transition to show that it successfully connected. You're probably in a better position than I am to determine why that may happen, based on details of the network you're working with.
Chris Tyler
May 4, 2021, 2:26:54 PM5/4/21
to Michael Lumish, grpc.io
I appreciate your help. I finally went down the Ingress-Nginx controller route and it appears to be close to working now. i’m actually connecting and getting into my server. I made some good progress between 2am and 4am last night.
Have a blessed day,
Chris Tyler
CTO | BlockSpaces, Inc.
214.673.0483
chris...@blockspaces.io
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee, you should not disseminate, distribute or copy this email. Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.
<PastedGraphic-3.png>
Reply all
Reply to author
Forward
0 new messages