Consul redis check continue printing "Check service:redis-6379-redis1 is warning now", while it's OK

1,038 views
Skip to first unread message

Дмитрий Ансимов

unread,
Dec 17, 2016, 4:44:52 AM12/17/16
to Consul
I've set up a cluster of 4 nodes, two of them are servers with redis instances on them.

consul.json:

{
 
"bootstrap_expect": 2,
 
"client_addr": "0.0.0.0",
 
"data_dir": "/var/lib/consul",
 
"enable_syslog": true,
 
"ports": {
   
"dns": 8600,
   
"http": 8500,
   
"rpc": 8400,
   
"serf_lan": 8301,
   
"serf_wan": 8302,
   
"server": 8300
 
},
 
"server": true,
 
"start_join": [
   
"10.0.1.2"
 
],
 
"verify_incoming": false,
 
"verify_outgoing": false
}

 conf.d/redis_service.json:


  "services": [


   
{
     
"name": "redis-6379-redis1",
     
"tags": ["redis"],
     
"address": "10.0.1.2",
     
"port": 6379,
     
"enableTagOverride": true,
     
"checks": [
       
{
         
"script": "/usr/local/script/check_redis_6379.sh",
         
"interval": "10s"
       
}
     
]
   
}
 
]


Redis check is trivial and was taken here. If I run it in the shell, $? is always 0.

While in consul agent logs I always see:

consul: 2016/12/17 11:41:10 [WARN] agent: Check 'service:redis-6379-redis1' is now warning

After I change redis port in the script to unexisting, $? eventually became 2, but messages in consul log are still the same.

What's wrong with my setup? Thanks in advance, comrades.

Matt Hite

unread,
Dec 18, 2016, 10:58:27 AM12/18/16
to consu...@googlegroups.com
Let me preface this with the following: I know nothing about redis.

Are you using the redis health check you linked exactly as-is?

#!/usr/bin/with-contenv sh

RESULT=`redis-cli ping`

if [ "$RESULT" = "PONG" ]; then
    exit 0
fi

exit 2

I suspect the user Consul runs as doesn't have redis-cli in its path, or does not have permissions to run said executable. Honestly, I'd clean that health check script up a bit, change the #! line to the actual path to bash, and hardcode the redis-cli path.

Good luck,

-M


--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/consul/issues
IRC: #consul on Freenode
---
You received this message because you are subscribed to the Google Groups "Consul" group.
To unsubscribe from this group and stop receiving emails from it, send an email to consul-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/consul-tool/ea5bc653-9de8-432a-bc8d-bc7059f8cb18%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Дмитрий Ансимов

unread,
Dec 18, 2016, 12:32:24 PM12/18/16
to consu...@googlegroups.com


18 дек. 2016 г. 5:58 PM пользователь "Matt Hite" <li...@beatmixed.com> написал:

Let me preface this with the following: I know nothing about redis.

Are you using the redis health check you linked exactly as-is?

#!/usr/bin/with-contenv sh

RESULT=`redis-cli ping`

if [ "$RESULT" = "PONG" ]; then
    exit 0
fi

exit 2

I suspect the user Consul runs as doesn't have redis-cli in its path, or does not have permissions to run said executable. Honestly, I'd clean that health check script up a bit, change the #! line to the actual path to bash, and hardcode the redis-cli path.

Hi Matt, 

Thanks for your suggestion, but no, I'm using it with changed #! and full path to redis-cli executable. No use :( I'm gonna check everything once more anyway. 
You received this message because you are subscribed to a topic in the Google Groups "Consul" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/consul-tool/3t1lrHY990g/unsubscribe.
To unsubscribe from this group and all its topics, send an email to consul-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/consul-tool/CAFOdRH%2BXsyobGZg00ekWVkDZiGurSvHOSKM1mgnytL3_e%2BfGcA%40mail.gmail.com.

James Phillips

unread,
Dec 19, 2016, 11:25:19 PM12/19/16
to consu...@googlegroups.com
Hi,

If you take a look in "Output" field of
https://www.consul.io/docs/agent/http/health.html#health_service for
the "redis" service does that give any insight? That should have the
output of the script, as seen from the Consul agent.

-- James
>> email to consul-tool...@googlegroups.com.
>>
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/consul-tool/ea5bc653-9de8-432a-bc8d-bc7059f8cb18%40googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> This mailing list is governed under the HashiCorp Community Guidelines -
> https://www.hashicorp.com/community-guidelines.html. Behavior in violation
> of those guidelines may result in your removal from this mailing list.
>
> GitHub Issues: https://github.com/hashicorp/consul/issues
> IRC: #consul on Freenode
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Consul" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/consul-tool/3t1lrHY990g/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> consul-tool...@googlegroups.com.
> --
> This mailing list is governed under the HashiCorp Community Guidelines -
> https://www.hashicorp.com/community-guidelines.html. Behavior in violation
> of those guidelines may result in your removal from this mailing list.
>
> GitHub Issues: https://github.com/hashicorp/consul/issues
> IRC: #consul on Freenode
> ---
> You received this message because you are subscribed to the Google Groups
> "Consul" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to consul-tool...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/consul-tool/CAK-RTX2wvtBU%2BnaEMe-Gr-do0tDkZSjLXdEcNPLL-50Obk1QBQ%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages