Oauth Token API Cannot get Jedis connection

[Andy Zhang]

Hi Team, 

I am working on the examples on https://github.com/SeldonIO/seldon-core/blob/master/notebooks/kubectl_demo_gcp.ipynb and I am not using Google Compute Engine to set up a k8s cluster myself.  so I do port-forward using the following command:

kubectl port-forward -n seldon $(kubectl get pod --selector=app=seldon-apiserver-container-app -o jsonpath='{.items..metadata.name}' -n seldon) 8080:8080

kubectl port-forward -n seldon $(kubectl get pod --selector=app=seldon-apiserver-container-app -o jsonpath='{.items..metadata.name}' -n seldon) 5000:5000

I modified the SELDON_API_IP as "127.0.0.1" instead of the external IP of a load balancer.

def get_token():
    payload = {'grant_type': 'client_credentials'}
    response = requests.post(
                "http://{}:8080/oauth/token".format(SELDON_API_IP),
                auth=HTTPBasicAuth('oauth-key', 'oauth-secret'),
                data=payload)
    token =  response.json()["access_token"]
    return token

I am getting 500 error when call get_token. further investigation shows the following error details. 

{

    "error": "server_error",

    "error_description": "Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool"

}

Can you please help with the issue? Thanks.

[Gurminder Sunner]

Hi,

We will try to help with this issue.

Have you tried re-starting seldon-core, and do you still get the same error?

Would be useful to know more details about the kubernetes cluster that you setup yourself, if there is something in the setup that is causing this.

Regards,

Gurminder