ACLs: Cannot connect to UI

2,105 views
Skip to first unread message

Roger Hill

unread,
May 23, 2016, 4:54:49 PM5/23/16
to Consul
I have a cluster of 3 Consul servers. I have enabled ACLs in the bootstrap server which also has the UI enabled. But when I try to connect to the https end point, I get an error. What could be the problem? 

Thanks for any insights.

My config.json:

{
    "bootstrap": true,
    "node_name": "host1",
    "server": true,
    "data_dir": "/tmp/consul",
    "encrypt": "blahblah",
    "log_level": "INFO",
    "ca_file": "/etc/consul.d/ssl/ca.cert",
    "cert_file": "/etc/consul.d/ssl/consul.cert",
    "key_file": "/etc/consul.d/ssl/consul.key",
    "verify_incoming": true,
    "verify_outgoing": true,
    "enable_syslog": true,
    "acl_token": "token123",
    "acl_datacenter": "dal",
    "acl_master_token": "master123",
    "ports": {
        "https": 8501
        }

}

When I point my browser to the HTTPport (8500), this is what I get in the browser:
HTTP error code from Consul: 403 Forbidden
This is an error page for the Consul web UI. You may have visited a URL that is loading an unknown resource, so you can try going back to the root.
Otherwise, please report any unexpected issues on the GitHub page.


When I point to HTTPS port, this is what I get: 
This site can’t provide a secure connection
10.204.42.21 didn’t accept your login certificate, or your login certificate may have expired.
Try contacting the system admin.
ERR_BAD_SSL_CLIENT_AUTH_CERT

In the latter case, it appears that Consul server is looking to do a 2-way SSL handshake. I verified that with openSSL - I can connect successfully to the server using s_client. 

The documentation on ACL appears to be very spotty and not comprehensive enough. Without ACLs, I think Consul is not secure enough for deployment in a production environment. Lack of good documentation with examples is a bit road block for us to adopt Consul. Does everyone feel this way? Any good pointers to ACL implementations?


Thanks!


Michael Fischer

unread,
May 24, 2016, 10:14:07 AM5/24/16
to consu...@googlegroups.com
Hi Roger,

You've enabled `verify_incoming` which causes Consul to require all incoming HTTPS connections to present a valid client certificate.  If you disable that, you should be able to connect with your browser to the HTTPS interface.

From the documentation, it appears that `verify_incoming` applies both to the server RPC and HTTP API endpoints.  Arguably, the configuration could be more fine-grained.  If you need different behavior for each, I would recommend filing an issue on GitHub.

Best regards,

--Michael

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/consul/issues
IRC: #consul on Freenode
---
You received this message because you are subscribed to the Google Groups "Consul" group.
To unsubscribe from this group and stop receiving emails from it, send an email to consul-tool...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/consul-tool/041d39cd-c41a-47ab-af2b-114c38db459d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Roger Hill

unread,
May 24, 2016, 12:19:56 PM5/24/16
to Consul
Michael, thanks for the info. When I disabled verify_incoming, it started to work. But when I accesses ACL tab in the UI, i was getting another problem. The browser response is similar to HTTP (not HTTPS) response as in the previous message. The server side I see this error message in the logs: 

http: Request GET /v1/internal/ui/nodes?dc=dal&token=<hidden>, error: ACL not found from=10.***.***.***:59504 (my browser client IP and port)

I also see a stream of messages for this error:

agent: failed to sync remote state: ACL not found

Any ideas?

Michael Fischer

unread,
May 24, 2016, 1:55:03 PM5/24/16
to consu...@googlegroups.com
Have you defined an `acl_token` in the Consul configuration?  If so, that token is probably incorrect (it does not point to an actual ACL). 

I generally don't recommend specifying `acl_token` as it will supply a token to anonymous (non-tokenized) requests from the HTTP API.  Better IMO to override the default (anonymous) ACL.

Best regards,

--Michael

To view this discussion on the web visit https://groups.google.com/d/msgid/consul-tool/7f2267fe-a265-4614-a051-a09bb6732147%40googlegroups.com.

Roger Hill

unread,
May 24, 2016, 5:53:11 PM5/24/16
to Consul
Yes Michael, I have defined acl_token in my config. I finally figured that I can access the UI by passing the acl_token value as a query string: 


But that still prevents me from accessing ACL tab on the UI. 

I do not understand what you mean by your last statement: 

"... it will supply a token to anonymous (non-tokenized) requests from the HTTP API.  Better IMO to override the default (anonymous) ACL"

Can you please elaborate?

Michael Fischer

unread,
May 24, 2016, 8:31:12 PM5/24/16
to consu...@googlegroups.com
On Tue, May 24, 2016 at 2:53 PM, Roger Hill <venkatr...@gmail.com> wrote:
Yes Michael, I have defined acl_token in my config. I finally figured that I can access the UI by passing the acl_token value as a query string: 


But that still prevents me from accessing ACL tab on the UI. 

My guess is that only the root token can be used to access the ACL tab.  Maybe someone else can assist with this problem.
 
I do not understand what you mean by your last statement: 

"... it will supply a token to anonymous (non-tokenized) requests from the HTTP API.  Better IMO to override the default (anonymous) ACL"

Can you please elaborate?

When you use the Consul API, you can supply an ACL token, or none at all.  

If you don't supply one, and `acl_token` is NOT defined in the agent configuration, then the built-in default/anonymous policy will be used to evaluate the request.

If you don't supply one, and `acl_token` IS defined in the agent configuration, then the value of `acl_token` in the agent configuration will be used as the token for the request.  However, Consul doesn't validate the value of `acl_token`, so if the token doesn't map to a valid ACL, you'll get the errors you saw earlier.

Does that answer your question?

Best regards,

--Michael
 

Francisco Javier Romero Mendiola

unread,
May 25, 2016, 5:52:58 AM5/25/16
to Consul
I think this is a problem in terms concepts with parameters. I also had the same problem and I will try to explain token and ACL parameters.

In order to enable ACLs you must define next parameters: "acl_datacenter" and "acl_mater_token". First one needs a valid name that usually match with "datacenter" parameter. Second one is a master token (you can generate it with uuidgen) that You use to create new ACLs, e.g. Commonly you want to deny "acl_default_policy" for security reasons but you need a token to operate and create news ACLs for your services, this is the purpose of "acl_master_token".

On the other hand, you do not need define "acl_token" parameter in your Consul servers, It should be created previously with valids rules and setting up in services.

I hope this solves your question.

Francisco.

Roger Hill

unread,
May 25, 2016, 11:50:16 AM5/25/16
to Consul
Now I see what you mean by default acl token. Thanks for the explanation Michael.

Now I have made some changes to the file config.json: (removed the entry for acl_token)

{
    "bootstrap": true,
    "node_name": "host",
    "server": true,
    "data_dir": "/tmp/consul",
    "encrypt": "blahblah",
    "log_level": "INFO",
    "ca_file": "/etc/consul.d/ssl/ca.cert",
    "cert_file": "/etc/consul.d/ssl/consul.cert",
    "key_file": "/etc/consul.d/ssl/consul.key",
    "verify_outgoing": true,
    "enable_syslog": true,
    "acl_datacenter": "abc",
    "acl_master_token": "master_token",
    "acl_default_policy": "deny",
    "ports": {
        "https": 8501
        }
}

Now I have created a file acl.json with the following contents:

{
  "key": {
    "": {
      "policy": "write"
    }
  }
}

And applied the acl:


Got an ACL ID (GUID) as a result of the above action.

I used that ACL ID to set the access token in the web UI. Now when I create a key value pair in the UI, I get the following error:  Received error while processing: Forbidden

The server log file shows the following corresponding message:  [ERR] http: Request PUT /v1/kv/key1?dc=abc&token=<hidden>, error: Permission denied from=10.***.***.**:17824

What am I missing? Any thoughts?




Roger Hill

unread,
May 25, 2016, 11:54:43 AM5/25/16
to Consul
Francisco, thank you for the explanation. It is certainly very helpful.

Francisco Javier Romero Mendiola

unread,
May 25, 2016, 12:17:26 PM5/25/16
to Consul
I used that ACL ID to set the access token in the wuo. Now when I create a key value pair in the UI, I get the following error:  Received error while processing: Forbidden

Your ACL has not enough permissions to create new ACLs, you need append "type=admin/master" ( I do not remember the word exactly) and other things surely.

You try to insert "master token" in web UI and you will get full access to ACLs.

Francisco.

Michiel Vanderlee

unread,
Jul 14, 2016, 5:27:58 PM7/14/16
to Consul
Came here because I had the same symptoms but different cause. Posting here simply in case others come here with same issue.

My issue turned out to be a weird cookie issue because my browser was overriding any provided token with an old one.
Clearing my cookies fixed this.
Reply all
Reply to author
Forward
0 new messages