http.HttpParser - Header is too large 8193>8192

[danifr]

Hello,

I configured Rundeck to use preauth mode with OpenID Connect.

The OIDC token contains all the groups the user is member of and for some users (users who belong to too many groups), I am getting the following error:

[2021-02-09T17:05:53,753] WARN  http.HttpParser - Header is too large 8193>8192

It seems I need to increase the grails headers size, but the solution described here does not work with my .groovy file.

https://github.com/rundeck/rundeck/issues/3832

I tried:

• server.max-http-header-size=16KB
• server.max-http-header-size=16384
• server.'max-http-header-size'=16384
• "server.max-http-header-size"=16384

But none of them seemed to work.

Any idea of what I need to do?

Thanks a lot,

Daniel.

[rac...@rundeck.com]

Hi Dani,

Could you try in the following way:

server."max-http-header-size"="16KB"

And make sure that you’re referencing the .groovy file at launch.

Regards.

​

[danifr]

Hello,

thanks a lot for your answer. Unfortunately it didn't seem to work. I am still getting "Header is too large 8193>8192"

I paste you part of my configuration and the rundeck logs in case I am missing something.

By the way, I did not mentioned it but I am using rundeck v3.3.6.

[root@rundeck ~]# grep header /etc/rundeck/rundeck-config.groovy

server."max-http-header-size"="16KB"

[root@rundeck ~]# grep groovy /etc/sysconfig/rundeckd

RDECK_CONFIG_FILE=/etc/rundeck/rundeck-config.groovy

[root@rundeck ~]# systemctl restart rundeckd; tail -F /var/log/rundeck/service.log

Session terminated, killing shell...[2021-02-09T20:23:22,097] INFO  rundeckapp.BootStrap - Rundeck Shutdown detected

 ...killed.

[2021-02-09T20:23:32,347] INFO  rundeckapp.Application - The following profiles are active: production
Configuring Spring Security Core ...
... finished configuring Spring Security Core

[2021-02-09T20:23:59,636] WARN  beans.GenericTypeAwarePropertyDescriptor - Invalid JavaBean property 'exceptionMappings' being accessed! Ambiguous write methods found next to actually used [public void grails.plugin.springsecurity.web.authentication.AjaxAwareAuthenticationFailureHandler.setExceptionMappings(java.util.List)]: [public void org.springframework.security.web.authentication.ExceptionMappingAuthenticationFailureHandler.setExceptionMappings(java.util.Map)]
[2021-02-09T20:24:10,190] INFO  rundeckapp.BootStrap - Starting Rundeck 3.3.6-20201111 (2020-11-11) ...
[2021-02-09T20:24:10,192] INFO  rundeckapp.BootStrap - using rdeck.base config property: /var/lib/rundeck
[2021-02-09T20:24:10,213] INFO  rundeckapp.BootStrap - loaded configuration: /etc/rundeck/framework.properties
[2021-02-09T20:24:10,277] INFO  rundeckapp.BootStrap - RSS feeds disabled
[2021-02-09T20:24:10,277] INFO  rundeckapp.BootStrap - Using jaas authentication
[2021-02-09T20:24:10,287] INFO  rundeckapp.BootStrap - Preauthentication is enabled
[2021-02-09T20:24:10,410] INFO  rundeckapp.BootStrap - Rundeck is ACTIVE: executions can be run.
[2021-02-09T20:24:11,292] INFO  rundeckapp.BootStrap - Rundeck startup finished in 1226ms
[2021-02-09T20:24:11,549] INFO  rundeckapp.Application - Started Application in 43.08 seconds (JVM running for 47.25)
Grails application running at http://localhost:4440/project/devworkflows in environment: production

[2021-02-09T20:24:19,729] WARN  http.HttpParser - Header is too large 8193>8192

I can try to increase the verbosity of the rundeck logs to verify if server."max-http-header-size"="16KB" is not being ignored.

Thank you very much,

Daniel.

[rac...@rundeck.com]

Hi Dani,

Also, you can set the parameter on rundeck-config.properties and use the rundeck-config.groovy as a complement, similar to groovy Gmail configuration (when the .groovy file only has the specific email settings).

Regards.

​

[danifr]

I tried what you suggested but it didn't work.

Even after creating a new rundeck.properties file with this content :

[root@rundeck ~]# cat /etc/rundeck/rundeck-config.properties
server.max-http-header-size=16384
[root@rundeck ~]#

I am still getting:

[2021-02-11T11:03:09,838] WARN  http.HttpParser - Header is too large 8193>8192

To configure rundeck I am using https://github.com/voxpupuli/puppet-rundeck which uses the .groovy file (this is why I need to stick to the .groovy file)

Thank you very much for all your help,

Daniel.

[danifr]

So the main issue here is that I cannot change the default value of server.max-http-header-size property, because the name contains hyphens and groovy interprets it as a binary expression.

    at org.springframework.boot.loader.Launcher.launch(Launcher.java:51) [rundeck-3.3.6-20201111.war:?]
    at org.springframework.boot.loader.WarLauncher.main(WarLauncher.java:58) [rundeck-3.3.6-20201111.war:?]
Caused by: org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
script16131430665521182758583.groovy: 45:
(((max - http) - header) - size) is a binary expression, but it should be a variable expression at line: 45 column: 24. File: script16131430665521182758583.groovy @ line 45, column 24.
     max-http-header-size = "16000"
                          ^

Any idea how to work around this? If I add quotes, jetty/spring-boot just ignores it.

It would be nice if Rundeck would just merge the contents of the rundeck-config.properties and rundeck-config.groovy files.

Thanks a lot,

Daniel.

[rac...@rundeck.com]

Hey Dani,

Testing on my environment I see the same behavior but it works using server."max-http-header-size"=16384, also works as I posted before, using server."max-http-header-size"="16KB" (on the .groovy file of course).

Could you test your instance without the puppet deployment to discard any external issues?

Regards.

​

[danifr]

That's probably because jetty is ignoring them. This is why you don't see any change.

I bet if you changed the value to something silly like: server."made-on-the-spot"="bbb" jetty would not complain either.

Anyway, I just found I workaround for my specific use case (I documented here in case someone else can benefit from it in the future):

Edit /etc/sysconfig/rundeckd and add:

RDECK_JVM_SETTINGS="$RDECK_JVM_SETTINGS -Dserver.max-http-header-size="16000"

After restarting rundeck I was finally able to login with that user that belongs to too many groups \o/

Many many thanks for your help, you pointed me into the right direction!!

Daniel.

[rac...@rundeck.com]

Awesome Dani! cheers!