Heads up on next release (tentatively 2.2.3.0) and on deprecated methods being removed

[Kevin W. Wall]

I am intending on doing a minor patch release of ESAPI in the next few days that for the most part consists of updating the version of AntiSamy to 1.6.0 (or to 1.6.1, if it available) and updating batik-css from 1.13 to 1.14 to address a few CVEs (that I am not certain affects ESAPI, but is hard to tell as some of the use was via AntiSamy), along with some minor bug fixes in the ESAPI Validator.

In addition, I am doing some minor code clean-up. In particular, much of the clean-up has to do ESAPI GitHub issue #593 but really goes beyond it.

Not only was there things like SecurityConfiguration.getLogLevel() that really never did anything useful (that was really always controlled by the logger configuration file of whatever implementation you were using), but there were also additional methods in that interface such as getLogFileName() and getMaxLogFileSize() that really made no sense since there are specific to the created Logger itself and the underlying logger implementation configuration file. In addition to those latter 2 methods, I am also removing the Logger.LogFileName and Logger.MaxLogFileSize that never really did anything, or at least haven't for a decade or more. (And there never was a Logger.LogLevel or similar ESAPI property to associate with getLogLevel().)

Most of these methods have been deprecated well over 2 years and they never really did anything in the first place (or, at least certainly not what you think they should have done), so if my removal ends up "breaking your code" by causing it to fail to compile, I apologize, but there is really no point leaving these methods in ESAPI so you can just remove them from your application code if you were calling them and start getting compilation errors. (Or don't update to the latest version of ESAPI until you do.)

If you have questions or concerns, please drop me an email ASAP.

-kevin

--

Blog: https://off-the-wall-security.blogspot.com/    | Twitter: @KevinWWall | OWASP ESAPI Project co-lead
NSA: All your crypto bit are belong to us.

[Kevin W. Wall]

For the stuff I referenced in the previous email (about deprecated methods and properties that are being deleted), please find details in the merged PR at https://github.com/ESAPI/esapi-java-legacy/pull/603.

In the interest of transparency (but almost against my better judgement), I am also sharing this full disclosure. There is going to be a slight delay on releasing ESAPI 2.2.3.0. The reasons are explained below. If someone is willing to step up and help, please drop me an email.

First, there was a need for the AntiSamy team to create a new 1.6.1 patch release. Kudos to Dave Wichers for jumping on this so quickly and working with us. That should be published to Maven Central tonight.

But secondly, as of AntiSamy 1.6.0 and later, AntiSamy is now doing XML schema validation of the AntiSamy policy files against its internal XSD. AntiSamy has this XML schema validation enabled by default, which is a good thing. Unfortunately, all of the previous ESAPI 2.x releases where AntiSamy was used had an extraneous '<html-entities>' node that needs to be removed to conform to AntiSamy's XSD. I can and am doing this for the new forthcoming ESAPI release, but there is nothing I can do to force those upgrading to this next ESAPI version to also update their antisamy-esapi.xml files (in part, because they may have been customized). Fortunately, because of that, AntiSamy is supporting a way to disable the schema validation to support backward compatibility mostly for ESAPI's sake. (See https://github.com/nahsra/antisamy#note-schema-validation-behavior-change-starting-with-antisamy-160 if you are interested.)

Unfortunately, as I attempt to test this disabling of the AntiSamy schema validation, it is causing one of our JUnit tests to pass which I expected to fail (when run as 'mvn test') and to fail when it was expected to pass (when executed as only a single JUnit test). Through it all, we found out that the default parallelism of maven-surefire-plugin reusing JVMs for multiple tests was actually masking a lot of test failures. (I counted 196 of them dating WAY back, like 10 to 12 or so years, dating back to the earliest 2.0 release candidates.) Obviously, we are not going to try to fix all of those for this next release. Many of failing tests have to do with ESAPI's reference implementation classes using singleton patterns and static initializers and then the JUnit tests trying to work around those imposed limitations by changing up values for various ESAPI properties via ESAPI.override() and other similar ill-advised mischief. (Don't try that at home kids!)

So a simple AntiSamy change [which is in fact, an improvement] caused a hidden failure to finally manifest itself. (That is, it was previously there, but unnoticed until now). This is unlikely to cause problems in real life because people are unlikely to be trying to use multiple AntiSamy policy files with ESAPI at once for any given application. But we do anticipate some people not carefully reading the release notes documentation for ESAPI (No! Say it ain't true!) and not updating your antisamy-esapi.xml policy file to remove the invalid '<html-entities>' node (which never was valid, in fact!) and getting a mysterious error message. Or no error message at all because ESAPI swallows it. We need to test to make sure that ESAPI is not hiding the AntiSamy exception messages or messages that it intends to log. So that is what is holding ESAPI 2.2.3.0 up. Lastly, I've been working on this for maybe 30 hours in the past 3 days and am getting really burned out and tomorrow is going to be a really long day at work because I took PTO Friday. Because, yes, I have a day job too.

-kevin

--

Blog: https://off-the-wall-security.blogspot.com/    | Twitter: @KevinWWall | OWASP ESAPI Project co-lead
NSA: All your crypto bit are belong to us.

[Kevin W. Wall]

We ran into a AntiSamy bug that didn't allow ESAPI users the opportunity to use old ESAPI AntiSamy policy file that would no longer pass AntiSamy's XML schema validation and as a result, I have decided to delay the 2.2.3.0 release until this can be addressed. I do not know how long that will be; hopefully soon. But without extensive code changes to how ESAPI is using AntiSamy in its HTMLValidationRule class to make it work with invalid, but otherwise well-formed AntiSamy policy files, this would force everyone to update their possibly customized policy files before deploying...a less than ideal situation to me. (All the previous delivered antisamy-esapi.xml policy files had an extraneous '<html-entities>' node that would need to be removed.)

If you want to follow along at home, the 2 relevant AntiSamy links are:

https://github.com/nahsra/antisamy/issues/75

We ran into a AntiSamy bug that didn't allow ESAPI users the opportunity to use old ESAPI AntiSamy policy file that would no longer pass AntiSamy's XML schema validation and as a result, I have decided to delay the 2.2.3.0 release until this can be addressed. I do not know how long that will be; hopefully soon. But without extensive code changes to how ESAPI is using AntiSamy in its HTMLValidationRule class to make it work with invalid, but otherwise well-formed AntiSamy policy files, this would force everyone to update their possibly customized policy files before deploying...a less than ideal situation to me. (All the previous delivered antisamy-esapi.xml policy files had an extraneous '<html-entities>' node that would need to be removed.) If you want to follow along at home, the 2 relevant AntiSamy links are: https://github.com/nahsra/antisamy/issues/75 https://github.com/nahsra/antisamy/pull/77 -kevin

[Kevin W. Wall]

Status update. I just finished initial draft of a new security advisory (for log4j 1.2.17 of course, CVE-2020-9488; you are only affected if you are using log4j 1.x's SMTPAppender), but I'm also still making updates to the 2.2.3.0 release notes because of impact of the AntiSamy changes in the new version we are using (1.6.2). I also need to make some minor updates to the README.md file.

Pretty much everything that will be in the new 2.2.3.0 release--aside from documentation--should now be there in our 'develop' branch, if any of you adventurous souls want to kick the tires.

None of the new documentation is there yet though.

But the impact of updating to AntiSamy 1.6.2 is huge because by default it enables XML schema checking and all the AntiSamy policy files ESAPI sent out for the past 10 years have an '<html-entities>' node that is invalid and needs either removed or you need to disable AntiSamy's XML schema checking in the meantime. Also AntiSamy now includes 'slf4j-simple' as a compile-time dependency so if you are using something else, like (say) 'slf4j-logback', you are going to need to exclude 'slf4j-simple' from your ESAPI dependency. So all that stuff needs documented.

-kevin