Thread Analytics -> Snapshot is running very slow on ThreadFix v2.3

[henr...@gmail.com]

Hi ThreadFix team,

We see that Analytics --> Snapshot feature is running very slow. It takes around 5 minutes for loading the data. In some case, Tomcat server is die. We have over 100 applications and every application has around 200 to 500 vulnerabilities. 

We find out a method we think it is running too slow. Could you help to check? 

Besides, 'Delete Scan' feature under an Application also have issue with performance, could you help to double-check?

We are using ThreadFix v2.3.

ReportsController.java

    ..................

    ..................

    @RequestMapping(value="/snapshot", method = RequestMethod.POST)

@JsonView(AllViews.VulnSearchApplications.class)

    public @ResponseBody RestResponse<Map<String, Object>> processSnapShot(@ModelAttribute ReportParameters reportParameters,

                                                                           HttpServletRequest request) throws IOException {

        log.info("Generating snapshot report");

        Map<String, Object> map = reportsService.generateSnapshotReport(reportParameters,

                request);

        map.put("tags", tagService.loadAllApplicationTags());

map.put("vulnTags", tagService.loadAllVulnTags());

        return RestResponse.success(map);

    }

    ..................

    ..................

Thank you,

Henry.

[Daniel Maldonado]

Henry,

Do you know how much memory you have allocated to Tomcat?

Thanks,

Daniel Maldonado

[henr...@gmail.com]

Hi Daniel,

Here you are our Tomcat's configuration. Do we need to make any changes to improve performance?

Java Home:             C:\Program Files\Java\jre1.8.0_72

JVM Version:           1.8.0_72-b15

JVM Vendor:            Oracle Corporation

CATALINA_BASE:         D:\Tomcat_8_0

CATALINA_HOME:         D:\Tomcat_8_0

Command line argument: -XX:PermSize=512m

Command line argument: -XX:MaxPermSize=512m

Command line argument: -Dcatalina.home=D:\Tomcat_8_0

Command line argument: -Dcatalina.base=D:\Tomcat_8_0

Command line argument: -Djava.endorsed.dirs=D:\Tomcat_8_0\endorsed

Command line argument: -Djava.io.tmpdir=D:\Tomcat_8_0\temp

Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager

Command line argument: -Djava.util.logging.config.file=D:\Tomcat_8_0\conf\logging.properties

Command line argument: -Xms128m

Command line argument: -Xmx256m

Thank you,

Henry.

[Daniel Maldonado]

Henry,

Yes, I suggest increasing your MaxPermSize to a minimum of 8GB. Here is some more information on system requirements.

Daniel Maldonado

[henr...@gmail.com]

Hi Daniel,

We had try to increased MaxPermSize to a minimum of 8G but we have the same problem. We also double-checked our server that has 24G RAM and we see that it only use around 3G RAM when running the feature.

We had download ThreadFix's source code v2.3 from GIT and try again. We see that these two methods below that a lot of time to retrieve data before generating in GUI.

map.put("tags", tagService.loadAllApplicationTags());

map.put("vulnTags", tagService.loadAllVulnTags()); 

We don't think the root cause from server's configuration. Could you help to unit-test with huge data? 

Server's log:

...............

...............

04-Feb-2016 13:29:49.325 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:PermSize=512m

04-Feb-2016 13:29:49.325 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:MaxPermSize=8192m

...............

...............

Thank you,

Henry.

[Daniel Maldonado]

Henry,

Thank you for the update and the information. Let me look into the stress testing that we have done previously and I will get back to you.

Daniel Maldonado