OutOfMemory when recording a huge number of requests

[Dalton Fury]

I want to use WireMock to record and then playback around 20,000 requests. What would be the memory requirements for this?

I setup WireMock in a container with 2GB allotted memory, and it ran out of heap space while recording the traffic. (I was sending requests in a loop, with random 500ms to 3s delays).

Then I increased the memory to 6GB, and it was able to record everything, but when I stopped the WireMock server, while saving the stubs, it ran out of memory. I wanted to know if it's common to use  WireMock recording and playback to record such amounts of stubs. We want to run some tests that involve a lot of permutations of requests.

1000 stubs I had recorded before took only 30MB of disk space, so 20,000 would have taken roughly 600MB. 

I believe I can't use --no-request-journal as my objective is to record stubs. Can I somehow trigger a save after each stub recording? Can I somehow hook into this?

Thanks a lot!

[Dalton Fury]

I did some analysis. I created a small python script to call WireMock in a loop:

import requests

  

for i in range(5100):

        requests.get('http://localhost:9500/response.json')

And I have repeatAsScenarios set to true, so the same request will get recorded 5000 times.

When the application starts, it takes 420MB (spring application, so this is normal). Then I send the requests, and it takes around 800MB. This is also fine. But when I stop recording, suddenly it starts taking 4.89GB, which is strange.

I took heap dump before and after stopping recording, and then used jhat to see what's happening, and this is what I am seeing:

Heap Histogram shows a huge amount number of [B objects: (2GB)

So I ran WireMock as a standalone jar, it started with 89MB, ran my 5000 recordings, and it took 700MB. Then when I click on stop from the WireMock UI, I see it taking again 4.2GB. So the issue is reproducible on the standalone jar v2.27.0

Thanks,

[Tom Akehurst]

Someone else has posted an issue about this too, so I've posted a bit of explanation and a possible workaround here: https://github.com/tomakehurst/wiremock/issues/1353#issuecomment-664346650