Browser tests run very slow with a lot of sources/resources

[Garrick Cheung]

Hey guys,

When developing, running tests quickly is important. No one wants to wait too long for their tests to complete. Good thing tests in node run super fast. Browser tests, on the other hand can get quite slow, so I need some help in this area.

I recently filed https://github.com/busterjs/buster/issues/399 in relation to what could be slowing down browser tests. Just trying to cover all bases.

It's taking approx. 18 - 20 seconds before tests start and 20 - 23 seconds for tests to complete on one browser running in my local environment. Two browsers takes approx. 18 - 20 seconds before tests start and 24 - 28 seconds to complete. There are only 14 test files with 74 tests and 165 assertions. It's not because the browser tab/window is not focused either. 

Suspicion #1: Its the large number of files in the source option in a buster config group that is being preloaded.

For example, the sources option would just be ['scripts/*.js'] and the scripts folder has A LOT of files. Like 12720 files, which could include js, css, images, markdown, and what not. Just imagine including the node_modules folder that has 10 or 15 modules and their dependencies. But the glob is only including *.js, which should shrink the number of files, but it's still a lot considering it's mainly JS in the scripts folder.

Suspicion #2: It's the communication between ramp and the other browsers.

Suspicion #3: Combination of both ramp communication and reading of the large number of files.

Is there any option I may have missed when running tests? I've tried flags like release-console, static-paths, and full-stacks but the results are the same.

I've tried switching from sources to resources and the behavior seems to be the same regardless of option.

I've played with buster-static and that actually seems to be much faster. Correct me, but all the libs/sources/resources are loaded when requested instead of being preloaded right? Maybe the sources and resources don't need to be preloaded for buster-test?

-Garrick-

[Christian Johansen]

> *Suspicion #1*: Its the large number of files in the source option in a

> buster config group that is being preloaded.
> For example, the sources option would just be ['scripts/*.js'] and the
> scripts folder has A LOT of files. Like 12720 files, which could include
> js, css, images, markdown, and what not. Just imagine including the
> node_modules folder that has 10 or 15 modules and their dependencies. But
> the glob is only including *.js, which should shrink the number of files,
> but it's still a lot considering it's mainly JS in the scripts folder.

In my experience, the glob resolution in
buster-configuration/ramp-resources has been unnecessarily slow. I would
guess that some of at least lies here.

>
> *Suspicion #2*: It's the communication between ramp and the other
> browsers.

I'm not convinced this is a problem.

>
> *Suspicion #3*: Combination of both ramp communication and reading of the

> large number of files.
>
> Is there any option I may have missed when running tests? I've tried flags
> like release-console, static-paths, and full-stacks but the results are the
> same.
>
> I've tried switching from sources to resources and the behavior seems to be
> the same regardless of option.
>
> I've played with buster-static and that actually seems to be much faster.
> Correct me, but all the libs/sources/resources are loaded when requested
> instead of being preloaded right? Maybe the sources and resources don't
> need to be preloaded for buster-test?

The files will all be read from disk and shipped to the server on the
first run. From there on, they should be cached (so long as you keep the
server running), and only changed files should be read again. However,
like I mentioned, I have experienced that the globbing (which always
need to be done, in case new files etc), occasionally take too long.

I'm sorry I don't have much concrete info for you at this point. But
after having been stuck with Karma/Jasmine at work for a short while, I
can assure you that I will be continuing work on Buster in the near
future. Almost all my spare time has been allocated to recording an
album the past two months, and we're soon done with that.

Christian

>
> -Garrick-