Test Lab Robo Test – WebView CSP crash

[Sampo Karjalainen]

While testing a React Native app using Test Lab Robo Test, I recently started to see crashes in a WebView that has a webpage open with a strict Content Security Policy. I haven't yet been able to produce these CSP errors in a stand-alone Chrome browser with the same webpage, or manually with the app. The error and the stack trace hint that Robo Test might be trying to control the WebView with injected JS, but fails without 'unsafe-eval' CSP directive.

Is this a known issue with Robo Test, or what could be the issue?

Error with stack trace:

FATAL EXCEPTION: Thread-5

Process: [REDACTED], PID: 8006

java.lang.RuntimeException: java.lang.RuntimeException: Error in evaluationEvaluation: status: 13 value: {message=Refused to evaluate a string as JavaScript because 'unsafe-eval' is not an allowed source of script in the following Content Security Policy directive: "script-src 'self'".

} hasMessage: true message: Refused to evaluate a string as JavaScript because 'unsafe-eval' is not an allowed source of script in the following Content Security Policy directive: "script-src 'self'".

at androidx.test.espresso.web.sugar.Web$WebInteraction$ExceptionPropagator.<init>(Web.java:2)

at androidx.test.espresso.web.sugar.Web$WebInteraction.doEval(Web.java:10)

at androidx.test.espresso.web.sugar.Web$WebInteraction.perform(Web.java:1)

at androidx.test.tools.crawler.platform.hybrid.HybridInteractionController.swipe(HybridInteractionController.java:15)

at androidx.test.tools.crawler.platform.ActionExecutor.execute(ActionExecutor.java:33)

at androidx.test.tools.crawler.platform.ActionExecutor.performAction(ActionExecutor.java:4)

at androidx.test.tools.crawler.platform.RemotePlatform.handlePerformAction(RemotePlatform.java:22)

at androidx.test.tools.crawler.platform.RemotePlatform.messageLoop(RemotePlatform.java:26)

at androidx.test.tools.crawler.platform.RemotePlatform.lambda$startCrawlAndWaitUntilFinished$0$RemotePlatform(RemotePlatform.java:1)

at androidx.test.tools.crawler.platform.RemotePlatform$$Lambda$0.run(Unknown Source:6)

at java.lang.Thread.run(Thread.java:764)

Caused by: java.lang.RuntimeException: Error in evaluationEvaluation: status: 13 value: {message=Refused to evaluate a string as JavaScript because 'unsafe-eval' is not an allowed source of script in the following Content Security Policy directive: "script-src 'self'".

} hasMessage: true message: Refused to evaluate a string as JavaScript because 'unsafe-eval' is not an allowed source of script in the following Content Security Policy directive: "script-src 'self'".

at androidx.test.espresso.web.model.SimpleAtom.handleBadEvaluation(SimpleAtom.java:1)

at androidx.test.espresso.web.model.SimpleAtom.transform(SimpleAtom.java:2)

at androidx.test.espresso.web.model.SimpleAtom.transform(SimpleAtom.java:3)

at androidx.test.espresso.web.action.AtomAction$3.apply(AtomAction.java:1)

at androidx.test.espresso.web.action.AtomAction$3.apply(AtomAction.java:2)

at androidx.test.tools.crawler.obfuscated.bf.d.doTransform(AbstractTransformFuture.java:1)

at androidx.test.tools.crawler.obfuscated.bf.d.doTransform(AbstractTransformFuture.java:2)

at androidx.test.tools.crawler.obfuscated.bf.e.run(AbstractTransformFuture.java:9)

at androidx.test.tools.crawler.obfuscated.bf.g.execute(DirectExecutor.java:1)

at androidx.test.tools.crawler.obfuscated.bf.a.executeListener(AbstractFuture.java:1)

at androidx.test.tools.crawler.obfuscated.bf.a.complete(AbstractFuture.java:10)

at androidx.test.tools.crawler.obfuscated.bf.a.set(AbstractFuture.java:2)

at androidx.test.tools.crawler.obfuscated.bf.aa.set(SettableFuture.java:1)

at androidx.test.espresso.web.action.AtomAction$1.setResult(AtomAction.java:1)

at androidx.test.espresso.web.action.AtomAction$2.run(AtomAction.java:1)

at androidx.test.tools.crawler.obfuscated.bf.g.execute(DirectExecutor.java:1)

at androidx.test.tools.crawler.obfuscated.bf.a.executeListener(AbstractFuture.java:1)

at androidx.test.tools.crawler.obfuscated.bf.a.complete(AbstractFuture.java:10)

at androidx.test.tools.crawler.obfuscated.bf.a.set(AbstractFuture.java:2)

at androidx.test.tools.crawler.obfuscated.bf.d.setResult(AbstractTransformFuture.java:1)

at androidx.test.tools.crawler.obfuscated.bf.e.run(AbstractTransformFuture.java:12)

at androidx.test.tools.crawler.obfuscated.bf.g.execute(DirectExecutor.java:1)

at androidx.test.tools.crawler.obfuscated.bf.a.executeListener(AbstractFuture.java:1)

at androidx.test.tools.crawler.obfuscated.bf.a.complete(AbstractFuture.java:10)

at androidx.test.tools.crawler.obfuscated.bf.a.set(AbstractFuture.java:2)

at androidx.test.espresso.web.action.JavascriptEvaluation$ValueCallbackFuture.onReceiveValue(JavascriptEvaluation.java:1)

at lH.onResult(chromium-Monochrome.aab-stable-432418123:1)

at gH.run(chromium-Monochrome.aab-stable-432418123:1)

at android.os.Handler.handleCallback(Handler.java:789)

at android.os.Handler.dispatchMessage(Handler.java:98)

at android.os.Looper.loop(Looper.java:164)

at android.app.ActivityThread.main(ActivityThread.java:6938)

at java.lang.reflect.Method.invoke(Native Method)

at com.android.internal.os.Zygote$MethodAndArgsCaller.run(Zygote.java:327)

at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1374)