Benchmarks

[Rohan Verma]

Hey,

I tried to write a benchmark for comparing govaluate and cel-go as I could not find the wiki page mentioned in the README.

The results:

go test -benchmem -bench .
goos: linux
goarch: amd64
pkg: benchcelgo
BenchmarkCelGo-4         3000000               487 ns/op              56 B/op          3 allocs/op
BenchmarkGovaluate-4     2000000               836 ns/op              80 B/op          6 allocs/op
PASS
ok      benchcelgo      4.698s

I have uploaded the code here:

https://gist.github.com/rhnvrm/db4567fcd87b2cb8e997999e1366d406

Please suggest if my methodology is correct or not and if any improvements can be made in the benchmark.

Thanks and regards

Rohan

he...@rohanverma.net

[Tristan Swadell]

Hi Rohan,

Nice work! The benchmark looks great. 

I just hadn't gotten around to writing up the wiki yet which is why you couldn't find it. I wanted to go into more depth on the trade-offs between different use cases and  considerations for picking an expression evaluator, such as ease of use, parse / eval only, etc.  However, from a pure evaluation standpoint, you've nailed it. :) 

-Tristan

--
You received this message because you are subscribed to the Google Groups "CEL Go Discussion Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cel-go-discus...@googlegroups.com.
To post to this group, send email to cel-go-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cel-go-discuss/2b128d60-c85c-4f2f-9049-6d5a704dda68%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tristan Swadell]

Looks like there is a small bug where there's a missing string concatenation for CEL-go, so once that's fixed you'll want to re-run.

-Tristan

[Rohan Verma]

Ah, I guess I missed that. After fixing it and running again, the benchmarks seem to be performing similarly.

go test -cpuprofile cpu.prof -memprofile mem.prof -benchmem -bench .

goos: linux
goarch: amd64
pkg: benchcelgo

BenchmarkCelGo-4         2000000               656 ns/op              88 B/op          5 allocs/op
BenchmarkGovaluate-4     2000000               702 ns/op              72 B/op          5 allocs/op
PASS
ok      benchcelgo      4.321s

(https://gist.github.com/rhnvrm/db4567fcd87b2cb8e997999e1366d406)

Regards

Rohan

he...@rohanverma.net

To unsubscribe from this group and stop receiving emails from it, send an email to cel-go-...@googlegroups.com.

[Tristan Swadell]

Thanks for re-running the benchmarks. Govaluate is a clean fast little library. I think in my own benchmarks There's a pretty aggressive function binding step at program plan time that happens in both CEL-Go and Govaluate, but CEL's is a bit more exhaustive in trying to bind extensions to program steps, so it happened to perform better in the example from the README.

The biggest time and memory cost in CEL-Go is the conversion of inputs to CEL types. It's possible to avoid this, but would require a pretty significant refactor of the code. Early tests showed a 2x speed improvement though. 

-Tristan

 

To unsubscribe from this group and stop receiving emails from it, send an email to cel-go-discus...@googlegroups.com.

To post to this group, send email to cel-go-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/cel-go-discuss/8ed4f114-c170-4219-9702-faca17dc800b%40googlegroups.com.