Redis Lua latency issue

[Bhuvan L]

I am trying to use lua for reading data from redis. 

Choosing lua script for reading because we want to  do some processing to save network IO. 

Read script is doing actions like:

  -  select only fields  from the  hash that are needed for current processing. 

  -  transform  content using cmsgpack to string and 

  - compress using lz4( was able to statically like lua-lz4 with redid-server bin).  

On performance  profiling found that from earlier vanilla(redis cmd) hgetall read to above lua read script with cmsgpack and lz4, latencies jumped from <2ms to 12ms. 

I went ahead to debug further as to why latencies are increasing, stripped down the script to contain just a call hgetall.  

i am observing significant difference in latencies   

1) hgetall  called Using Jedis - 99 percentile latency is around <3ms. 

2) hgetall under lua script called using Jedis - 99 percentile latency is around < 11ms. 

Script one liner as below. I am calling this using Jedis  evalsha post script load: 

    "return redis.call('hgetall','foo')"

why do we have significant overhead in executing same command under lua scripts. ? 

Is it recommended to call lua scripts on read path,when there is stringent latency requirement <5ms? 

[hva...@gmail.com]

You haven't mentioned pre-loading the Lua routine with SCRIPT LOAD.  Have you verified that Jedis is not sending the Lua routine each time with EVAL instead of EVALSHA, or that Jedis is not removing it with SCRIPT FLUSH?  The overhead of re-sending the Lua routine each time could explain the timing you've measured.