Re: SHA1 digest different from commandline digest

[Josiah Carlson]

Redis' sha1 is the result of hashing the string *without* a trailing newline.
The command-line sha1 is the result of hashing the string *with* a
trailing newline.
Both use \n as new lines.

I don't know if this is because Redis strips the trailing newline, or
whether your client does.

Regards,
- Josiah

On Tue, Aug 28, 2012 at 1:47 AM, Kaan Ozdokmeci <ka...@solidict.com> wrote:
> Hi,
>
> After I load my script the sha1 digest I receive is different from the one I
> obtain on the commandline
> by using sha1sum.
>
> sha1 by redis: bd8d94deaa1113ee0c1beae00bab6de0cc88be89
> output of sha1sum: 1e89c0d7552a4185839bf075da24a25914a8af41
> (openssl dgst -sha1 also outputs the same result sha1sum outputs)
>
> script:
>
> local key = KEYS[1]
> local desired_size = tonumber(ARGV[1])
> local member_controller = {}
> local member_controller_count = 0
> local final_set = {}
> local setSize = tonumber(redis.call('scard',key))
>
> if setSize <= desired_size then
> final_set = redis.call('smembers',key)
> else
> repeat
> local candidate = redis.call('srandmember',key)
> if not member_controller[candidate] then
> member_controller[candidate] = true
> member_controller_count = member_controller_count + 1
> end
> until member_controller_count == desired_size
> for k,_ in pairs(member_controller) do final_set[#final_set+1] = k end
> end
>
> return final_set
>
>
> I am using redis-2.6.0-rc6 and ubuntu 12.04LTS (64bit) if that helps :)
>
> What might be the reason for this?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/redis-db/-/IWk5zCQJBfYJ.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to
> redis-db+u...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/redis-db?hl=en.

[Salvatore Sanfilippo]

Hi,

AFAIK Redis does not perform any post-processing of the script, so it
should not remove the trailing newline.

Salvatore

--
Salvatore 'antirez' Sanfilippo
open source developer - VMware
http://invece.org

Beauty is more important in computing than anywhere else in technology
because software is so complicated. Beauty is the ultimate defence
against complexity.
— David Gelernter

[Javier Guerra Giraldez]

On Tue, Aug 28, 2012 at 10:04 AM, Salvatore Sanfilippo
<ant...@gmail.com> wrote:
> Hi,
>
> AFAIK Redis does not perform any post-processing of the script, so it
> should not remove the trailing newline.

but the commandline often adds a newline if you're not careful:

#: echo 'hi' | sha1sum
55ca6286e3e4f4fba5d0448333fa99fc5a404a73 -

#: echo -n 'hi' | sha1sum
c22b5f9178342609428d6f51b2c5af4c0bde6a42 -



--
Javier

[Kaan Ozdokmeci]

#: cat scriptfile.lua | sha1sum

1e89c0d7552a4185839bf075da24a25914a8af41

#: redis-cli script load "$(cat /path/to/scriptfile.lua)"

bd8d94deaa1113ee0c1beae00bab6de0cc88be89

#: cat scriptfile.lua | head -c -1 | sha1sum

bd8d94deaa1113ee0c1beae00bab6de0cc88be89 

also,

#: cat scriptfile.lua | head -c -1 > script.txt

#: cat script.txt | sha1sum

bd8d94deaa1113ee0c1beae00bab6de0cc88be89 

#: redis-cli script load "$(cat /path/to/script.txt)"

bd8d94deaa1113ee0c1beae00bab6de0cc88be89

and,

#:diff scriptfile.lua script.txt ---> points out that script.txt has no newline at the end of the file (as expected)

this leads me to conclude that

1) cat is not the culprit here, it doesn't append new lines

2) since Salvatore mentioned that there is no post-processing of the scripts, redis-cli is stripping newlines.

Am I right? If it is, is this intentional?

Thanks for your responses,

[Kaan Ozdokmeci]

Also 

#: redis-cli set example "$(cat /path/to/scriptFile/withNewLine/atTheEnd)"

#: redis-cli get example

-----> output doesn't contain \n at the end

[Pieter Noordhuis]

Your shell removes the newline. Bash does this according to its
internal IFS variable, that determines how data is split.

You can make redis-cli read the last argument from STDIN to bypass
this. For example:

$ cat /path/to/scriptfile.lua | redis-cli -x script load

Cheers,
Pieter

> --
> You received this message because you are subscribed to the Google Groups
> "Redis DB" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/redis-db/-/3zoXbgwBn8wJ.