[Haskell-cafe] Non-deterministic behaviour of aeson's parser

[Roman Cheplyaka]

I am observing a non-deterministic behaviour of aeson's parser.

I'm writing here in addition to filing a bug report [1] to draw
attention to this (pretty scary) problem.

To try to reproduce this problem, do this:

git clone https://gist.github.com/5604887.git aeson
cd aeson
ghc aeson.hs
./aeson | sort | uniq -c

This is my result:

32 Left "key \"module\" not present"
55 Left "When parsing the record SymValue of type Main.SymValueInfo the key fixity was not present."
1913 Right ()

Can others reproduce this in their environments?

Does anyone have ideas about where the bug may lie?
Many aeson's dependencies do unsafe IO stuff that could lead to
such consequences.

Roman

[1]: https://github.com/bos/aeson/issues/125

_______________________________________________
Haskell-Cafe mailing list
Haskel...@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Mateusz Kowalczyk]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Here's my result:

3 Left "key \"module\" not present"
8 Left "When parsing the record SymValue of type

Main.SymValueInfo the key fixity was not present."

1989 Right ()


- --
Mateusz K.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iQIcBAEBAgAGBQJRl7RYAAoJEM1mucMq2pqXt+AQALBsY3qgOawn4lHWK4myxXms
ofwxyED2fJC+4BjjdfMrbHcWYpEM43K+QlAElx7sQcM/VCCjqRiJ61zHyLnicILE
PwxF0/J0QQOkLF3/Q4ND7zgKqxbJWjOW2V6tf/WQulL4eKl1o40Xl4wuhoNqFrIA
g5GKcY/MPdBzbcnOwep6DdRhm42SrO+WnmmWoNYPNWA43dZwatlBsCklS3Pj+SdS
AYl10ZwNkMBT8Pj4N5A/z9C8LPKiZUDjuZxpMQT+PY4Ct318HtZVWDX3fM5QGvuE
HVqBPlBZIQn+VWBDFVe3LqhfvAVpvmflJ99DVGeO73kBhEf4FMhVfnXcGDayYYn9
PGF4n7L7MP4hNVlQtQhWkLKgWz0Z+VND+6c8yGceEpak4WqpzSmBKXIpMa8fw7tz
UXHUdJiVt6ZLGr1LjqDD3399+DCySj56XfyQk57xTn5gRjHtfz7QPSzgydDZzsJJ
8WHwQG1yHzNDfPATQwuZWss8FR53TXZbsn7zPfcJsro8bogNc2OH53HiVKoOm8l3
xH+A0D7169VCcKLvcVLmOb8gzAS2PiClm9/9H3/ho7iaU7gcYkZTaUO3JNz72mpn
Z0dS2YIdgmUdEY6Ws5DSmySOmP0QdFUt7xMK2lfExugdPLatcT0D+CGvNRJpfNXB
uL4I9+taoheV+3SLbM6M
=iGjG
-----END PGP SIGNATURE-----

[MigMit]

My result: 2000 lines of "Right ()"

ghc-pkg list aeson says "aeson-0.6.1.0"

[Niklas Hambüchen]

Can't reproduce:

% ./aeson | sort | uniq -c
2000 Right ()
% ./aeson | sort | uniq -c
2000 Right ()
% ./aeson | sort | uniq -c
2000 Right ()
% ./aeson | sort | uniq -c
2000 Right ()
% ./aeson | sort | uniq -c
2000 Right ()

Time 100:

% ./aeson | sort | uniq -c
200000 Right ()


My packages:

% ghc-pkg list
/var/lib/ghc/package.conf.d
Cabal-1.16.0
array-0.4.0.1
base-4.6.0.1
bin-package-db-0.0.0.0
binary-0.5.1.1
bytestring-0.10.0.2
containers-0.5.0.0
deepseq-1.3.0.1
directory-1.2.0.1
filepath-1.3.0.1
ghc-7.6.2
ghc-prim-0.3.0.0
haskell2010-1.1.1.0
haskell98-2.0.0.2
hoopl-3.9.0.0
hpc-0.6.0.0
integer-gmp-0.5.0.0
old-locale-1.0.0.5
old-time-1.1.0.1
pretty-1.1.1.0
process-1.1.0.2
rts-1.0
template-haskell-2.8.0.0
time-1.4.0.1
unix-2.6.0.1
/home/niklas/.ghc/x86_64-linux-7.6.2/package.conf.d
HTTP-4000.2.8
HUnit-1.2.5.2
QuickCheck-2.6
Xauth-0.1
aeson-0.6.1.0
ansi-terminal-0.6
attoparsec-0.10.4.0
attoparsec-binary-0.2
base-unicode-symbols-0.2.2.4
blaze-builder-0.3.1.1
byteorder-1.0.4
cipher-aes-0.1.8
convertible-1.0.11.1
cpphs-1.16
dlist-0.5
ghc-paths-0.1.0.9
ghc-syb-utils-0.2.1.1
hashable-1.2.0.6
haskell-lexer-1.0
haskell-src-exts-1.13.5
hidapi-1.0
hlint-1.8.44
hscolour-1.20.3
hspec-1.5.4
hspec-expectations-0.3.2
io-choice-0.0.3
lifted-base-0.2.0.4
monad-control-0.3.2.1
mtl-2.1.2
network-2.4.1.2
parsec-3.1.3
pretty-show-1.5
primitive-0.5.0.1
quickcheck-io-0.1.0
random-1.0.1.1
robot-1.0.1.1
robot-1.1
setenv-0.1.0
stm-2.4.2
storable-record-0.0.2.5
syb-0.4.0
text-0.11.3.0
transformers-0.3.0.0
transformers-base-0.4.1
uniplate-1.6.10
unordered-containers-0.2.3.1
utility-ht-0.0.9
vector-0.10.0.1
vector-th-unbox-0.2.0.1
xhb-0.5.2012.11.23
zlib-0.5.4.1

[Clark Gaebel]

CNR with aeson 0.6.1.0 and ghc 7.6.3.

pkg-list output can be found at http://pastebin.com/Zuuujcaz

[Gregory Collins]

First off, everyone reporting results to this thread: your bug report would be much more helpful if you included your OS/architecture/GHC version combo, as well as the results of re-running the tests if you build "hashable" with "cabal install -f-sse2".

I have a funny feeling that this is a bug in hashable or unordered-containers. I'm guessing hashable, especially because of this:

    https://github.com/tibbe/hashable/issues/66

and because hashable has had subtle bugs in its C code before (see https://github.com/tibbe/hashable/issues/60).

G

--
Gregory Collins <gr...@gregorycollins.net>

[Clark Gaebel]

$ uname -a

Linux clark-laptop 3.9.0-2-ARCH #1 SMP PREEMPT Tue Apr 30 09:48:29 CEST 2013 x86_64 GNU/Linux

It'd take too long for my helpfulness to build with cabal install -fsse2 hashable and rebuild an environment.

If someone writes a bash script to do it (using cabal-dev please!), I'd be more than happy to run it and post the results.

  - Clark

[Roman Cheplyaka]

Indeed it looks like a bug in hashable — it goes away with hashable-1.1.2.5.

Building with -f-sse2 results in a linker error

Loading package hashable-1.2.0.7 ... linking ... ghc: /home/feuerbach/tmp/aeson/.cabal-sandbox/lib/i386-linux-ghc-7.6.3/hashable-1.2.0.7/libHShashable-1.2.0.7.a: unknown symbol `hashable_siphash24_sse2'
ghc: unable to load package `hashable-1.2.0.7'

Roman

* Gregory Collins <gr...@gregorycollins.net> [2013-05-18 22:19:38+0200]

[Gregory Collins]

It's definitely hashable. Here's a minimal failing test case: https://gist.github.com/gregorycollins/5748445

--
Gregory Collins <gr...@gregorycollins.net>