Screwy access violation

[Scott Jones]

I was trying to load a very large array into Julia (some of the utf8proc_data arrays), and because of a typo, I had:

const global utf8proc_sequences = Int32(...)

(with thousands of numbers between the ( and ) )

I know that should have been `Int32[...]`, but Julia accepted the syntax, but got an access violation...

Anybody know why it is dying like that?

Thanks!

 

[Yichao Yu]

On Sun, May 17, 2015 at 10:30 PM, Scott Jones
<scott.pa...@gmail.com> wrote:
> I was trying to load a very large array into Julia (some of the
> utf8proc_data arrays), and because of a typo, I had:
> const global utf8proc_sequences = Int32(...)

```
yuyichao% lldb -- ./julia -f -e 'Int32(rand(1:100, 10000)...)'
(lldb) target create "./julia"
Current executable set to './julia' (x86_64).
(lldb) settings set -- target.run-args "-f" "-e" "Int32(rand(1:100, 10000)...)"
(lldb) run
Process 31331 launching
Process 31331 launched: './julia' (x86_64)
Process 31331 stopped
* thread #1: tid = 31331, 0x00007ffff6af37bf libjulia.so`push_root + 9
at gc.c:1539, name = 'julia', stop reason = invalid address (fault
address: 0x2)
frame #0: 0x00007ffff6af37bf libjulia.so`push_root + 9 at gc.c:1539
1536 assert(v != NULL);
1537 jl_taggedvalue_t* o = jl_astaggedvalue(v);
1538 verify_val(v);
-> 1539 int bits = gc_bits(o);
1540 if (!gc_marked(o)) {
1541 return push_root((jl_value_t*)v, d, bits);
1542 }
(lldb) bt
```

Time for another GC but report. Or maybe Oscar can fix it right away? @carnaval

[Yichao Yu]

It also seems like there's a super deep recursion in inference (it
takes forever if I replace 10000 by 1000) so it would be nice to let
the typeinference give up at some point as well.

The segfault happens much faster with a call stack only ~300-400
levels deep so I guess it is not a stack overflow.

[Scott Jones]

That super deep recursion in inference *might* be what was causing me so many headaches with #11004...

[Yichao Yu]

On Sun, May 17, 2015 at 11:53 PM, Scott Jones
<scott.pa...@gmail.com> wrote:
> That super deep recursion in inference *might* be what was causing me so
> many headaches with #11004...

Both the crash and the deep recursion happens when julia is trying to
specialize a parametrized type on a super long tuple[1]. Is that what
you were doing?

(You've seen it already but just for the record, the segfault is
reported here[2], reported after finding a minimum example)

[1] https://github.com/JuliaLang/julia/issues/11321
[2] https://github.com/JuliaLang/julia/issues/11320

[Scott Jones]

I've got another problem, that may be related... now the tuple loads, I can access it in the REPL, but when I try to write a little loop to extract the values and store them in a better format,

it seems to be indexing the wrong locations...

I had the following:

const global NULL = 0
const global utf8proc_sequences = 1
const global unicode_properties = (
 (0, 0, 0, 0, NULL, NULL, -1, -1, -1, -1, -1, false,false,false,false, UTF8PROC_BOUNDCLASS_OTHER, 0),
(UTF8PROC_CATEGORY_CC, 0, UTF8PROC_BIDI_CLASS_BN, 0, NULL, NULL, -1, -1, -1, -1, -1, false, true, false, true, UTF8PROC_BOUNDCLASS_CONTROL, 0),
 (UTF8PROC_CATEGORY_CC, 0, UTF8PROC_BIDI_CLASS_BN, 0, NULL, NULL, -1, -1, -1, -1, -1, false, false, false, true, UTF8PROC_BOUNDCLASS_CONTROL, 0),
(UTF8PROC_CATEGORY_CC, 0, UTF8PROC_BIDI_CLASS_S, 0, NULL, NULL, -1, -1, -1, -1, -1, false, false, false, true, UTF8PROC_BOUNDCLASS_CONTROL, 0),
(UTF8PROC_CATEGORY_CC, 0, UTF8PROC_BIDI_CLASS_B, 0, NULL, NULL, -1, -1, -1, -1, -1, false, false, false, true, UTF8PROC_BOUNDCLASS_LF, 0),
...
)

There are 6447 tuples (all of length 17)

When I try to look at the values of the tuples, I get values all mixed up...

 

 

[Scott Jones]

Note - I changed it to a vector of tuples, and that sems to be working... I really want to change it to an array of immutable types though...

[Yichao Yu]

On Mon, May 18, 2015 at 9:46 AM, Scott Jones <scott.pa...@gmail.com> wrote:
> Note - I changed it to a vector of tuples, and that sems to be working... I
> really want to change it to an array of immutable types though...
>

So do you mean you've changed it to

const global unicode_properties = [

(0, 0, 0, 0, NULL, NULL, -1, -1, -1, -1, -1, false,false,false,false,
UTF8PROC_BOUNDCLASS_OTHER, 0),
(UTF8PROC_CATEGORY_CC, 0, UTF8PROC_BIDI_CLASS_BN, 0, NULL, NULL, -1,
-1, -1, -1, -1, false, true, false, true, UTF8PROC_BOUNDCLASS_CONTROL,
0),
(UTF8PROC_CATEGORY_CC, 0, UTF8PROC_BIDI_CLASS_BN, 0, NULL, NULL, -1,
-1, -1, -1, -1, false, false, false, true,
UTF8PROC_BOUNDCLASS_CONTROL, 0),
(UTF8PROC_CATEGORY_CC, 0, UTF8PROC_BIDI_CLASS_S, 0, NULL, NULL, -1,
-1, -1, -1, -1, false, false, false, true,
UTF8PROC_BOUNDCLASS_CONTROL, 0),
(UTF8PROC_CATEGORY_CC, 0, UTF8PROC_BIDI_CLASS_B, 0, NULL, NULL, -1,
-1, -1, -1, -1, false, false, false, true, UTF8PROC_BOUNDCLASS_LF, 0),
...

]

Or are you still using a **tuple** of tuples. (instead of an **Array**
of tuples)

If you are using a tuple of tuples, then the result you've got is not
surprising.

If you check my bug report, I've isolated the cause to be overflow in
the offset, which happens with 8192 Int64's and each element if your
big tuple is a small tuple and I expect it to be much bigger than a
Int64. You could easily get a wrong index.

Note that this doesn't only apply to tuple types, just that it is very
easy to create a long tuple type. You would probably have the same
issue with any immutable type and my guess is that currently the best
solution is to use an array (or maybe a C array and access with
cglobal?)

[Scott Jones]

I saw the problem with a tuple of tuples, it went away when I changed it to an array of tuples...

I'm trying to eliminate if possible the dependence on C... (and learn more about Julia), but I may be forced to stick with keeping

the structures in C...

This is a pretty nasty bug, if data just starts coming back incorrectly if you hit some internal hidden limit :-(

 

[Yichao Yu]

On Mon, May 18, 2015 at 10:51 AM, Scott Jones

Interfacing with C is also a big/important part of julia IMHO :P I
guess you can decide what is the most effecient way to do these.

> This is a pretty nasty bug, if data just starts coming back incorrectly if
> you hit some internal hidden limit :-(
>

Agree. There are other bug reports open for huge (tuple) types so I
guess there should be some discussion before I try to change the
offset field to a longer integer.

Raising an error on overflow should be easy though.

[Scott Jones]

At least, for now, please do the error on overflow!

[Yichao Yu]

On Mon, May 18, 2015 at 11:27 AM, Scott Jones
<scott.pa...@gmail.com> wrote:
> At least, for now, please do the error on overflow!

Done. https://github.com/JuliaLang/julia/pull/11330 ...

I was hesitated before submitting a PR because I think this issue is
not important enough to have a temporary workaround. However, I think
this overflow check is the right thing to do anyway....