A compilation error with SCASM

[Victor Sucasas]

Hello,

I am trying to compile a large program, 300M lines, with a convolution. It compiles well the ‘asm’ file but then it throws an error when running SCASM. It is a big file, but I am not running out of memory. Below the backtrace. I would really appreciate if someone can give me some insights on how to proceed.

Thanks a lot!

Victor

———————  THE COMPILATION ERROR ——————— 

Processing tape test_obliv_nn_resnet18-0 with 17 blocks

Not merging open instructions in tape test_obliv_nn_resnet18-0

Compile offline data requirements...

Tape requires 97943552 bits in modp

Tape requires prime bit length 124

Program requires: {('modp', 'bit'): 97943552}

Memory size: defaultdict(<function <lambda> at 0x7f5e8a2cfdd0>, {'sr': 8192, 'c': 160448, 'r': 8192, 's': 7991378, 'sb': 8192})

Writing to ../../tii-oblivnn-may22/tii-obliviousnnlib/test_obliv_nn_resnet18/test_obliv_nn_resnet18.sch

Writing to ../../tii-oblivnn-may22/tii-obliviousnnlib/test_obliv_nn_resnet18/test_obliv_nn_resnet18-0.asm

Now running 

   ./scasm -O0 ../../tii-oblivnn-may22/tii-obliviousnnlib/test_obliv_nn_resnet18

+ cargo run --manifest-path Assembler/Cargo.toml --release --bin scale_repo_helper --quiet -- --verbose ../../tii-oblivnn-may22/tii-obliviousnnlib/test_obliv_nn_resnet18 -- --hide-warnings -O0

reading all `.asm` files in ../../tii-oblivnn-may22/tii-obliviousnnlib/test_obliv_nn_resnet18

processing: ../../tii-oblivnn-may22/tii-obliviousnnlib/test_obliv_nn_resnet18/test_obliv_nn_resnet18-0.asm... 

Failed to compile ../../tii-oblivnn-may22/tii-obliviousnnlib/test_obliv_nn_resnet18/test_obliv_nn_resnet18-0.asm

thread 'main' panicked at 'begin <= end (4294967294 <= 25) when slicing `# test_obliv_nn_resnet18-0--0

print_char4 757935405 # 0

print_char4 757935405 # 1

print_char4 757935405 # 2

print_char4 757935405 # 3

print_char4 757935405 # 4

print_char4 757935405 # 5

print_char4 757935405 # 6

print_char4 757935405 # 7

print_char4 539831`[...]', Assembler/src/span.rs:138:30

note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

———————  WITH MORE DETAILED BACKTRACE ——————— 

print_char4 539831`[...]', Assembler/src/span.rs:138:30

stack backtrace:

   0: rust_begin_unwind

             at /build/rustc-6496Ax/rustc-1.57.0+dfsg1+llvm/library/std/src/panicking.rs:517:5

   1: core::panicking::panic_fmt

             at /build/rustc-6496Ax/rustc-1.57.0+dfsg1+llvm/library/core/src/panicking.rs:100:14

   2: core::str::slice_error_fail

   3: scasm::span::Span::split_at_first_char

   4: scasm::lexer::parser::<impl scasm::lexer::Lexical>::parse

   5: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold

   6: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter

   7: scasm::compiler::Compiler::lex

   8: scasm::compiler::Compiler::parse_asm_buf

   9: scasm::compiler::Compiler::parse_asm

  10: scasm::main

note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

———————  Here if I try with CARGO RUN SCASM ——————— 

  Compiling structopt v0.3.20

   Compiling scale_documentation v0.1.0 (/home/suki/tii-mpclib-may22/tii-mpclib/Documentation)

   Compiling tracing-serde v0.1.2

   Compiling tracing-subscriber v0.2.14

   Compiling tracing-tree v0.1.7

   Compiling scasm v0.1.0 (/home/suki/tii-mpclib-may22/tii-mpclib/Assembler)

    Finished dev [unoptimized + debuginfo] target(s) in 45.04s

     Running `target/debug/scasm ../../tii-oblivnn-may22/tii-obliviousnnlib/test_obliv_nn_resnet18/test_obliv_nn_resnet18-0.asm file.bc`

thread 'main' panicked at 'attempt to add with overflow', Assembler/src/span.rs:77:22

stack backtrace:

   0: rust_begin_unwind

             at /build/rustc-7HAmMa/rustc-1.57.0+dfsg1+llvm/library/std/src/panicking.rs:517:5

   1: core::panicking::panic_fmt

             at /build/rustc-7HAmMa/rustc-1.57.0+dfsg1+llvm/library/core/src/panicking.rs:100:14

   2: core::panicking::panic

             at /build/rustc-7HAmMa/rustc-1.57.0+dfsg1+llvm/library/core/src/panicking.rs:50:5

   3: scasm::span::Span::lines::{{closure}}

             at ./Assembler/src/span.rs:77:22

   4: core::iter::adapters::map::map_try_fold::{{closure}}

             at /build/rustc-7HAmMa/rustc-1.57.0+dfsg1+llvm/library/core/src/iter/adapters/map.rs:91:28

   5: core::iter::traits::iterator::Iterator::try_fold

             at /build/rustc-7HAmMa/rustc-1.57.0+dfsg1+llvm/library/core/src/iter/traits/iterator.rs:1995:21

   6: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold

             at /build/rustc-7HAmMa/rustc-1.57.0+dfsg1+llvm/library/core/src/iter/adapters/map.rs:117:9

   7: core::iter::traits::iterator::Iterator::find_map

             at /build/rustc-7HAmMa/rustc-1.57.0+dfsg1+llvm/library/core/src/iter/traits/iterator.rs:2415:9

   8: <core::iter::adapters::filter_map::FilterMap<I,F> as core::iter::traits::iterator::Iterator>::next

             at /build/rustc-7HAmMa/rustc-1.57.0+dfsg1+llvm/library/core/src/iter/adapters/filter_map.rs:61:9

   9: alloc::vec::Vec<T,A>::extend_desugared

             at /build/rustc-7HAmMa/rustc-1.57.0+dfsg1+llvm/library/alloc/src/vec/mod.rs:2614:35

  10: <alloc::vec::Vec<T,A> as alloc::vec::spec_extend::SpecExtend<T,I>>::spec_extend

             at /build/rustc-7HAmMa/rustc-1.57.0+dfsg1+llvm/library/alloc/src/vec/spec_extend.rs:18:9

  11: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter_nested::SpecFromIterNested<T,I>>::from_iter

             at /build/rustc-7HAmMa/rustc-1.57.0+dfsg1+llvm/library/alloc/src/vec/spec_from_iter_nested.rs:37:9

  12: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter

             at /build/rustc-7HAmMa/rustc-1.57.0+dfsg1+llvm/library/alloc/src/vec/spec_from_iter.rs:33:9

  13: <alloc::vec::Vec<T> as core::iter::traits::collect::FromIterator<T>>::from_iter

             at /build/rustc-7HAmMa/rustc-1.57.0+dfsg1+llvm/library/alloc/src/vec/mod.rs:2517:9

  14: core::iter::traits::iterator::Iterator::collect

             at /build/rustc-7HAmMa/rustc-1.57.0+dfsg1+llvm/library/core/src/iter/traits/iterator.rs:1745:9

  15: scasm::lexer::parser::parse

             at ./Assembler/src/lexer/parser.rs:9:5

  16: scasm::compiler::Compiler::lex

             at ./Assembler/src/compiler.rs:91:9

  17: scasm::compiler::Compiler::parse_asm_buf

             at ./Assembler/src/compiler.rs:98:21

  18: scasm::compiler::Compiler::parse_asm

             at ./Assembler/src/compiler.rs:87:9

  19: scasm::read_input

             at ./Assembler/src/main.rs:187:34

  20: scasm::main

             at ./Assembler/src/main.rs:133:9

  21: core::ops::function::FnOnce::call_once

             at /build/rustc-7HAmMa/rustc-1.57.0+dfsg1+llvm/library/core/src/ops/function.rs:227:5

note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

[Victor Sucasas]

It seems that the problem shows up when more than 150 million lines (aprox) are compiled. Does someone know if there is any limitation in the number of lines that SCASM can support?

Thanks!

Victor

[Iván Santos González]

Hello Victor!

The problem is caused by a type conversion error in the original span.rs file on line 73.

                72 let start = substr_offset(snippet, file_str).unwrap();
                73 let start = u32::try_from(start).unwrap();

The type usize (type of the variable start in line 72) in rust has a maximum value of 2^64 - 1, while the type we want to convert it to, u32, has a maximum value of 2^32 - 1, so in case of files with a large number of lines like the one we are talking about, an overflow of the data type is produced.
The solution to this problem is to change the data types in the Span file for both start and end, and perform this and other Span-related conversions using u64, which would be the correct data type.