AOT compilation of user libraries and generics

[Toby Gifford]

I'm trying to aot-compile some libraries of my own, think I have it mostly figured out. However, since I'm building on adt.xtm I want to also aot-compile this library. To this end I have edited the adt.xtm file to look as follows (the bits in red are added or changed from the original adt.xtm)

;; lib-loading config

(sys:load "libs/aot-cache/adt.xtm" 'quiet)

(sys:load-preload-check 'adt)

(define *xtmlib-adt-loaded* #f)

(impc:aot:suppress-aot-do

 (sys:load "libs/base/base.xtm"))

(impc:aot:insert-forms

 (sys:load "libs/base/base.xtm" 'quiet))

(impc:aot:insert-header "xtmadt")

...

...

(define *xtmlib-adt-loaded* #t)

(impc:aot:insert-footer "xtmadt")

I don't really understand what most of this stuff does, I just tried to copy what other aot-compiled libraries seem to have.

Compilation goes fine, and loading goes fine with 

(sys:load "libs/core/adt.xtm") 

which definitely loads to aot version. But I have a function in my library called join which (when manually evaluated) spits out some warnings.

(bind-func join:[String*,String*,List{String*}*]*

  (lambda (delim lst)

    (cond

     ((null? lst) null)

     (else

      (let* ((stl:List{String*}* (reverse lst))

             (loop (lambda (input:List{String*}* output:String*)

                     (cond

                      ((null? input) output)

                      (else

                       (loop (cdr input) (cat (car input) (cat delim output))))))))

        (loop (cdr stl) (car stl)))))))

Warning: attempting to re-type already typed closure cdr:[List{String*}*,List{String*}*]* to [List{String*}*,List{String*}*]*
Spec'zed:  cdr >>> [List{String*}*,List{String*}*]*
Warning: attempting to re-type already typed closure cons:[List{String*}*,String*,List{String*}*]* to [List{String*}*,String*,List{String*}*]*
Spec'zed:  cons >>> [List{String*}*,String*,List{String*}*]*
Warning: attempting to re-type already typed closure car:[String*,List{String*}*]* to [String*,List{String*}*]*
Spec'zed:  car >>> [String*,List{String*}*]*
Spec'zed:  reverse_inline >>> [List{String*}*,List{String*}*,List{String*}*]*
Spec'zed:  reverse >>> [List{String*}*,List{String*}*]*
Compiled:  join >>> [String*,String*,List{String*}*]*

If, however, I load the non-aot-compiled version of adt.xtm (by commenting out the first line of adt.xtm that tries to load the version in aot-cache), then no warnings are generated.

I wouldn't be concerned about the warnings, except that they seem to convert into actual errors when I attempt to aot-compile my own library file ybot_base.xtm, which looks something like this

(sys:load "libs/aot-cache/ybot_base.xtm" 'quiet)

(sys:load-preload-check 'ybot_base)

(define *xtmlib-ybot_base-loaded* #f)

(impc:aot:suppress-aot-do

 (sys:load "libs/core/adt.xtm"))

(impc:aot:insert-forms

 (sys:load "libs/core/adt.xtm" 'quiet))

(impc:aot:insert-header "xtmybot_base")

...

...

(bind-func join:[String*,String*,List{String*}*]*

  (lambda (delim lst)

    (cond

     ((null? lst) null)

     (else

      (let* ((stl:List{String*}* (reverse lst))

             (loop (lambda (input:List{String*}* output:String*)

                     (cond

                      ((null? input) output)

                      (else

                       (loop (cdr input) (cat (car input) (cat delim output))))))))

        (loop (cdr stl) (car stl)))))))

(define *xtmlib-ybot_base-loaded* #t)

(impc:aot:insert-footer "xtmybot_base")

which produces the following when aot-compiled:

...

...

Compiled:  join >>> [String*,String*,List{String*}*]*
Finished compiling: "libs/contrib/ybot/ybot_base.xtm"

JIT-compiling IR
**** DECL ****
declare %clsvar* @add_address_table (%mzone*, i8*, i32, i8*, i32, %clsvar*)
declare %String* @cat2_adhoc_W1N0cmluZyosU3RyaW5nKixTdHJpbmcqXQ (%String*, %String*)
declare i8* @cptr_value (i8*)
declare i8* @list_ref (i8*, i32, i8*)
declare void @llvm_destroy_zone_after_delay (%mzone*, i64)
declare %mzone* @llvm_pop_zone_stack ()
declare %mzone* @llvm_zone_callback_setup ()
declare i8* @llvm_zone_malloc (%mzone*, i64)
declare i8* @mk_cptr (i8*, i8*)
declare i8* @mk_i1 (i8*, i1)
declare i8* @mk_i64 (i8*, i64)
declare i8* @mk_i8 (i8*, i8)
declare i32 @strcmp (i8*, i8*)
declare i64 @strlen (i8*)
**** ENDDECL ****

LLVM IR: <string>:122:1: error: redefinition of type
%List_poly_PFN0cmluZyo_ = type {%String*,%List_poly_PFN0cmluZyo_*}
^
Compiler Error  Failed compiling LLVM IR

Again it has no complaints if I don't pre-compile the adt.xtm

My guess is that it doesn't really make sense to pre-compile generic functions, as you really only want the 'reified' specialisations compiled. In which case, can one somehow skip loading the generic functions once the specialisations are compiled?  More broadly, does this mean that generic function definitions should be kept in separate translation units for library design?

[Andrew Sorensen]

Hey Toby,

Yes, AOT and generics don't play nicely at the moment - it's a known issue. At the moment my advice is, don't try.

Sorry :(

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.