IREE execution model, dialects, and serialization

[Do Po]

Hello everybody,

We are trying to understand how IREE works internally (we want to try to add some real-time aspects to it).

For now, we are exploring the sequence of IR dumps along the compilation flow, and we have two questions:

1. In addition to the dialects that appear in the various presentations, a new dialect seems to be used, named "util". Can you tell us what is the purpose of this dialect?

2. We're not sure of understanding serialization (SerializeTargetExecutablePass, SerializeExecutablePass). Assuming that code is executed on a x86_64 CPU, is this pass doing compilation to binary x86_64 code, with no references to external function, OR does it compile to some form of bytecode?

Best regards,

Dumitru

[Scott Todd]

1. Per the source code, the util dialect is an "IREE base dialect used for types common across IREE subdialects.". Sometimes there are types /traits/operations that either don't exist yet in general purpose upstream dialects (arith, standard, etc.) or that we need for modeling programs across IREE internal dialects. This is partly related to the RFC to introduce a "ml_program" dialect upstream for modeling lists, global variables, and other useful constructs.

2. Executable serialization is dependent on the target and can have several flavors.

In the CPU (LLVM) case there is "static", "dynamic", and "embedded" linking (source here). Each of these produces an LLVM module with no external functions, emits object files, then links them in some way. Static linking outputs object files for future linking (using standard toolchains), dynamic/system linking produces .so/.dll/.dylib files (platform specific shared libraries), and embedded linking produces a portable ELF file.

The executable code (what you would run in GPU shaders or on a CPU thread pool) is not bytecode (though it could be if that target wanted it to be), and you can generally interact with compiled executables directly. For example, using the system linker ("iree-llvm-link-embedded=false") with the "iree-llvm-keep-linker-artifacts" flag will preserve .so/.dll/.dylib files that you can inspect with platform tools or even load + run outside of IREE (see for example executable_library_test). The rest of the .vmfb that IREE generates contains the host code that interfaces with those executables - loading them, preparing inputs, sequencing function calls together, etc. That host code runs on IREE's VM (or is used to generate C code via EmitC)

--
You received this message because you are subscribed to the Google Groups "iree-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to iree-discuss...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/iree-discuss/52d69744-ebb7-4e12-bd8c-b108c50da16bn%40googlegroups.com.