[RFC] Rethinking Functions/Modules As Operations

[River Riddle]

Hi everyone,

I worked with Mehdi on two related proposals, this one being a companion to the Arbitrary Regions in MLIR sent out earlier today.

Rethinking Functions/Modules As Operations

Introduction

This document details a proposal for changing the Function and Module abstraction to be represented via dialect operations instead of separate IR constructs. The main motivation for this proposal is to make MLIR more generic by reducing the amount of builtin concepts.

Module and Functions are concepts that are present in some form in most languages, and LLVM (amongst others) has been been very successfully built on top of these. However, having these as core concepts in MLIR forces a rigid structure at the top level of an IR that we try to otherwise keep abstract and generic.

Recognizing MLIR’s ambition to prevent such one-size-fits-all approaches, here are some examples of practical limitations that derive from the "Module" in LLVM (according to our experience):

• For heterogeneous computing, the unit of compilation includes both accelerator and host computations. This breaks a lot of the abstraction usually associated with a Module.

• When performing LTO (or other "advanced usage"), multiple modules are merged in order to enable inter-procedural transformations. Ideally, we would want the infrastructure to work seamlessly across module boundaries without having to actually unique names of internal symbols, named types, other various named objects that are internal to the module. MLIR should be able to represent an serialize a collection of related Modules inside a top-level LTO region.

• When converting a TensorFlow graph ("sea of nodes") into MLIR, we artificially create a Module and a single function to hold these. It is totally artificial to force this structure in the way we model the input.

Instead we propose that the top-level construct in MLIR is a dialect specific operation, and the concept of Module and Functions are provided by the standard dialect in MLIR but aren’t necessary to produce valid IR. We consider this proposal to be a similar generalization of the IR to the introduction of Region and non-CFG regions.

We will go over the challenges in transforming each of these IR structures into an operation.

Module

An MLIR Module is currently the top-level entity in the IR. It provides a list of functions and a symbol table (mapping names to function). It provides utility methods to insert a new function with auto-renaming by suffixing on name-collision. We propose to decouple this functionality from the IR operation representing the module. In the IR, the Module becomes represented as a regular operation with a single custom non-CFG region attached (more regions could be added in the future if needed). The region contains a single block, with operations (in no particular order), optionally using a specific attribute `std.symbol_name` to define their unique name within the module. The verifier for the Module operation ensures that symbol names are unique. The raw printer would look like:

"std.module"() ({  "std.global"() {std.symbol_name: "foo"} : () -> () }) {std.noncfg} : () -> ()

The `std.global` operation is used conceptually, as an example of the potential operations that could live in a Module in the future. A custom printer/parser for the module operation would reduce the module operation to the following syntax:

module {  "std.global"() {std.symbol_name: "foo"} : () -> () }

Function

A new operation `std.func` is introduced to declare and define functions within a Module. A dialect is allowed to reuse `std.func` in a dialect specific region (not a `std.module`), the semantic is then specific to this region and the enclosing operation. The `std.func` operation encloses an optional region that forms the function body in the case of a function definition. Other functions, are referenced via an attribute that matches the `std.symbol_name` attached of the `std.func` operation. This attribute has a special type `#std.ref` wrapping a string. We use a wrapper around the string to be able to opaquely find reference to global symbols in the IR. This referencing through string names allows for referencing globals within a Module without the need to manage use-lists or pointer invalidation (as we currently do with FunctionAttr).

Here is an example without custom printer/parser pretty syntax, illustrating the in-memory representation:

module {  "std.func"() ({    ^bb0(%arg0: i1):      %res = "std.call"(%arg0) { callee: #std.ref<"bar"> } : i1 -> i32      return %res : i32  }) {std.symbol_name: "foo", type: (i1) -> i32} : () -> ()  "std.func"() {std.symbol_name: "bar", type: (i1) -> i32} : () -> () }

The custom parser/printer should be able to turn this example into a more friendly, and hopefully familiar, syntax:

module {  func @foo(%arg0: i1) : (i1) -> i32 {    %res = call @bar(%arg0) : (i1) -> i32    return %res : i32  }  func @bar() : (i1) -> i32 }

Block

The concept of block is fundamental to CFG/SSA representation. After extensive considerations, we decided that some fundamental properties of blocks do not make them a good candidate to be turned into operations: in particular the values defined within a block are visible to the operations defined within other blocks. In terms of operations and regions, it means that a block operation would hold a region that defines values escaping in the parent region scope. This breaks a core design point of regions scoping, which we found to have deep implications in the current system. We consider that keeping blocks as a first class construct for this purpose seemed to be the best tradeoff.

A block in a non-CFG region is defined as a list of operations without any particular execution order associated to their order in the block.

Parser

The top-level entity returned by the parser when processing a textual MLIR assembly file is always a Module operation. If the first operation encountered by the parser in a mlir textual file is not a module, a new one is implicitly created. This preserves the existing behavior and won’t be surprising to anyone writing test files. It should be legal to write a file with a `std.func` at the top level but it will be returned wrapped in a module.

This implicit module does not constrain the kind of operations it can contain. Some potential use-cases that we would like to be able to support in the future could look like:

// implicitly wrapped in a top-level module thinlto_unit {  module {    ...  }  module {    ...  }  ... }

// implicitly wrapped in a top-level module heterogeneous {  module [target=”gpu”] {    ...  }  module [target=”host”] {    ...  }  ... }

// implicitly wrapped in a module dataflow.graph {  ... }

// implicitly wrapped in a module clang.tus {  // Multiple clang translation units, maybe summarized, for whole program static  // analysis.  clang.tu {    // internal representation  }  clang.tu {    // AST  }  ... }

Pass Manager

The ModulePass manager will schedule “Module-level Operation passes” in parallel on the operations immediately present in the Module. A FunctionPass is a specialization of an OperationPass that filters `std.func` operations.

Alternative Approaches Considered

Module Symbol Table Functionality

Remapping the module symbol table functionality essentially boils down to the need to have extra information attached to an operation that is mutable and non-serializable. It only exists in memory and must be modifiable. The approach that we decided on is to have a side data structure that is rebuilt on-demand. This works well for modules given the reasoning detailed above, but there are several other alternatives that were discussed during various brainstorming sessions.

Inline(and Mutable) Attributes

Attributes are currently the standard way to attach bags of information opaquely to any operation. These attributes need to be immutable and uniqueable, thus meaning that they will not work for storing a symbol table and thread lock. As an extension of attributes, we could allow for the ability to attach non-unique’d/immutable attributes that have the same lifetime as the operation they are attached to. This would provide a means to attach whatever we want to an operation, but many complexities arise that are mostly orthogonal to this design proposal. For example, a few unknowns: What are the other use cases for this feature? What happens during cloning? How are these attributes stored on an operation? Do we need two separate lists of attributes? Are these attributes ever printed, i.e. what is the round-tripping story?

“Immutable” Module Analysis

Another “interesting” alternative is to model this functionality as an analysis in the pass manager. This would provide a thread-safe interface to the Module operation that could be queried by function passes as needed. The problem with this is that Module analyses are generally not preserved by function passes, and thus every pass would have to somehow preserve it. Given the problem with preservation, this analysis would have to always be preserved regardless of the changes during a function pass. We would need a type of “structural” analysis on the modules that would promise not to touch anything accessible by function passes. It’s unclear if this would ever be needed outside of this use case.

“std.func” operation that returns a value

One interesting approach would be to have the `std.func` operation return a value. References to other functions/globals would then be explicitly captured as operands to the `std.func` operation, with implicit captures being forbidden. The entry block arguments of the region would match the arguments of the function type, followed by the mapping of the explicit captures as operands to the `std.func` operation.

Here is an example without custom printer/parser pretty syntax, illustrating the in-memory representation:

module {  %func_foo = "std.func"(%func_bar) {    ^bb0(%arg0: i1, %bar_capture : (i1) -> i32):      %res = "std.call"(%bar_capture, %arg0) : ((i1) -> i32, i1) -> i32      return %res : i32  } {std.symbol_name: "foo"} : ((i1) -> i32) -> ((i1) -> i32)  %func_bar = "std.func"() {std.symbol_name: "bar"} : () -> ((i1) -> i32) }

Custom parser/printer should be able to turn this example into a more friendly syntax like:

module {  %func_foo = func @foo [%func_bar as %bar : (i1) -> i32] (%arg0: i1)     : (i1) -> i32 {    %res = call %bar_capture(%arg0) : (i1) -> i32    return %res : i32  }  %func_bar = func @bar() : (i1) -> i32 }

The explicit captures would ensure that references to other functions in a module from a function body are kept local to the region: adding a call to a captured function would not affect the use-list of the value returned by the `std.func` defining it. This would help keep function passes isolated from each other and preserve our ability to run them in parallel. Adding a reference to another function in the module would then require adding a new capture. Finding the uses of a function would be reduced to a use-list traversal

However this approach would be similar to eliminate any direct references to globals through their name and have only the equivalent indirect call. While it seems trivial to walk the use-list on simple examples, regions with explicit capture would make a simple call target opaque.

[Chris Lattner]

On May 16, 2019, at 11:02 PM, 'River Riddle' via MLIR <ml...@tensorflow.org> wrote:

Rethinking Functions/Modules As Operations

I’m very excited to see this, thank you both!

Introduction

This document details a proposal for changing the Function and Module abstraction to be represented via dialect operations instead of separate IR constructs. The main motivation for this proposal is to make MLIR more generic by reducing the amount of builtin concepts.

And also to make MLIR more expressive and general. :-)

Module

An MLIR Module is currently the top-level entity in the IR. It provides a list of functions and a symbol table (mapping names to function). It provides utility methods to insert a new function with auto-renaming by suffixing on name-collision. We propose to decouple this functionality from the IR operation representing the module. In the IR, the Module becomes represented as a regular operation with a single custom non-CFG region attached (more regions could be added in the future if needed). The region contains a single block, with operations (in no particular order), optionally using a specific attribute `std.symbol_name` to define their unique name within the module. The verifier for the Module operation ensures that symbol names are unique. The raw printer would look like:

"std.module"() ({  "std.global"() {std.symbol_name: "foo"} : () -> () }) {std.noncfg} : () -> ()

The `std.global` operation is used conceptually, as an example of the potential operations that could live in a Module in the future. A custom printer/parser for the module operation would reduce the module operation to the following syntax:

module {  "std.global"() {std.symbol_name: "foo"} : () -> () }

It isn’t core to this design, but xref the other thread, I think that all unknown regions should default to being assumed to be “non cfg” and this bit should go on abstract operation.  Let's continue that discussion on the other thread tho.

More generally, what do you see as the advantage to having a std.symbol_name attribute?  Your introduction lays out many motivations which seem to indicate that a *single* global symbol table isn’t sufficient, and I’d argue it is required.  If you have heterogenous program with two programs in a top level “container” (which we could call std.module, but maybe renaming it to something else might reduce confusion) means you really want two different symbol tables.

Would it be reasonable to just leave it up to the dialect in question to name (and verify) their symbols in a sensible way?

If you go with this design, then “std.func” would still have a symbol name, and it would verify that they are unique etc, so I don’t think you’d lose anything.   It would make the top level container also completely independent of its contents.

Function

A new operation `std.func` is introduced to declare and define functions within a Module. A dialect is allowed to reuse `std.func` in a dialect specific region (not a `std.module`), the semantic is then specific to this region and the enclosing operation.

By “reuse” I think you’re saying that std.func is allowed to exist in arbitrary other regions: if so, +1.  This just means that FunctionPassManager does a walk of the module to find all the std.func and then runs its passes on each that it finds (no matter what the container is).

People will ask about nested functions, and they would be allowed by your description above.  Out of conservatism, I’d recommend making it very clear that operations within an std.func are never allowed to refer to operations outside of the std.func.  The std.func is a scope.

We may eventually want to allow that in some cases, but it is a can of worms, solvable by other dialect specific operations, and we can extend std.func when/if we have a use case for it.

The `std.func` operation encloses an optional region that forms the function body in the case of a function definition. Other functions, are referenced via an attribute that matches the `std.symbol_name` attached of the `std.func` operation. This attribute has a special type `#std.ref` wrapping a string. We use a wrapper around the string to be able to opaquely find reference to global symbols in the IR. This referencing through string names allows for referencing globals within a Module without the need to manage use-lists or pointer invalidation (as we currently do with FunctionAttr).

I don’t understand why we need a new attribute kind?  Are you saying the attribute is a string, but the `Type` of the string attribute is a new `std.ref` `Type`?  If so, that makes sense to me - I’d recommend going with a longer type name like function.ref or global ref or “symbol name”.

Here is an example without custom printer/parser pretty syntax, illustrating the in-memory representation:

module {  "std.func"() ({    ^bb0(%arg0: i1):      %res = "std.call"(%arg0) { callee: #std.ref<"bar"> } : i1 -> i32      return %res : i32  }) {std.symbol_name: "foo", type: (i1) -> i32} : () -> ()  "std.func"() {std.symbol_name: "bar", type: (i1) -> i32} : () -> () }

The custom parser/printer should be able to turn this example into a more friendly, and hopefully familiar, syntax:

module {  func @foo(%arg0: i1) : (i1) -> i32 {    %res = call @bar(%arg0) : (i1) -> i32    return %res : i32  }  func @bar() : (i1) -> i32 }

Nice.  I hope that the pretty syntax doesn’t include module{} at all.  Because the top level entity is always a module, it can be implicit in the syntax.  <<Later>> Ah, I see you mention that in the parser section, great!

Block

+1

Pass Manager

The ModulePass manager will schedule “Module-level Operation passes” in parallel on the operations immediately present in the Module. A FunctionPass is a specialization of an OperationPass that filters `std.func` operations.

The writing here confused me.  I think you’re saying that there are four things: ModulePassManager, ModulePass, OperationPass, and FunctionPass.  ModulePass’s are not implicitly parallel, OperationPass is parallel on arbitrary outer level entities in a module, and FunctionPass is a specialization of OperationPass.

The last bit is not obviously right to me.  Would it make sense for FunctionPassManager to be a module pass that finds std.func’s in potentially nested locations and run them?

Similarly, in your LTO example, you can have recursive parallelism.  For example, the top level operation pass operates on clang translation units, and wants to be parallel over the clang decls that are contained within it.

In any case, I’m super excited about this work.  Thank you both for pushing this forward!

-Chris

[River Riddle]

On Friday, May 17, 2019 at 9:59:20 AM UTC-7, Chris Lattner wrote:

On May 16, 2019, at 11:02 PM, 'River Riddle' via MLIR <ml...@tensorflow.org> wrote:

Rethinking Functions/Modules As Operations

I’m very excited to see this, thank you both!

Introduction

This document details a proposal for changing the Function and Module abstraction to be represented via dialect operations instead of separate IR constructs. The main motivation for this proposal is to make MLIR more generic by reducing the amount of builtin concepts.

And also to make MLIR more expressive and general. :-)

Module

An MLIR Module is currently the top-level entity in the IR. It provides a list of functions and a symbol table (mapping names to function). It provides utility methods to insert a new function with auto-renaming by suffixing on name-collision. We propose to decouple this functionality from the IR operation representing the module. In the IR, the Module becomes represented as a regular operation with a single custom non-CFG region attached (more regions could be added in the future if needed). The region contains a single block, with operations (in no particular order), optionally using a specific attribute `std.symbol_name` to define their unique name within the module. The verifier for the Module operation ensures that symbol names are unique. The raw printer would look like:

"std.module"() ({  "std.global"() {std.symbol_name: "foo"} : () -> () }) {std.noncfg} : () -> ()

The `std.global` operation is used conceptually, as an example of the potential operations that could live in a Module in the future. A custom printer/parser for the module operation would reduce the module operation to the following syntax:

module {  "std.global"() {std.symbol_name: "foo"} : () -> () }

It isn’t core to this design, but xref the other thread, I think that all unknown regions should default to being assumed to be “non cfg” and this bit should go on abstract operation.  Let's continue that discussion on the other thread tho.

Yes, the idea is to have unknown regions default to "non cfg", this is somewhat similar to the known bits of terminators.

 

More generally, what do you see as the advantage to having a std.symbol_name attribute?  Your introduction lays out many motivations which seem to indicate that a *single* global symbol table isn’t sufficient, and I’d argue it is required.  If you have heterogenous program with two programs in a top level “container” (which we could call std.module, but maybe renaming it to something else might reduce confusion) means you really want two different symbol tables.

Would it be reasonable to just leave it up to the dialect in question to name (and verify) their symbols in a sensible way?

Leave it up to which dialect? This is MLIR, we have N dialects :). The idea for std.symbol_name is that it only has context for the enclosing operation, i.e. only the parent of a std.func. It's sort of an opt-in system where the verifier of module ensure that each std.symbol_name within its region is unique. We can have the verifier be a trait that top-level operations hook into, but this is functionality that is currently provided by the module. We are trying to take the incremental step by preserving parts of the functionality we get from the current abstraction and then slowly/carefully rethinking what we actually need to preserve from the old world moving forward.

 

If you go with this design, then “std.func” would still have a symbol name, and it would verify that they are unique etc, so I don’t think you’d lose anything.   It would make the top level container also completely independent of its contents.

You mean having "std.func" verify that no other operations have its name? This is a bit weird in that an operation can now verify semantics on different operations.

Function

A new operation `std.func` is introduced to declare and define functions within a Module. A dialect is allowed to reuse `std.func` in a dialect specific region (not a `std.module`), the semantic is then specific to this region and the enclosing operation.

By “reuse” I think you’re saying that std.func is allowed to exist in arbitrary other regions: if so, +1.  This just means that FunctionPassManager does a walk of the module to find all the std.func and then runs its passes on each that it finds (no matter what the container is).

People will ask about nested functions, and they would be allowed by your description above.  Out of conservatism, I’d recommend making it very clear that operations within an std.func are never allowed to refer to operations outside of the std.func.  The std.func is a scope.

 I thought it was mentioned in the RFC, but "std.func" regions do not allow implicit captures.

We may eventually want to allow that in some cases, but it is a can of worms, solvable by other dialect specific operations, and we can extend std.func when/if we have a use case for it.

The `std.func` operation encloses an optional region that forms the function body in the case of a function definition. Other functions, are referenced via an attribute that matches the `std.symbol_name` attached of the `std.func` operation. This attribute has a special type `#std.ref` wrapping a string. We use a wrapper around the string to be able to opaquely find reference to global symbols in the IR. This referencing through string names allows for referencing globals within a Module without the need to manage use-lists or pointer invalidation (as we currently do with FunctionAttr).

I don’t understand why we need a new attribute kind?  Are you saying the attribute is a string, but the `Type` of the string attribute is a new `std.ref` `Type`?  If so, that makes sense to me - I’d recommend going with a longer type name like function.ref or global ref or “symbol name”.

The idea is to have a new dialect attribute kind 'GlobalRefAttr : public Attr...' that stores the reference as a string internally. It provides a kind differentiation between regular string attributes and global references. 

 

Quick hypothetical:

  Attribute attr = ...

  auto refAttr = attr.dyn_cast<GlobalRefAttr>();

  Identifier refName = refAttr.getName();

Here is an example without custom printer/parser pretty syntax, illustrating the in-memory representation:

module {  "std.func"() ({    ^bb0(%arg0: i1):      %res = "std.call"(%arg0) { callee: #std.ref<"bar"> } : i1 -> i32      return %res : i32  }) {std.symbol_name: "foo", type: (i1) -> i32} : () -> ()  "std.func"() {std.symbol_name: "bar", type: (i1) -> i32} : () -> () }

The custom parser/printer should be able to turn this example into a more friendly, and hopefully familiar, syntax:

module {  func @foo(%arg0: i1) : (i1) -> i32 {    %res = call @bar(%arg0) : (i1) -> i32    return %res : i32  }  func @bar() : (i1) -> i32 }

Nice.  I hope that the pretty syntax doesn’t include module{} at all.  Because the top level entity is always a module, it can be implicit in the syntax.  <<Later>> Ah, I see you mention that in the parser section, great!

Block

+1

Pass Manager

The ModulePass manager will schedule “Module-level Operation passes” in parallel on the operations immediately present in the Module. A FunctionPass is a specialization of an OperationPass that filters `std.func` operations.

The writing here confused me.  I think you’re saying that there are four things: ModulePassManager, ModulePass, OperationPass, and FunctionPass.  ModulePass’s are not implicitly parallel, OperationPass is parallel on arbitrary outer level entities in a module, and FunctionPass is a specialization of OperationPass.

The last bit is not obviously right to me.  Would it make sense for FunctionPassManager to be a module pass that finds std.func’s in potentially nested locations and run them?

Similarly, in your LTO example, you can have recursive parallelism.  For example, the top level operation pass operates on clang translation units, and wants to be parallel over the clang decls that are contained within it.

This PassManager section is not in any way a final design suggestion. The full details of how the pass manager is going to work in this new world is still being designed and thought about. The function pass manager could very well traverse and find function operations to operate on in parallel. This functionality is parallel to what a dialect may want for their own function or graph operations, i.e. any non implicitly capturing region could potentially operate in parallel.  

 

In any case, I’m super excited about this work.  Thank you both for pushing this forward!

-Chris

Thanks,

 River Riddle 

[Chris Lattner]

On May 17, 2019, at 10:28 AM, 'River Riddle' via MLIR <ml...@tensorflow.org> wrote:

More generally, what do you see as the advantage to having a std.symbol_name attribute?  Your introduction lays out many motivations which seem to indicate that a *single* global symbol table isn’t sufficient, and I’d argue it is required.  If you have heterogenous program with two programs in a top level “container” (which we could call std.module, but maybe renaming it to something else might reduce confusion) means you really want two different symbol tables.

Would it be reasonable to just leave it up to the dialect in question to name (and verify) their symbols in a sensible way?

Leave it up to which dialect? This is MLIR, we have N dialects :). The idea for std.symbol_name is that it only has context for the enclosing operation, i.e. only the parent of a std.func. It's sort of an opt-in system where the verifier of module ensure that each std.symbol_name within its region is unique.

If I have one mlir module with two programs in it (say for two devices) then I’ll need two different symbol tables, and I don’t want the functions from each program to be mixed together into one symbol table.

This can’t be done with std.func, but as soon as you have an std.func, someone will realize they can create “their own functions”, and we’ll soon have gpu.func, cpu.func, etc.

We can have the verifier be a trait that top-level operations hook into, but this is functionality that is currently provided by the module. We are trying to take the incremental step by preserving parts of the functionality we get from the current abstraction and then slowly/carefully rethinking what we actually need to preserve from the old world moving forward.

Got it, but I think you can preserve the functionality in a more narrow way: just say that “std.func” has a symbol_name attribute?

If you go with this design, then “std.func” would still have a symbol name, and it would verify that they are unique etc, so I don’t think you’d lose anything.   It would make the top level container also completely independent of its contents.

You mean having "std.func" verify that no other operations have its name? This is a bit weird in that an operation can now verify semantics on different operations.

We had an in-person discussion today about the need for a “module verifier” sort of pass that is independent of the per-function/top-level-operation verifier pass.  I think this could fit naturally into a design like that.

People will ask about nested functions, and they would be allowed by your description above.  Out of conservatism, I’d recommend making it very clear that operations within an std.func are never allowed to refer to operations outside of the std.func.  The std.func is a scope.

 I thought it was mentioned in the RFC, but "std.func" regions do not allow implicit captures.

Great.  “Allows capture” is another bit that we should eventually have an operations with regions, that affine.if/affine.for should have but other regions should not.

I don’t understand why we need a new attribute kind?  Are you saying the attribute is a string, but the `Type` of the string attribute is a new `std.ref` `Type`?  If so, that makes sense to me - I’d recommend going with a longer type name like function.ref or global ref or “symbol name”.

The idea is to have a new dialect attribute kind 'GlobalRefAttr : public Attr...' that stores the reference as a string internally. It provides a kind differentiation between regular string attributes and global references. 

 

Quick hypothetical:

  Attribute attr = ...

  auto refAttr = attr.dyn_cast<GlobalRefAttr>();

  Identifier refName = refAttr.getName();

Ok, but what purpose does this serve over-and-above a string attribute?

Pass Manager

The ModulePass manager will schedule “Module-level Operation passes” in parallel on the operations immediately present in the Module. A FunctionPass is a specialization of an OperationPass that filters `std.func` operations.

The writing here confused me.  I think you’re saying that there are four things: ModulePassManager, ModulePass, OperationPass, and FunctionPass.  ModulePass’s are not implicitly parallel, OperationPass is parallel on arbitrary outer level entities in a module, and FunctionPass is a specialization of OperationPass.

The last bit is not obviously right to me.  Would it make sense for FunctionPassManager to be a module pass that finds std.func’s in potentially nested locations and run them?

Similarly, in your LTO example, you can have recursive parallelism.  For example, the top level operation pass operates on clang translation units, and wants to be parallel over the clang decls that are contained within it.

This PassManager section is not in any way a final design suggestion. The full details of how the pass manager is going to work in this new world is still being designed and thought about. The function pass manager could very well traverse and find function operations to operate on in parallel. This functionality is parallel to what a dialect may want for their own function or graph operations, i.e. any non implicitly capturing region could potentially operate in parallel.  

Ok!

Thanks again for driving this ahead, I’m very excited to see this happen.

-Chris

[River Riddle]

On Friday, May 17, 2019 at 5:42:00 PM UTC-7, Chris Lattner wrote:

On May 17, 2019, at 10:28 AM, 'River Riddle' via MLIR <ml...@tensorflow.org> wrote:

More generally, what do you see as the advantage to having a std.symbol_name attribute?  Your introduction lays out many motivations which seem to indicate that a *single* global symbol table isn’t sufficient, and I’d argue it is required.  If you have heterogenous program with two programs in a top level “container” (which we could call std.module, but maybe renaming it to something else might reduce confusion) means you really want two different symbol tables.

Would it be reasonable to just leave it up to the dialect in question to name (and verify) their symbols in a sensible way?

Leave it up to which dialect? This is MLIR, we have N dialects :). The idea for std.symbol_name is that it only has context for the enclosing operation, i.e. only the parent of a std.func. It's sort of an opt-in system where the verifier of module ensure that each std.symbol_name within its region is unique.

If I have one mlir module with two programs in it (say for two devices) then I’ll need two different symbol tables, and I don’t want the functions from each program to be mixed together into one symbol table.

This can’t be done with std.func, but as soon as you have an std.func, someone will realize they can create “their own functions”, and we’ll soon have gpu.func, cpu.func, etc.

Then you will need gpu.call, cpu.call, etc. Every operation that holds a reference will need a different operation for each of these different symbol tables. Otherwise, symbol resolution is impossible. This should be fine for the most part I suppose.

 

We can have the verifier be a trait that top-level operations hook into, but this is functionality that is currently provided by the module. We are trying to take the incremental step by preserving parts of the functionality we get from the current abstraction and then slowly/carefully rethinking what we actually need to preserve from the old world moving forward.

Got it, but I think you can preserve the functionality in a more narrow way: just say that “std.func” has a symbol_name attribute?

Yes, that is fine if we are relying on some module-level per-op-type verifier.  

 

If you go with this design, then “std.func” would still have a symbol name, and it would verify that they are unique etc, so I don’t think you’d lose anything.   It would make the top level container also completely independent of its contents.

You mean having "std.func" verify that no other operations have its name? This is a bit weird in that an operation can now verify semantics on different operations.

We had an in-person discussion today about the need for a “module verifier” sort of pass that is independent of the per-function/top-level-operation verifier pass.  I think this could fit naturally into a design like that.

People will ask about nested functions, and they would be allowed by your description above.  Out of conservatism, I’d recommend making it very clear that operations within an std.func are never allowed to refer to operations outside of the std.func.  The std.func is a scope.

 I thought it was mentioned in the RFC, but "std.func" regions do not allow implicit captures.

Great.  “Allows capture” is another bit that we should eventually have an operations with regions, that affine.if/affine.for should have but other regions should not.

I don’t understand why we need a new attribute kind?  Are you saying the attribute is a string, but the `Type` of the string attribute is a new `std.ref` `Type`?  If so, that makes sense to me - I’d recommend going with a longer type name like function.ref or global ref or “symbol name”.

The idea is to have a new dialect attribute kind 'GlobalRefAttr : public Attr...' that stores the reference as a string internally. It provides a kind differentiation between regular string attributes and global references. 

 

Quick hypothetical:

  Attribute attr = ...

  auto refAttr = attr.dyn_cast<GlobalRefAttr>();

  Identifier refName = refAttr.getName();

Ok, but what purpose does this serve over-and-above a string attribute?

The idea is that you can understand the context of the attribute when you get it opaquely. Think about constant folding for example, if we fold a constant that holds a reference to a function we will want to propagate this to new operations. A StringAttr is completely opaque to any operations that want to use this newly folded constant.

  %cst = my.constant "foo" : () -> ()

  call_indirect %cst() : () -> ()

When folding 'call_indirect' we just get a StringAttr from %cst, folding this into a direct call involves assuming the context of the string. 'call_indirect' has to assume that the string attribute contains a function reference that it knows how to materialize. The thing I like about having an explicit attribute type is that it makes the semantics of the reference explicit. I can immediately know that it is a reference, and that I can handle it.

-- River

[Chris Lattner]

On May 18, 2019, at 10:11 AM, 'River Riddle' via MLIR <ml...@tensorflow.org> wrote:

On Friday, May 17, 2019 at 5:42:00 PM UTC-7, Chris Lattner wrote:

On May 17, 2019, at 10:28 AM, 'River Riddle' via MLIR <ml...@tensorflow.org> wrote:

More generally, what do you see as the advantage to having a std.symbol_name attribute?  Your introduction lays out many motivations which seem to indicate that a *single* global symbol table isn’t sufficient, and I’d argue it is required.  If you have heterogenous program with two programs in a top level “container” (which we could call std.module, but maybe renaming it to something else might reduce confusion) means you really want two different symbol tables.

Would it be reasonable to just leave it up to the dialect in question to name (and verify) their symbols in a sensible way?

Leave it up to which dialect? This is MLIR, we have N dialects :). The idea for std.symbol_name is that it only has context for the enclosing operation, i.e. only the parent of a std.func. It's sort of an opt-in system where the verifier of module ensure that each std.symbol_name within its region is unique.

If I have one mlir module with two programs in it (say for two devices) then I’ll need two different symbol tables, and I don’t want the functions from each program to be mixed together into one symbol table.

This can’t be done with std.func, but as soon as you have an std.func, someone will realize they can create “their own functions”, and we’ll soon have gpu.func, cpu.func, etc.

Then you will need gpu.call, cpu.call, etc. Every operation that holds a reference will need a different operation for each of these different symbol tables. Otherwise, symbol resolution is impossible. This should be fine for the most part I suppose.

Right, I don’t see a problem with that.

The other likely direction (if you want a single unified symbol table, which is weird but may be interesting for some cases) is to give std.func a “device” attribute or set of devices, or something.  The issue there is that you’d end up needing/wanting different function bodies in some cases.

There could be some (very long term) interesting things that could be done with that in some domains. Something like that could be interesting for multi versioning, for things like the apple macho architecture slices, etc.

In any case, none of these specific applications seem important, but “one” seems like a weird number to build in symbol table support for in this case.  I’m supportive of incremental development though, so if you think this is the right first step, then I’m +1 on that.  We can always revisit it later.

I don’t understand why we need a new attribute kind?  Are you saying the attribute is a string, but the `Type` of the string attribute is a new `std.ref` `Type`?  If so, that makes sense to me - I’d recommend going with a longer type name like function.ref or global ref or “symbol name”.

The idea is to have a new dialect attribute kind 'GlobalRefAttr : public Attr...' that stores the reference as a string internally. It provides a kind differentiation between regular string attributes and global references. 

 

Quick hypothetical:

  Attribute attr = ...

  auto refAttr = attr.dyn_cast<GlobalRefAttr>();

  Identifier refName = refAttr.getName();

Ok, but what purpose does this serve over-and-above a string attribute?

The idea is that you can understand the context of the attribute when you get it opaquely. Think about constant folding for example, if we fold a constant that holds a reference to a function we will want to propagate this to new operations. A StringAttr is completely opaque to any operations that want to use this newly folded constant.

  %cst = my.constant "foo" : () -> ()

  call_indirect %cst() : () -> ()

When folding 'call_indirect' we just get a StringAttr from %cst, folding this into a direct call involves assuming the context of the string. 'call_indirect' has to assume that the string attribute contains a function reference that it knows how to materialize. The thing I like about having an explicit attribute type is that it makes the semantics of the reference explicit. I can immediately know that it is a reference, and that I can handle it.

I don’t get it: what other meaning could a function-typed string constant being indirectly called mean?

I’m not strongly against a new attribute kind here, I just don’t see anything that requires it.

-Chris

[Mehdi AMINI]

Another motivating aspect I remember when discussing this with River was to be able to traverse the IR and find all references to a global symbol opaquely.

-- 

Mehdi

[Chris Lattner]

On May 18, 2019, at 1:15 PM, Mehdi AMINI <joke...@gmail.com> wrote:

When folding 'call_indirect' we just get a StringAttr from %cst, folding this into a direct call involves assuming the context of the string. 'call_indirect' has to assume that the string attribute contains a function reference that it knows how to materialize. The thing I like about having an explicit attribute type is that it makes the semantics of the reference explicit. I can immediately know that it is a reference, and that I can handle it.

I don’t get it: what other meaning could a function-typed string constant being indirectly called mean?

I’m not strongly against a new attribute kind here, I just don’t see anything that requires it.

Another motivating aspect I remember when discussing this with River was to be able to traverse the IR and find all references to a global symbol opaquely.

Ok, I could imagine a use case for that given a ‘replace all uses' sort of thing, e.g. where you are merging two identical functions.

I’m not sure that this is something that would be a reasonable dialect independent pass to do though, but I’m not strongly opposed to it if you think this is important.

-Chris

  

[Ben Vanik]

I'm really excited about the proposal for making function/module/etc operations (and support for dialects to add their own). Just wanted to call out that FMV is what I'm trying to do and anything that makes it easier gets a massive thumbs up from me :)

--
You received this message because you are subscribed to the Google Groups "MLIR" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mlir+uns...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/mlir/F93FC0B1-1240-41F7-B070-AFC6ACA0B01C%40google.com.

[Eric Schweitz US]

Interesting ideas on where MLIR might be heading.  Thank you, River.

It might be worth mentioning that you could fold "std.global" and "std.func" into a common abstraction in the design as well.  They are both objects that can be referenced by name from anywhere in the module context, as I understand it.  The distinction merely being their type and, consequently, how they can (ought) to be used. (Although the example below suggests the "std.global" object has a function type, I would assume that's a malleable choice.)

--

Eric

[Mehdi AMINI]

On Wed, May 22, 2019 at 8:31 AM Eric Schweitz US <esch...@nvidia.com> wrote:

Interesting ideas on where MLIR might be heading.  Thank you, River.

It might be worth mentioning that you could fold "std.global" and "std.func" into a common abstraction in the design as well.  They are both objects that can be referenced by name from anywhere in the module context, as I understand it.  The distinction merely being their type and, consequently, how they can (ought) to be used. (Although the example below suggests the "std.global" object has a function type, I would assume that's a malleable choice.)

To be clear: the idea of the example with "std.global" was just to show that we want to open the module to be able to contain other operations than function. We don't have immediate plan for actually introducing "std.global" and we haven't designed anything about it at this point.

Best,

-- 

Mehdi

 

--
You received this message because you are subscribed to the Google Groups "MLIR" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mlir+uns...@tensorflow.org.

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/mlir/4375a6fa-d735-438d-89cb-7862bbab50a9%40tensorflow.org.

[Mehdi AMINI]

FYI, the work on implementing this started: https://github.com/tensorflow/mlir/commit/974a65cba535b76cca2ae1dd9beaf1f2241274a3

(this will likely take a few weeks to get to the end state)

-- 

Mehdi

[Mehdi AMINI]

Just an update on this: River implemented it all :)

There is likely some more cleanup to do but in general everything is in place.

-- 

Mehdi