RFC: Operation availability

[Lei Zhang]

Hi All,

I’d like to introduce a generic mechanism to specify op availability. It can be used to implement op versioning, extension requirements, and others. If you don’t care about these, feel free to stop reading now. :)

Introduction

Dialects are evolving op collections. Typically there are backward/forward compatibility implications when introducing changes. This is especially true for edge dialects, which model external programming models or hardware so must mirroring the requirements.

A typical mechanism to guarantee compatibility is versioning. We see versioned ops in TensorFlow (Lite). But there are other ways. For example, in SPIR-V, some ops are available only under certain capabilities or extensions. Generally, these are all mechanisms to control the availability of ops.

This RFC proposes to introduce a generic mechanism to specify op availability.

Design Goals

• Generic. Different dialects have different requirements over op availability. Dialects should be able to define their own availability dimensions (version, extension, capability, etc.).

• Descriptive. We already use ODS for specifying op definitions in a concise way. Availability information is part of that and should be integrated into op definitions.

• Auto-generation as much as possible. From the descriptive spec, optimally we should generate all C++ code to ease development.

There are a few common cases we see regarding op availability: 

• An op itself can be present or missing depending on the version/extension/capability. 

• An op’s operand/attribute/result can be present or missing.

• The types supported by an op’s operands/attributes/results can differ among versions.

• An enum attribute case can be present or missing.

So we need to provide mechanisms to control the availability of the op itself, its operands, attributes, results, and also enum attribute cases.

Design Proposal

The plan is to add an Availability class and allow Op, Type, Attr, and EnumAttrCaseInfo in ODS to carray a list<Availability> field. These descriptive specifications will be used to generate C++ OpInterfaces for the availability dimension and each op’s implementation of those interfaces. Specifically:

// The base class for defining op availability dimensions. class Availability {   // Fields for controlling the generated C++ OpInterface   // The name for the generated C++ OpInterface subclass.   string interfaceName = ?;   // The documentation for the generated C++ OpInterface subclass.   string interfaceDescription = "";   // Fields for controlling the query function signature   // The query function's return type in the generated C++ OpInterface subclass.   string queryFnRetType = ?;   // The query function's name in the generated C++ OpInterface subclass.   string queryFnName = ?;   // Fields for controlling the query function implementation   // The logic for merging two availability requirements.   // This is used to derive the final availability requirement when,   // for example, an op has two operands and these two operands have   // different availability requirements.    code mergeAction = ?;   // The initializer for the final availability requirement.   string initializer = ?;   // A specific availability requirement's type.   string instanceType = ?;   // Fields for a concrete availability instance   // The specific availability requirement carried by a concrete instance.   string instance = ?; } // A modifier to attach availability spec to a certain type.  def TypeAvailableAt<Type type, list<Availability> avail> :      TypeConstraint<type.predicate, type.description> {   Type baseType = type;   list<Availability> availability = avail; } // A modifier to attach availability spec to a certain attribute kind.  class AttrAvailableAt<Attr attr, list<Availability> avail> :     Attr<attr.predicate, attr.description> {   let baseAttr = attr;   list<Availability> availability = avail; } class Op<...> {   // We can use `TypeAvailableAt` and `AttrAvailableAt` to attach availability   // spec on op operands/attributes/results.   let arguments = (ins ...);   let results = (outs ...);   // The list of availability specs for this op itself.   list<Availability> availability = []; }

Each Availability subclass is a distinct availability dimension. Version is a common one; we can also see extensions, capabilities for SPIR-V, and other possible dimensions.

Versioning

For example, versioning can be defined as:

class MinVersionBase<string name, I32EnumAttr scheme, I32EnumAttrCase min>     : Availability {   let interfaceName = name;   let queryFnRetType = scheme.returnType;   let queryFnName = "getMinVersion";   let mergeAction = "$final = static_cast<" # scheme.returnType # ">("                       "std::max(static_cast<int>($overall), "                       "static_cast<int>($instance)))";   let initializer = "static_cast<" # scheme.returnType # ">(uint32_t(0))";   let instanceType = scheme.cppNamespace # "::" # scheme.className;   let instance = scheme.cppNamespace # "::" # scheme.className # "::" #                  min.symbol; } class MaxVersionBase<string name, I32EnumAttr scheme, I32EnumAttrCase max>     : Availability {   let interfaceName = name;   let queryFnRetType = scheme.returnType;   let queryFnName = "getMaxVersion";   let mergeAction = "$final = static_cast<" # scheme.returnType # ">("                       "std::min(static_cast<int>($overall), "                       "static_cast<int>($instance)))";   let initializer = "static_cast<" # scheme.returnType # ">(~uint32_t(0))";   let instanceType = scheme.cppNamespace # "::" # scheme.className;   let instance = scheme.cppNamespace # "::" # scheme.className # "::" #                  max.symbol; }

Note that in the above, MinVersionBase and MaxVersionBase are designed to take an IntEnumAttrCaseBase. The rationale behind is that versioning is not continuous; it’s discrete. But each dialect may have its own versioning scheme. A unified way of handling this is to let each dialect define a versioning enum attribute. So that In ODS we refer to the enum cases uniformly. EnumsGen will help to generate a corresponding C++ enum class so a dialect can map it to whatever scheme it may want.

Example

An example usage would be:

// Define dialect versions. def EXAMPLE_V1 : I32EnumAttrCase<"V_1", 0>; def EXAMPLE_V2 : I32EnumAttrCase<"V_2", 1>; def EXAMPLE_V3 : I32EnumAttrCase<"V_3", 2>; // This attribute here is mainly for generating the C++ enum class for the version. // But it can also be used in op definition if suitable. EXAMPLEVersionAttr : I32EnumAttr<     "Version", "valid versions", [EXAMPLE_V1, EXAMPLE_V2, EXAMPLE_V3]> {   let cppNamespace = "::mlir::example"; } class MinVersion<I32EnumAttrCase min> : MinVersionBase<     "QueryMinVersionInterface", EXAMPLEVersionAttr, min> {   let interfaceDescription = [{ ... }]; } class MaxVersion<I32EnumAttrCase max> : MaxVersionBase<     "QueryMaxVersionInterface", EXAMPLEVersionAttr, max> {   let interfaceDescription = [{ ... }]; }

The above will generate two op interfaces: QueryMinVersionInterface and QueryMaxVersionInterface:

class QueryMinVersionInterface : public OpInterface<QueryMinVersionInterface, detail::QueryMinVersionInterfaceTraits> { public:   ::mlir::example::Version getMinVersion(); }; class QueryMaxVersionInterface : public OpInterface<QueryMaxVersionInterface, detail::QueryMaxVersionInterfaceTraits> { public:   ::mlir::example::Version getMaxVersion(); };

Each op in EXAMPLE dialect with availability spec will derive from the corresponding interface and have the implementation for the query methods (getMinVersion and/or getMaxVersion) synthesized. For example, if we have the following op:

def EXAMPLE_A_Op: Op<"a", ...> {   let availability = [MinVersion<EXAMPLE_V1>];   let arguments = (ins     AttrAvailabeAt<F32Attr, [MinVersion<Example_V2>]>:$attr   );   let results = (outs     AnyTypeOf<[I32, TypeAvailableAt<F32, [MinVersion<EXAMPLE_V3]>]>:$result   ); }

ODS should generate:

class AOp: Op<..., QueryMinVersionInterface::Trait> { public:   ::mlir::example::Version getMinVersion(); }; ::mlir::example::Version AOp::getMinVersion() {   ::mlir::example::Version final = static_cast<::mlir::example::Version>(uint32_t(0));   // Update with op availability requirement   final = static_cast<::mlir::example::Version>(std::max(static_cast<int>(final), static_cast<int>(::mlir::example::Version::V_1));   // Update with attribute availability requirement   {     if (/* check whether attr is f32 here */)       final = static_cast<::mlir::example::Version>(std::max(static_cast<int>(final), static_cast<int>(::mlir::example::Version::V_2));   }   // Update with result availability requirement   {     auto type = this->result()->getType();     if (/* check whether type is f32 here */)       final = static_cast<::mlir::example::Version>(std::max(static_cast<int>(final), static_cast<int>(::mlir::example::Version::V_3));   }   return final; }

The getMinVersion generated from the above look at the whole op’s availability info, and computes a final version requirement accordingly. This is where the mergeAction in Availability definition is used.

The above shows specifying the availability for a specific op. One can also set a default availability in the dialect base op to let every op have it and override in each op accordingly.

The above example can also give an indication that TypeAvailableAt and AttrAvailableAt, together with Availability subclasses, should be flexible enough to support common cases like adding more operands/attributes/results in newer versions or extending their supported types. 

Usage 

• Querying the availability. The above allows an op instance to tell what version(s) it is available in, what extension it requires, etc. This is the foundation of all other uses.

• Validating according to a specific target environment. This is based on the querying in the above and essentially looping over all ops to see whether they are not available to the execution environment.

• Controlling pattern application in dialect conversions. See the below.

Conversion Target

With op availability modelled, a conversion target can be further enhanced to be more fine-grained. We can register ops as dynamic legal depending on their availability. This allows us to target a dialect of a specific version. This way patterns can be written without worrying about op availability; the legalization framework will take care of pattern application and reject those generating ops not available in the current target environment.

For example, if we want to target Vulkan 1.1 (which requires up to SPIR-V 1.3 if without additional extension), we can describe a Vulkan 1.1 conversion target as:

ConvesionTarget vulkan11(...); // Returns true if the given `op` is legal to use under the current target // environment. auto isLegalOp = [&](Operation *op) {   // Make sure this op is available at the given version. Ops not implementing   // QueryMinVersionInterface/QueryMaxVersionInterface are available to all   // SPIR-V versions.   if (auto minVersion = dyn_cast<QueryMinVersionInterface>(op))     if (minVersion.getMinVersion() > SPV_V_1_3)       return false;   if (auto maxVersion = dyn_cast<QueryMaxVersionInterface>(op))     if (maxVersion.getMaxVersion() < SPV_V_1_3)       return false;      // Check extension/capability…      return true; }; vulkan11.addDynamicallyLegalDialect<SPIRVDialect>(       Optional<ConversionTarget::DynamicLegalityCallbackFn>(isLegalOp));

Comments welcome! :)

Thanks,

Lei

[Ronan Keryell]

On 12/3/19 11:14 AM, 'Lei Zhang' via MLIR wrote:

> Hi All,
>
> I’d like to introduce a generic mechanism to specify op availability. It
> can be used to implement op versioning, extension requirements, and
> others. If you don’t care about these, feel free to stop reading now. :)
>
> Introduction
>
> Dialects are evolving op collections. Typically there are
> backward/forward compatibility implications when introducing changes.
> This is especially true for edge dialects, which model external
> programming models or hardware so must mirroring the requirements.

Great!

> A typical mechanism to guarantee compatibility is versioning. We see
> versioned ops in TensorFlow (Lite). But there are other ways. For
> example, in SPIR-V, some ops are available only under certain

> capabilitiesor extensions. Generally, these are all mechanisms to
> control the availabilityof ops.

>
> This RFC proposes to introduce a generic mechanism to specify op
> availability.

We had some similar issues and discussion in SYCL about how to query for
some features & extensions from a pure C++ perspective.

> ODS should generate:
>
> classAOp:Op<...,QueryMinVersionInterface::Trait>{
>
> public:
>
> ::mlir::example::VersiongetMinVersion();
>
> };
>
>
> ::mlir::example::VersionAOp::getMinVersion(){
>
> ::mlir::example::Versionfinal=static_cast<::mlir::example::Version>(uint32_t(0));

>
>
> // Update with op availability requirement
>

> final=static_cast<::mlir::example::Version>(std::max(static_cast<int>(final),static_cast<int>(::mlir::example::Version::V_1));

It is not clear in your example what is run-time only and what is or can
be constexpr and known directly at compile-time.

If you generate a meta-description of your MLIR capabilities, it would
be nice to be able to introspect it at compile time with "if constexpr",
SFINAE, concepts or whatever to have the most efficient code matching
the description.

Thanks,
--
Ronan KERYELL, Xilinx Research Labs / San José, California.

[Lei Zhang]

This RFC mostly focuses on probing such properties at runtime. The example in question inspects a concrete op instance at runtime by looking at all bits (opcode/operands/results/attributes/etc.) to deduce what minimal version this specific op instance will require. Similarly we can do this kind of queries for a particular op instance regarding its maximal version (which can be used to deprecate an op) or capability/extension (SPIR-V has these availability dimensions). This has the benefit of working nicely with ConversionTarget's dynamic legal dialect/op settings so it allows us to target a specific version/extension/etc. of a dialect/op. Version conversion can also be achieved via ConversionTarget in this way, provided necessary patterns.

The availability spec is encoded statically in ODS though. I plan to support specifying availability information on the op itself, its operands/results/attributes, and enum cases. That are the places we can have customizations w.r.t. different versions/etc. based on my understanding. I guess they should cover a majority of cases. There may exist other cases; we can evaluate and extend later.

 

If you generate a meta-description of your MLIR capabilities, it would
be nice to be able to introspect it at compile time with "if constexpr",
SFINAE, concepts or whatever to have the most efficient code matching
the description.

This is interesting. These static availability spec may be useful to generate other stuff but I don't have a concrete use case yet so I haven't thought about them carefully. Do you mean using it to selectively compile patterns/conversions? The way I envisioned it is that we don't need to. We can just write all patterns without caring about required versions/extensions/capabilities and let the legalization framework to reject the ones generating ops unavailable in the target environment. (And we can give preferred patterns higher benefits if two patterns are both available under the same target environment. That is also consistent with what we are doing right now.) But I may misunderstand your points. Would you mind to elaborate a bit here?

[Lei Zhang]

A friendly reminder on this RFC. If you care about op versioning or other ways to control op availability (extensions, etc.), please feel free to share your thoughts! :)

Thanks,

Lei

[Ronan Keryell]

Hi Lei!

On 12/6/19 6:39 AM, Lei Zhang wrote:

> If you generate a meta-description of your MLIR capabilities, it would
> be nice to be able to introspect it at compile time with "if
> constexpr",
> SFINAE, concepts or whatever to have the most efficient code matching
> the description.
>
>
> This is interesting. These static availability spec may be useful to
> generate other stuff but I don't have a concrete use case yet so I
> haven't thought about them carefully. Do you mean using it to
> selectively compile patterns/conversions? The way I envisioned it is
> that we don't need to. We can just write all patterns without caring
> about required versions/extensions/capabilities and let the legalization
> framework to reject the ones generating ops unavailable in the target
> environment. (And we can give preferred patterns higher benefits if two
> patterns are both available under the same target environment. That is
> also consistent with what we are doing right now.) But I may
> misunderstand your points. Would you mind to elaborate a bit here?

I do not have any concrete use cases since I do not use MLIR yet.
But if I need to write some concrete C++ code for a specific dialect in
the back-end that has to behave differently on some operation versions
and I know precisely that I will have only some version in the current
operation description, it seems suboptimal to have some loop nests with
some runtime tests depending on some static information lost in the process.

Perhaps just having the data-structure describing the operations and
their introspection functions constexpr would allow this just because of
C++...

But I do not want to derail your RFC, since I do not have a real use
case and have only some vague ideas.

[Lei Zhang]

I see your points. I guess another way is to selectively populate patterns into the pass. MLIR core repo follows the convention that patterns belonging to the same category are placed together and exposed via a single populating function, like populateAffineToStdConversionPatterns(). We can have populate*V1Patterns(), populate*V2Patterns() and so on. If we are certain that the compilation is surely to only handle a subset, we can just configure to call the related populating functions. 

 

Perhaps just having the data-structure describing the operations and
their introspection functions constexpr would allow this just because of
C++...

But I do not want to derail your RFC, since I do not have a real use
case and have only some vague ideas.

These are good food for thought. Thanks! We can certainly look into making necessary extensions later.  :)