Operators for element-wise tensor logical operations

[Anthony Platanios]

I was wondering why there are no `.&&`, `.||`, etc. operators, similar to `.==`, `.!=`, etc. To me it felt natural trying to use `.&&` after being aware of `.==`.

Also, I noticed that there is an op defined in Ops.swift that is never used: `infix operator ≈ : ComparisonPrecedence`. Was that supposed to be replacing `elementsApproximatelyEqual`?

Thanks,

Anthony

[Chris Lattner]

On Apr 17, 2019, at 6:44 PM, Anthony Platanios <e.a.pl...@gmail.com> wrote:
> I was wondering why there are no `.&&`, `.||`, etc. operators, similar to `.==`, `.!=`, etc. To me it felt natural trying to use `.&&` after being aware of `.==`.

&& and || in swift are short circuiting operators like in C, and always return Bool.

The “.” operators are point wise vector operators, e.g. .== is a point wise comparison, which produces a vector/tensor of boolean values.

Because point wise operators execute on every element, they can’t be short circuiting, so .&& doesn’t make sense. You can use & though to get pointwise bitwise and.

> Also, I noticed that there is an op defined in Ops.swift that is never used: `infix operator ≈ : ComparisonPrecedence`. Was that supposed to be replacing `elementsApproximatelyEqual`?

I’m not sure about that, it is probably a dead experiment that should be removed.

-Chris

[Anthony Platanios]

Thanks Chris!

That's an interesting point! Does that have any effect in how the compiler would treat say `.&&`, or is it mainly a semantics issue? Because there are two parts to the interpretation of `&&`: (i) it performs a logical and operation, and (ii) it is short-circuiting. Given that it could not be short-circuiting for tensors when performed element-wise, using `.&&`, it could potentially still be used to perform a logical and operation. The main disadvantage to this would potentially be the expectation from users that this is short circuiting, but I don't really know how much of an issue this would be for `.&&` (assuming the convention of `.XX` being element-wise operations).

My concern is that the current `elementsLogicalAnd`, etc. are very verbose and can be used quite frequently. But yeah this is not super important and was more of a curiosity.

The bitwise operators are not really the same because the use case I'm thinking about is when you have two `Tensor<Bool>`s and you want to perform element-wise logical operations between them.

-Anthony

On Wednesday, April 17, 2019 at 11:12:16 PM UTC-4, Chris Lattner wrote:

[Brennan Saeta]

Quick question: what should the behavior for `.&` be given two `Tensor<Bool>`'s? (i.e. is it any different from `elementsLogicalAnd`?) -Brennan

--
You received this message because you are subscribed to the Google Groups "Swift for TensorFlow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to swift+un...@tensorflow.org.

[Richard Wei]

Hi Anthony,

On Apr 18, 2019, at 9:14 AM, Anthony Platanios <e.a.pl...@gmail.com> wrote:

Thanks Chris!

That's an interesting point! Does that have any effect in how the compiler would treat say `.&&`, or is it mainly a semantics issue? Because there are two parts to the interpretation of `&&`: (i) it performs a logical and operation, and (ii) it is short-circuiting. Given that it could not be short-circuiting for tensors when performed element-wise, using `.&&`, it could potentially still be used to perform a logical and operation. The main disadvantage to this would potentially be the expectation from users that this is short circuiting, but I don't really know how much of an issue this would be for `.&&` (assuming the convention of `.XX` being element-wise operations).

There’s no implementation issues with `.&&`. We prefer `.&` because we plan to align our operators with those that are already accepted by Swift Evolution, namely `.&`, `.^` and `.|` introduced in SE-0229 (the SIMD vectors proposal).

  static func .&(lhs: SIMDMask, rhs: SIMDMask) -> SIMDMask
  static func .^(lhs: SIMDMask, rhs: SIMDMask) -> SIMDMask
  static func .|(lhs: SIMDMask, rhs: SIMDMask) -> SIMDMask

My concern is that the current `elementsLogicalAnd`, etc. are very verbose and can be used quite frequently. But yeah this is not super important and was more of a curiosity.

We should switch to operators defined by SE-0229, but the problem is that we have not yet merged from master which has SIMD vectors available. That said, it’s reasonable to define those operators with the exact precedence and associativity in the TensorFlow library for now. Would you like to send a PR and remove `elementsLogicalAnd(_:)` and other methods?

The bitwise operators are not really the same because the use case I'm thinking about is when you have two `Tensor<Bool>`s and you want to perform element-wise logical operations between them.

I understand. Those operators are actually chosen by the Swift community to be the standard elementwise logical operators between vectors, so we are confident enough to align our APIs with that direction.

-Richard

-Anthony

On Wednesday, April 17, 2019 at 11:12:16 PM UTC-4, Chris Lattner wrote:

On Apr 17, 2019, at 6:44 PM, Anthony Platanios <e.a.p...@gmail.com> wrote:
> I was wondering why there are no `.&&`, `.||`, etc. operators, similar to `.==`, `.!=`, etc. To me it felt natural trying to use `.&&` after being aware of `.==`.

&& and || in swift are short circuiting operators like in C, and always return Bool.

The “.” operators are point wise vector operators, e.g.  .== is a point wise comparison, which produces a vector/tensor of boolean values.

Because point wise operators execute on every element, they can’t be short circuiting, so .&& doesn’t make sense.  You can use & though to get pointwise bitwise and.

> Also, I noticed that there is an op defined in Ops.swift that is never used: `infix operator ≈ : ComparisonPrecedence`. Was that supposed to be replacing `elementsApproximatelyEqual`?

I’m not sure about that, it is probably a dead experiment that should be removed.

-Chris

[Richard Wei]

Hi Brennan,

The semantics of `.&(_:_:)` should be the same as `elementsLogicalAnd(_:)`, as defined in SE-0229.

-Richard

[Chris Lattner]

On Apr 18, 2019, at 9:14 AM, Anthony Platanios <e.a.pl...@gmail.com> wrote:

Thanks Chris!

That's an interesting point! Does that have any effect in how the compiler would treat say `.&&`, or is it mainly a semantics issue? Because there are two parts to the interpretation of `&&`: (i) it performs a logical and operation, and (ii) it is short-circuiting.

Hey Anthony,

The compiler doesn’t know anything about these operators, they are defined in the standard library.  For example, here is the implementation of &&:

https://github.com/apple/swift/blob/master/stdlib/public/core/Bool.swift#L280

It uses the @autoclosure feature to implement the lazy behavior (assert also uses it, to avoid evaluating the argument when assertions are off).

Given that it could not be short-circuiting for tensors when performed element-wise, using `.&&`, it could potentially still be used to perform a logical and operation. The main disadvantage to this would potentially be the expectation from users that this is short circuiting, but I don't really know how much of an issue this would be for `.&&` (assuming the convention of `.XX` being element-wise operations).

My concern is that the current `elementsLogicalAnd`, etc. are very verbose and can be used quite frequently. But yeah this is not super important and was more of a curiosity.

I don’t think vector operations can be usefully short circuiting (unless implicitly all of the element wise comparisons have to match, or have to fail).  For non-lazy binary operations, we generally just use the normal & and | operators.

The .& and .| operators are weird ones that are defined over the SIMD mask types, I’m not sure that we should follow their pattern for tensors.  I’d be curious to know what Steve Canon thinks about this :-)

-Chris

The bitwise operators are not really the same because the use case I'm thinking about is when you have two `Tensor<Bool>`s and you want to perform element-wise logical operations between them.

-Anthony

On Wednesday, April 17, 2019 at 11:12:16 PM UTC-4, Chris Lattner wrote:

On Apr 17, 2019, at 6:44 PM, Anthony Platanios <e.a.p...@gmail.com> wrote:
> I was wondering why there are no `.&&`, `.||`, etc. operators, similar to `.==`, `.!=`, etc. To me it felt natural trying to use `.&&` after being aware of `.==`.

&& and || in swift are short circuiting operators like in C, and always return Bool.

The “.” operators are point wise vector operators, e.g.  .== is a point wise comparison, which produces a vector/tensor of boolean values.

Because point wise operators execute on every element, they can’t be short circuiting, so .&& doesn’t make sense.  You can use & though to get pointwise bitwise and.

> Also, I noticed that there is an op defined in Ops.swift that is never used: `infix operator ≈ : ComparisonPrecedence`. Was that supposed to be replacing `elementsApproximatelyEqual`?

I’m not sure about that, it is probably a dead experiment that should be removed.

-Chris