Question re: definition of new int preds (int?, pos-int?, etc.)

[Peter Taoussanis]

Hi there,

Am wondering what the rationale is for choosing to define these new preds as true only for *fixed precision* integers?

`integer?` is true for any integer type (including BigInt, BigInteger).

`int?` is currently true only for fixed precision integers.

So `(pos-int? 4N)` is currently `false`.

My understanding is that the new preds (pos-int?, neg-int?, nat-int?, etc.) are being added to core largely for the convenience of clojure.spec?

If that's correct, I'd suggest that defining `pos-int?` and co. to support the usual arb precision integers would be more useful and less surprising.

Three options as I see them:

1. Keep the current definitions, and surprise/inconvenience folks who want to support big integers (quite common in my experience).

2. Switch the definitions to include big integers, and inconvenience folks who might want to specifically check for *fixed precision* integers (rare in my experience).

3. Offer an equivalent set of preds: `pos-integer?`, `neg-integer?`, `nat-integer?` (excessive IMO, and possibly also confusing).

So I'm advocating for option 2.

If you require further philosophical motivations-

These preds are inevitably going to get used often for specs. Most/many Clojure functions (in core and libs) that deal with integers tend to support arb precision ints, a useful feature. Which means that specs are going to be written (esp. by beginners) that unintentionally exclude arb precision support.

Conversely: those that have requirements to specifically *exclude* arb precision ints, are probably more sophisticated users and thus better able to understand when/where they may need to provide this additional spec info to exclude arb precision ints.

For example, it'd be easy enough to write a `fixed-precision?` pred that composes with `pos-int?` and co.

Much thanks for taking the time to consider this!

Cheers :-)

[Peter Taoussanis]

Forgot to clarify, I'm talking here about Clojure 1.9.0-alpha8.

[Alex Miller]

On Monday, July 4, 2016 at 12:27:58 AM UTC-5, Peter Taoussanis wrote:

Hi there,

Am wondering what the rationale is for choosing to define these new preds as true only for *fixed precision* integers?

We considered both options and decided that fixed precision integers were the most useful set to target for extended predicates.

 

`integer?` is true for any integer type (including BigInt, BigInteger).

`int?` is currently true only for fixed precision integers.

So `(pos-int? 4N)` is currently `false`.

My understanding is that the new preds (pos-int?, neg-int?, nat-int?, etc.) are being added to core largely for the convenience of clojure.spec?

That is certainly one important motivation but these are potentially useful in other places as well.

If that's correct, I'd suggest that defining `pos-int?` and co. to support the usual arb precision integers would be more useful and less surprising.

I think that's a hypothesis without evidence. Clojure makes a visible distinction between fixed and arbitrary precision integers. Many functions work only with fixed precision due to either Java interop or performance. As Rich has said, "people find many things surprising" - that's rarely the best rationale for decision-making.

 

Three options as I see them:

1. Keep the current definitions, and surprise/inconvenience folks who want to support big integers (quite common in my experience).

I don't expect these to change.

 

2. Switch the definitions to include big integers, and inconvenience folks who might want to specifically check for *fixed precision* integers (rare in my experience).

I think your perception of common and rare is anecdotal. 

 

3. Offer an equivalent set of preds: `pos-integer?`, `neg-integer?`, `nat-integer?` (excessive IMO, and possibly also confusing).

Possibly something we'll do in the future. "No is temporary, yes is forever." Nothing is stopping you from doing this yourself for now.

 

So I'm advocating for option 2.

If you require further philosophical motivations-

These preds are inevitably going to get used often for specs. Most/many Clojure functions (in core and libs) that deal with integers tend to support arb precision ints, a useful feature. Which means that specs are going to be written (esp. by beginners) that unintentionally exclude arb precision support.

I think you should also consider the built-in generator support in clojure.spec.gen. If the *-int? preds supported arbitrary precision integers, then the generators would also generate bigints and I think in many cases you would find that this would not match users expectations. For example, every Java API (collections, strings, arrays, etc) uses fixed precision integers and in general many counter/index ops in Clojure expect fixed precision integers, not arbitrary precision.

 

Conversely: those that have requirements to specifically *exclude* arb precision ints, are probably more sophisticated users and thus better able to understand when/where they may need to provide this additional spec info to exclude arb precision ints.

For example, it'd be easy enough to write a `fixed-precision?` pred that composes with `pos-int?` and co.

This predicate would not compose well for generator support.

[Sean Corfield]

On 7/4/16, 7:13 AM, "Alex Miller" <cloju...@googlegroups.com on behalf of al...@puredanger.com> wrote:
> I think your perception of common and rare is anecdotal. 

Anecdotal evidence from World Singles:

I just did a quick pass over our 42K lines of Clojure and we use big integer and big decimal once each, and in both cases they are essentially for specific Java APIs. So fixed precision is common for us and arbitrary precision is vanishingly rare.

Sean Corfield -- (904) 302-SEAN -- (970) FOR-SEAN
World Singles -- http://worldsingles.com/

[Peter Taoussanis]

Hey Alex, thanks for the feedback!

I think your perception of common and rare is anecdotal.

Oh, entirely anecdotal.

 

As Rich has said, "people find many things surprising"

Sure enough :-) Wanted to test the hypothesis that there might be a majority feeling in this case that one thing might be significantly more surprising than the other.

I think you should also consider the built-in generator support in clojure.spec.gen. If the *-int? preds supported arbitrary precision integers, then the generators would also generate bigints and I think in many cases you would find that this would not match users expectations. For example, every Java API (collections, strings, arrays, etc) uses fixed precision integers and in general many counter/index ops in Clojure expect fixed precision integers, not arbitrary precision.

Appreciate the explanation. Was operating under the mistaken impression that Clojure's generally taken an arb-precision-support-by-default view. Can't put my finger on where I got that from, but certainly happy to adjust.

Keep up the great work, cheers :-)

[Peter Taoussanis]

Thanks for the anecdote Sean. It's looking likely then that I just tend to deal with more arb precision stuff than most. Wasn't sure.

[Sean Corfield]

On 7/5/16, 2:06 AM, "Peter Taoussanis" <cloju...@googlegroups.com on behalf of ptaou...@gmail.com> wrote:
> It's looking likely then that I just tend to deal with more arb precision stuff than most. Wasn't sure.

Yeah, I remember all the discussions around the changes in numeric handling (back in the 1.3 days?) and it seemed clear the community falls into two broad camps: those who live almost entirely in fixed precision land (and favor performance) and those who live almost entirely in arbitrary precision land (and favor accuracy).

Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood

[Mark Engelberg]

For me, the surprising thing about the names of the new predicates was that they worked on longs, and not just ints as the names would suggest.

[Alex Miller]

There are a paucity of words used to mean many overlapping things in this area, but our most recent conversations used

"int" to mean "fixed precision integers", covering all Java fixed precision integer types (long, int, short, byte). 

"integer" to mean the broader "fixed or arbitrary precision integer"

"long" to specifically mean Java longs

It would not surprise me if there are places where these terms are used in conflicting or contradictory ways in core due to shifting of terms over time. We favor backwards compatibility over consistency if there are such cases. In general, it's preferable if Clojure users primarily think in terms of fixed or arbitrary precision integers, and not Java types like "long" other than when doing interop or performance work.

On Tue, Jul 5, 2016 at 1:19 PM, Mark Engelberg <mark.en...@gmail.com> wrote:

For me, the surprising thing about the names of the new predicates was that they worked on longs, and not just ints as the names would suggest.

--
You received this message because you are subscribed to the Google Groups "Clojure Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure-dev...@googlegroups.com.
To post to this group, send email to cloju...@googlegroups.com.
Visit this group at https://groups.google.com/group/clojure-dev.
For more options, visit https://groups.google.com/d/optout.

[Mikera]

On Wednesday, 6 July 2016 01:53:18 UTC+8, Sean Corfield wrote:

On 7/5/16, 2:06 AM, "Peter Taoussanis" <cloju...@googlegroups.com on behalf of ptaou...@gmail.com> wrote:
> It's looking likely then that I just tend to deal with more arb precision stuff than most. Wasn't sure.

Yeah, I remember all the discussions around the changes in numeric handling (back in the 1.3 days?) and it seemed clear the community falls into two broad camps: those who live almost entirely in fixed precision land (and favor performance) and those who live almost entirely in arbitrary precision land (and favor accuracy).

Being a bit pedantic, but it depends on your definition of "accuracy". Fixed precision can by 100% accurate according to its spec (e.g. IEE754 for fixed precision doubles). Fixed precision integers like long are "accurate" according to the well-understood rules of modular arithmetic. And even things like BigRationals are not truly "mathematically accurate" when you consider things like transcendental numbers.

I think the debate is better framed as "performance" vs. "arbitrary precision"

FWIW I think very few people actually need "arbitrary precision". Financial transactions are probably the only common case, and even there the whole of the world's GDP will fit in a single long (expressed in US cents). People working with money will might want something like http://www.joda.org/joda-money/ anyway rather than rolling their own. I guess there may also be some people working in crypto where they need very big integers, but not seen much of that in the Clojure community.

I certainly appreciate the presence of rationals in Clojure however. Makes mathematical work much more elegant!