Conversions without mathematical sense between polynomial rings

[Jean-Pierre Flori]

Dear all,

Currently one can obtain surprising results in Sage when converting polynomial over finite fields (or elements of quotient rings of univariate polynomial ring even though tht's not the primary concern of the ticket).
See http://trac.sagemath.org/ticket/11239
Basically the generators of the finite fields are just exchanged, even when the characteristics are different.
The changes suggested in this ticket would more or less only let a coercion be used if there is one, and rant if the characteristics are different or the base finite fields are not part of a common lattice when there is no canonical embedding.

Note though that the original disturbing conversion fits with the current behavior described in the doc:
* http://www.sagemath.org/doc/reference/coercion/index.html#maps-between-parents
where it's stated that

"Conversions need not be canonical (they may for example involve a choice of lift) or even make sense mathematically (e.g. constructions of some kind)."

Anyone has a strong opinion about what we should let Sage do in such a situation?

Should we leave the old conversion when there is no coercion even though that might easily to wrong mathematical results for a careless user? Or never look for a coercion unless the user explicitly asks for it (i.e. what the situation is currently in Sage without the patches of the ticket)?

Best,

JP

[Peter Bruin]

Jean-Pierre Flori wrote:

 

Currently one can obtain surprising results in Sage when converting polynomial over finite fields (or elements of quotient rings of univariate polynomial ring even though tht's not the primary concern of the ticket).
See http://trac.sagemath.org/ticket/11239
Basically the generators of the finite fields are just exchanged, even when the characteristics are different.
The changes suggested in this ticket would more or less only let a coercion be used if there is one, and rant if the characteristics are different or the base finite fields are not part of a common lattice when there is no canonical embedding.

Note though that the original disturbing conversion fits with the current behavior described in the doc:
* http://www.sagemath.org/doc/reference/coercion/index.html#maps-between-parents
where it's stated that

"Conversions need not be canonical (they may for example involve a choice of lift) or even make sense mathematically (e.g. constructions of some kind)."

Anyone has a strong opinion about what we should let Sage do in such a situation?

I posted a comment at #11239, so let me just say here that I think this principle (that conversions need not be canonical) shouldn't be pushed further than reasonable.  For example, if K is a field and f, g in K[x] are polynomials, then applying a conversion K[x]/(f) -> K[x]/(g) is a sensible thing to do if g divides f (the canonical map) or if f divides g (the lifting map).  However, I don't see why it is important to insist that the composition of two such maps (e.g. K[x]/(f) -> K[x] -> K[x]/(g) when f, g have nothing to do with each other) should again be a valid conversion that can be invoked with the same ease.  Hence I think that R(h), for R = K[x]/(g) and h in L[x]/(f), should only be allowed when K has a canonical map to L and either f divides g (in L[x]) or vice versa.  This should cover almost all practical uses, and in the rare remaining cases it seems better to require being a bit more explicit and doing the conversion in two steps.

Should we leave the old conversion when there is no coercion even though that might easily to wrong mathematical results for a careless user? Or never look for a coercion unless the user explicitly asks for it (i.e. what the situation is currently in Sage without the patches of the ticket)?

I thought that conversion should always give the same result as a coercion map whenever one exists; is there any (intended) situation where this is not the case?

Peter

[Simon King]

Hi Peter,

On 2013-12-31, Peter Bruin <pjb...@gmail.com> wrote:
> I posted a comment at #11239,

Then perhaps I should answer there, but anyway, here it goes.

Warning: I am partially playing "advocatus diavoli" here.

> so let me just say here that I think this
> principle (that conversions need not be canonical) shouldn't be pushed
> further than reasonable.

Yes, but for a rather relaxed notion of "reasonable". For example, you
couldn't even say that you have a "conversion from K[x]/(f) to
K[x]/(g)", since a conversion is not a map. It is a *partial* map,
hence, it often is the case that only polynomials of degree zero can be
converted and higher degree polynomials result in an error. Think of
QQ[x]->ZZ:
sage: a = QQ['x'](1)
sage: ZZ(a)
1
but
sage: a = QQ['x'](1/2)
sage: ZZ(a)
<boom>

> For example, if K is a field and f, g in K[x] are
> polynomials, then applying a conversion K[x]/(f) -> K[x]/(g) is a sensible
> thing to do if g divides f (the canonical map) or if f divides g (the
> lifting map). However, I don't see why it is important to insist that the
> composition of two such maps (e.g. K[x]/(f) -> K[x] -> K[x]/(g) when f, g
> have nothing to do with each other) should again be a valid conversion that
> can be invoked with the same ease.

Hang on. Nobody says that the composition of two conversions A->B and
B->C yields a conversion A->C (even if we forget for the moment that
conversions are not more than partially defined maps)! Composibility
is an axiom for *coercions*, but not for *conversions*. Actially conversion
have no axioms worth mentioning.

Hence, it would be perfectly fine to have K[x]/(f)->K[x] and
K[x]->K[x]/(g) for *all* elements of the involved rings, but
no conversion from K[x]/(f)->K[x]/(g), or actually: Only conversion for
polynomials of degree zero.

That said, do you really think one should do more or less expensive
operations (such as: compute gcd or a prime factorisation) to
verify that an easy conversion rule makes sense in a particular case?

You say it should not be pushed further than reasonable. Is it reasonable
to have an expensive test to decide whether or not to apply an easy
conversion rule? Recall: Coercions should obey axioms and thus expensive
tests may be needed. But why should one have expensive tests for something
that has no defined properties? How *could* you possibly test something
that has no defined properties?

> Hence I think that R(h), for R =
> K[x]/(g) and h in L[x]/(f), should only be allowed when K has a canonical
> map to L and either f divides g (in L[x]) or vice versa.

Do you claim we should drop conversion entirely? After all, we also have a
*coercion* system, and coercions (as opposed to conversions) are supposed to
be mathematically consistent. Do you think this would be reasonable?

> Should we leave the old conversion when there is no coercion even though
>> that might easily to wrong mathematical results for a careless user? Or
>> never look for a coercion unless the user explicitly asks for it (i.e. what
>> the situation is currently in Sage without the patches of the ticket)?
>>
> I thought that conversion should always give the same result as a coercion
> map whenever one exists; is there any (intended) situation where this is
> not the case?

If there is a coercion map then conversion must give the same result.
Everything else would be a bug.

Best regards,
Simon

[Peter Bruin]

Hi Simon,

Warning: I am partially playing "advocatus diavoli" here.

That can be very useful!
 

> so let me just say here that I think this
> principle (that conversions need not be canonical) shouldn't be pushed
> further than reasonable.

Yes, but for a rather relaxed notion of "reasonable". For example, you
couldn't even say that you have a "conversion from K[x]/(f) to
K[x]/(g)", since a conversion is not a map. It is a *partial* map,
hence, it often is the case that only polynomials of degree zero can be
converted and higher degree polynomials result in an error.

Sure, that still counts as reasonable for me.
 

>  For example, if K is a field and f, g in K[x] are
> polynomials, then applying a conversion K[x]/(f) -> K[x]/(g) is a sensible
> thing to do if g divides f (the canonical map) or if f divides g (the
> lifting map).  However, I don't see why it is important to insist that the
> composition of two such maps (e.g. K[x]/(f) -> K[x] -> K[x]/(g) when f, g
> have nothing to do with each other) should again be a valid conversion that
> can be invoked with the same ease.

Hang on. Nobody says that the composition of two conversions A->B and
B->C yields a conversion A->C (even if we forget for the moment that
conversions are not more than partially defined maps)! Composibility
is an axiom for *coercions*, but not for *conversions*. Actially conversion
have no axioms worth mentioning.

Yes, I shouldn't have said "insist" above, maybe "suggest".  I don't know if the fact that we have maps K[x]/(f) -> K[x]/(g) for arbitrary f, g (or Z/mZ -> Z/nZ for arbitrary m, n) was intended or just a side effect of not doing any checks and just doing conversions if the implementation allows it.  However, I'm really not convinced that it is easy for the average user to avoid the idea that such conversions apparently have some mathematical meaning, just because Sage makes it so easy to do these conversions.

Hence, it would be perfectly fine to have K[x]/(f)->K[x] and
K[x]->K[x]/(g) for *all* elements of the involved rings, but
no conversion from K[x]/(f)->K[x]/(g), or actually: Only conversion for
polynomials of degree zero.

Yes, that is more or less what I wanted to say.
 

That said, do you really think one should do more or less expensive
operations (such as: compute gcd or a prime factorisation) to
verify that an easy conversion rule makes sense in a particular case?

I don't know of an example where one has to do such expensive things; in the above examples, it is just a divisibility check.  In the examples of K[x]/(f) and Z/mZ, this divisibility check does look like a good sanity check to me.

You say it should not be pushed further than reasonable. Is it reasonable
to have an expensive test to decide whether or not to apply an easy
conversion rule? Recall: Coercions should obey axioms and thus expensive
tests may be needed. But why should one have expensive tests for something
that has no defined properties? How *could* you possibly test something
that has no defined properties?

It depends on how expensive the test is.  If I may play the devil's advocate for a moment: why would you want to *use* something that has no defined properties?

 

> Hence I think that R(h), for R =
> K[x]/(g) and h in L[x]/(f), should only be allowed when K has a canonical
> map to L and either f divides g (in L[x]) or vice versa.

Do you claim we should drop conversion entirely? After all, we also have a
*coercion* system, and coercions (as opposed to conversions) are supposed to
be mathematically consistent. Do you think this would be reasonable?

No, I am certainly not suggesting that conversion should go away.  Using conversion to find preimages in the maps Z -> Z/mZ or K[x] -> K[x]/(f) is something that makes a lot of sense.  But when it comes to being maximally permissive in allowing conversions (allowing for example direct conversions Z/3Z -> Z/2Z): despite some arguments for it (less checking, allowing people who really want to do that to save a few keystrokes) I don't see why it is a good thing.

If there is a coercion map then conversion must give the same result.
Everything else would be a bug.

At least there is no misunderstanding about that!

Best regards,

Peter

[john_perry_usm]

I sometimes illustrate techniques of polynomial factorization in a class by starting with a polynomial in ZZ[x] and converting it to a polynomial in GF(insert_large_prime)[x]. I realize this is not an instance of two _finite_ fields, but (a) you mention "the characteristics are different", and (b) _are_ there similar applications for finite fields, when the characteristics are different?

Or am I misunderstanding the proposal?

john perry

[Jean-Pierre Flori]

On Wednesday, January 1, 2014 5:30:09 AM UTC+1, john_perry_usm wrote:

I sometimes illustrate techniques of polynomial factorization in a class by starting with a polynomial in ZZ[x] and converting it to a polynomial in GF(insert_large_prime)[x]. I realize this is not an instance of two _finite_ fields, but (a) you mention "the characteristics are different", and (b) _are_ there similar applications for finite fields, when the characteristics are different?

I was rather thinking about positive characteristic, let's leave zero alone :)

[John Cremona]

On 1 January 2014 12:51, Jean-Pierre Flori <jpf...@gmail.com> wrote:
>
>
> On Wednesday, January 1, 2014 5:30:09 AM UTC+1, john_perry_usm wrote:
>>
>> I sometimes illustrate techniques of polynomial factorization in a class
>> by starting with a polynomial in ZZ[x] and converting it to a polynomial in
>> GF(insert_large_prime)[x]. I realize this is not an instance of two _finite_
>> fields, but (a) you mention "the characteristics are different", and (b)
>> _are_ there similar applications for finite fields, when the characteristics
>> are different?

This seems absolutely fine to me since there is a natural ring
homomorphism in that case (which is why it is useful for
factorization!). What I don't like is this:

sage: GF(7)(ZZ(GF(11)(7)))
0

where the convenient lifting function from GF(p) to ZZ is done
automatically instead of via the functions lift() and centerlift()
which give different answers of course:

sage: GF(7)((GF(11)(7)).lift())
0
sage: GF(7)((GF(11)(7)).centerlift())
3

John

>
> I was rather thinking about positive characteristic, let's leave zero alone
> :)
>

> --
> You received this message because you are subscribed to the Google Groups
> "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to sage-devel+...@googlegroups.com.
> To post to this group, send email to sage-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/sage-devel.
> For more options, visit https://groups.google.com/groups/opt_out.

[Peter Bruin]

Hello John,

What I don't like is this:

sage: GF(7)(ZZ(GF(11)(7)))
0

where the convenient lifting function from GF(p) to ZZ is done
automatically instead of via the functions lift() and centerlift()

It is even possible to completely obscure the fact that this conversion is done by lifting to ZZ:

sage: GF(7)(GF(11)(7))
0

The reason why this syntax works is the same as the problem from #11239 that started this thread, namely that conversion (as opposed to coercion) is done with very little checking.

It gets worse: another disastrous consequence is that one can construct an element of an empty set.

sage: f = GF(11).convert_map_from(GF(7))
sage: f
Conversion map:
  From: Finite Field of size 7
  To:   Finite Field of size 11
sage: f.parent()
Set of field embeddings from Finite Field of size 7 to Finite Field of size 11
sage: H = f.parent()
sage: H.order()
0
sage: H.list()
[]
sage: f in H
True

Not a very good way to start the new year!

Peter

[Simon King]

Hi Peter,

On 2014-01-01, Peter Bruin <pjb...@gmail.com> wrote:
> sage: f = GF(11).convert_map_from(GF(7))
> sage: f
> Conversion map:
> From: Finite Field of size 7
> To: Finite Field of size 11
> sage: f.parent()
> Set of field embeddings from Finite Field of size 7 to Finite Field of size 11

I'd say this is (close to) a bug.

A conversion map is (by definition) a partial map, that in some cases
happens to be an actual map (not just partial), in other cases happens
to even be a morphism, and in the best case is a coerce map.

So, we should think of being more precise, introduce "Set of partial maps from
R to S", and use it as parents of partial maps that aren't maps.
Similarly, one should more consistently distinguish between the set of
all maps and the set of all morphisms.

Cheers,
Simon

[Nils Bruin]

On Wednesday, January 1, 2014 3:32:05 PM UTC-8, Simon King wrote:

A conversion map is (by definition) a partial map, that in some cases
happens to be an actual map (not just partial), in other cases happens
to even be a morphism, and in the best case is a coerce map.

 

Given how few properties a "conversion map" is actually guaranteed to have, wouldn't it be sufficient to  just have a callable represent the conversion and not demand it's a map? I don't think we really need domain and codomain anyway. It also means that in a lot of cases, one could simply store to codomain rather than a map wrapping a call to the codomain.

[Peter Bruin]

Hi Simon,
 

I'd say this is (close to) a bug.

It is a bug, without qualification.
 

So, we should think of being more precise, introduce "Set of partial maps from
R to S", and use it as parents of partial maps that aren't maps.
Similarly, one should more consistently distinguish between the set of
all maps and the set of all morphisms.

That seems to be the only reasonable solution.

I would like to add that in my opinion the first line of this example,

sage: f = GF(11).convert_map_from(GF(7))

should not be assumed to work at all, and it would be preferable to raise an error rather than returning something that is mathematically completely arbitrary.

Given the principle that no conditions are imposed on conversions (except that if a coercion exists, the conversion equals the coercion), it should in particular be perfectly fine if some of them do not satisfy the property of existing at all.

Peter

[Nils Bruin]

On Wednesday, January 1, 2014 4:17:28 PM UTC-8, Peter Bruin wrote:

I would like to add that in my opinion the first line of this example,

sage: f = GF(11).convert_map_from(GF(7))

should not be assumed to work at all, and it would be preferable to raise an error rather than returning something that is mathematically completely arbitrary.

I think the fact that it is defined is a corollary of the fact that GF(11)(input) accepts as input anything that can be turned into an integer, and elements of GF(7) can be turned into an integer. I agree that it's probably preferable to not have this conversion, but its existence may be a natural consequence of the implementation and ensuring that it does not exist may require introducing special cases. You'll have to see if that can be done at acceptable cost.

[Peter Bruin]

New ticket: #15618 (convert_map should not be a map)

I also found another bug:

sage: A=Set([])
sage: A.convert_map_from(ZZ)
...
RuntimeError: BUG in coercion model, no element constructor for <class 'sage.sets.set.Set_object_enumerated_with_category'>

[Peter Bruin]

Hi Nils,

 

Given how few properties a "conversion map" is actually guaranteed to have, wouldn't it be sufficient to  just have a callable represent the conversion and not demand it's a map?

This sounds like a good idea if conversion remain the low-level things it is (i.e. possibly without any mathematical interpretation).  If at some point it is decided that conversions should have some nice properties (e.g. being actual maps with some mathematical meaning, or (partial) sections of them), then it would make more sense to make them some map-like type, but that wouldn't really be the concept that is now called conversion.

I don't think we really need domain and codomain anyway. It also means that in a lot of cases, one could simply store to codomain rather than a map wrapping a call to the codomain.

Except that currently (if A is the codomain) A.__call__() looks for a conversion and calls it, where the default conversion consists of calling A._element_constructor_(), so just letting the conversion f be the callable A itself instead would create a circularity (A(x) = A.__call__(x) = f(x) = A(x) = ...).  To implement your proposal, one could return "lambda x: return A._element_constructor_(x)" as the conversion.

Peter

[Nils Bruin]

On Thursday, January 2, 2014 2:42:13 AM UTC-8, Peter Bruin wrote:

Except that currently (if A is the codomain) A.__call__() looks for a conversion and calls it, where the default conversion consists of calling A._element_constructor_(), so just letting the conversion f be the callable A itself instead would create a circularity (A(x) = A.__call__(x) = f(x) = A(x) = ...).  To implement your proposal, one could return "lambda x: return A._element_constructor_(x)" as the conversion.

No, it could simply be (a weak reference to) A._element_constructor_, (yes, bound methods turn out to allow weak references!). A benefit would be that we do not need superfluous wrappers.

This brings up another point:I think the mail reason for reifying a "conversion" in the first place is because we want them cached for efficiency reasons. If they're going to be _element_constructor_ all the time anyway, I'm not so sure we need to cache them.

[Robert Bradshaw]

They're not always _element_constructor_, in particular, when it
exists, they should be the (partial) inverses of coercion maps (which
is automatically detected). It is unfortunate that conversions
encapsulate everything from section like Z/nZ -> Z and canonical
partial maps Q -> GF(p) to non-map constructors like list/dict -> Z[x]
to nonsense like GF(p) -> GF(q).

- Robert

[David Roe]

On Wed, Jan 1, 2014 at 2:17 PM, Peter Bruin <pjb...@gmail.com> wrote:

Hi Simon,
 

I'd say this is (close to) a bug.

It is a bug, without qualification.

Yes, it is a bug.

So, we should think of being more precise, introduce "Set of partial maps from
R to S", and use it as parents of partial maps that aren't maps.
Similarly, one should more consistently distinguish between the set of
all maps and the set of all morphisms.

That seems to be the only reasonable solution.

We have this already: sage.categories.sets_with_partial_maps.  But it looks like there are lots of places where it's not used appropriately:

sage: ZZ.convert_map_from(QQ).parent().category()
Category of hom sets in Category of rings
 

I would like to add that in my opinion the first line of this example,

sage: f = GF(11).convert_map_from(GF(7))

should not be assumed to work at all, and it would be preferable to raise an error rather than returning something that is mathematically completely arbitrary.

Given the principle that no conditions are imposed on conversions (except that if a coercion exists, the conversion equals the coercion), it should in particular be perfectly fine if some of them do not satisfy the property of existing at all.

Peter

[Simon King]

Hi David,

On 2014-01-08, David Roe <roed...@gmail.com> wrote:
> We have this already: sage.categories.sets_with_partial_maps.

We have the "category of sets with partial maps", but we don't have
fully-fledges partial maps, as much as I know.

Cheers,
Simon

[David Roe]

Various conversion "morphisms" (e.g. pAdicConvert_QQ_CR in sage/rings/padics/CR_template.pxi) raise an error on some input.  I would argue that this behavior is exactly what SetsWithPartialMaps is meant to encapsulate, together with the fact that that the map is just a map of sets on its valid domain).

David