Best way to package code written in SAGE

[David Møller Hansen]

Hi

So I'm getting to the point in my masters thesis where I have coded
some different .sage files and now I have to package it in some way
making it easy to use with SAGE for my professor and the censor.

I just want to explain the content of my implementation in short:
It is basically a slow pairing based signature system (BLS).
signature_scheme.sage file which uses some of the different components
in sage: field_ext, elliptic_curves etc.
And then it also uses a weil pairing function I've written in sage, I
do not want to make this part of the signature scheme code, since if I
have some extra time later I want to work on trying to make this
function faster (haven't really figured out how)
I'm currently in the process of wrapping the functions in my
signature_cheme.sage file in a signature_scheme class providing a
static scheme setup and maybe making it possible to generate a
interact in notebook() mode.

I've looked at making a .spkg but I'm not really sure that this is the
correct thing to do, since I want to call components in SAGE and all
the optional packages I've looked at is mostly just code ported from c
by wrapping with some python setup files (I haven't looked at all the
packages).

I guess what I am asking is: Is making a spkg from my sage code, the
right thing to do or is it complete nonsense? Since I've yet only
encountered spkg's made from external code bases. If there is a
counter example of this, then please let me know, then I can use it as
a template if I were to package my code.

[William Stein]

First, I think the best thing to do is to work on getting your code into
Sage itself soon. Even if it isn't optimal speedwise, that's not a show
stopper -- correct code that adds new functionality is something we
definitely want. We release new versions of Sage every 2 weeks or
less, so once in Sage your code would rapidly get distributed.
Regarding pairings, we specifically don't have *anything* in Sage for
computing any pairings.

Regarding your above question, probably the best thing for you to do
is a make a tar-ball or zip archive that contains all the sage files,
and instructions
about how to use them (e.g., attach "foo.sage" and type "blah" to test that
this works). Then put that code on a web page. Installing an
spkg has the drawback that it requires modifying the
Sage install (e.g., something that requires write permissions on a Sage
install), so wouldn't work for "end users" who didn't install Sage themselves
(e.g., on a multi-user system).

William

William

[David Møller Hansen]

Good point on the multi-user system. Maybe a script is the best way to
go about the problem of initializing the signature scheme for my
thesis professors.

I will gladly contribute, but I'm guessing that my code will be pretty
obsolete when the pbc library is ported into SAGE...

- I'll just have to be fast then :-)

Thank you for your advice.

/David

[William Stein]

On Thu, Nov 13, 2008 at 8:02 AM, David Møller Hansen
<da...@mollerhansen.com> wrote:
>
> Good point on the multi-user system. Maybe a script is the best way to
> go about the problem of initializing the signature scheme for my
> thesis professors.
>
> I will gladly contribute, but I'm guessing that my code will be pretty
> obsolete when the pbc library is ported into SAGE...
>
> - I'll just have to be fast then :-)

1. There is value to having a native Sage implementation of algorithms,
even if it is slower, since it can be more flexible and useful for teaching
and some research purposes.

2. Maybe you could help with getting "the pbc library is ported into SAGE",
which might mean making a Cython wrapper for it.

--
William Stein
Associate Professor of Mathematics
University of Washington
http://wstein.org

[Martin Albrecht]

> I will gladly contribute, but I'm guessing that my code will be pretty
> obsolete when the pbc library is ported into SAGE...

Is anyone working on this?

Cheers,
Martin

--
name: Martin Albrecht
_pgp: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x8EF0DC99
_www: http://www.informatik.uni-bremen.de/~malb
_jab: martinr...@jabber.ccc.de

[mabshoff]

On Nov 13, 8:13 am, Martin Albrecht <m...@informatik.uni-bremen.de>
wrote:

> > I will gladly contribute, but I'm guessing that my code will be pretty
> > obsolete when the pbc library is ported into SAGE...
>
> Is anyone working on this?

Not that I am aware of. The issue last time this came up was that pbc
switched to GPL V2 around the middle of 2008.

> Cheers,
> Martin

Cheers,

Michael

> --
> name: Martin Albrecht
> _pgp:http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x8EF0DC99
> _www:http://www.informatik.uni-bremen.de/~malb

> _jab: martinralbre...@jabber.ccc.de

[mabshoff]

On Nov 13, 8:18 am, mabshoff <mabsh...@googlemail.com> wrote:
> On Nov 13, 8:13 am, Martin Albrecht <m...@informatik.uni-bremen.de>
> wrote:
>
> > > I will gladly contribute, but I'm guessing that my code will be pretty
> > > obsolete when the pbc library is ported into SAGE...
>
> > Is anyone working on this?
>
> Not that I am aware of. The issue last time this came up was that pbc
> switched to GPL V2 around the middle of 2008.

This obviously should be GPL *V3* - sorry for the typo and hence the
double post, but I guess it is better to correct myself here.

Cheers,

Michael

[William Stein]

On Thu, Nov 13, 2008 at 8:18 AM, mabshoff <mabs...@googlemail.com> wrote:
>
>
>
> On Nov 13, 8:13 am, Martin Albrecht <m...@informatik.uni-bremen.de>
> wrote:
>> > I will gladly contribute, but I'm guessing that my code will be pretty
>> > obsolete when the pbc library is ported into SAGE...
>>
>> Is anyone working on this?
>
> Not that I am aware of. The issue last time this came up was that pbc
> switched to GPL V2 around the middle of 2008.

Do you mean GPLV2 as opposed to GPLV2+? Is that what the issue is?
I.e., is it GPLV3 incompatible?

>
>> Cheers,
>> Martin
>
> Cheers,
>
> Michael
>
>> --
>> name: Martin Albrecht
>> _pgp:http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x8EF0DC99
>> _www:http://www.informatik.uni-bremen.de/~malb
>> _jab: martinralbre...@jabber.ccc.de
> >
>

[mabshoff]

On Nov 13, 8:19 am, "William Stein" <wst...@gmail.com> wrote:

> On Thu, Nov 13, 2008 at 8:18 AM, mabshoff <mabsh...@googlemail.com> wrote:
>
> > On Nov 13, 8:13 am, Martin Albrecht <m...@informatik.uni-bremen.de>
> > wrote:
> >> > I will gladly contribute, but I'm guessing that my code will be pretty
> >> > obsolete when the pbc library is ported into SAGE...
>
> >> Is anyone working on this?
>
> > Not that I am aware of. The issue last time this came up was that pbc
> > switched to GPL V2 around the middle of 2008.
>
> Do you mean GPLV2 as opposed to GPLV2+?  Is that what the issue is?
> I.e., is it GPLV3 incompatible?

Nope, I made a typo in the initial email. As I pointed out in my
correction above it is GPL V3 - sorry for causing confusion.

Cheers,

Michael

[David Møller Hansen]

I agree with the teaching and research justifying a slow
implementation.

But in the not so distant future when every one is making Cython
wrappers of their c code and placing their code in SAGE won't it then
be a problem with duplication of functionality?
Couldn't a lot of similar functions "water down" the whole quality of
SAGE?

Is there a defined group of people to join on getting the pbc library
into SAGE?

/David

[David Møller Hansen]

I agree with the teaching and research justifying a slow
implementation.

But in the not so distant future when every one is making Cython
wrappers of their c code and placing their code in SAGE won't it then
be a problem with duplication of functionality?
Couldn't a lot of similar functions "water down" the whole quality of
SAGE?

Is there a defined group of people to join on getting the pbc library
into SAGE?

/David

[Martin Albrecht]

On Friday 14 November 2008, David Møller Hansen wrote:
> I agree with the teaching and research justifying a slow
> implementation.
>
> But in the not so distant future when every one is making Cython
> wrappers of their c code and placing their code in SAGE won't it then
> be a problem with duplication of functionality?

Sure, that could be a problem but can be dealt with on a case to case basis.

> Couldn't a lot of similar functions "water down" the whole quality of
> SAGE?

As long as we have one really good implementation, I don't see an issue.

> Is there a defined group of people to join on getting the pbc library
> into SAGE?

I am not aware of anybody working on this. Do you want to start such a
project?

Also, note that these days new packages have to be voted in to be included
with Sage. See also:

http://www.sagemath.org/doc/prog/node41.html

Cheers,
Martin

--
name: Martin Albrecht
_pgp: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x8EF0DC99
_www: http://www.informatik.uni-bremen.de/~malb

_jab: martinr...@jabber.ccc.de

[David Møller Hansen]

I would really like to start such a project.

But I wouldn't know the first thing on how to do so..

- Think I'll start by reading up on Sage dev. guide.

Do you usually keep track of projects here in the sage-devel group?

On 14 Nov., 13:39, Martin Albrecht <m...@informatik.uni-bremen.de>
wrote:

> _jab: martinralbre...@jabber.ccc.de

[Burcin Erocal]

On Fri, 14 Nov 2008 05:38:03 -0800 (PST)

David Møller Hansen <da...@mollerhansen.com> wrote:

>
> I would really like to start such a project.
>
> But I wouldn't know the first thing on how to do so..
>
> - Think I'll start by reading up on Sage dev. guide.
>
> Do you usually keep track of projects here in the sage-devel group?

You can get a trac account, and create an entry in trac about your
project.

http://wiki.sagemath.org/TracGuidelines

As a first step in writing such a wrapper, you should create an spkg
that installs the library in the sage environment. (i.e., with
prefix=SAGE_LOCAL)

Look in the directories under sage/interfaces/ for examples of how to go
about calling c/c++ libraries, you will also need to write some python
code to make these functions easily available to the user. I suppose
this would be in sage/crypto/.

Cheers,

Burcin

[Martin Albrecht]

On Friday 14 November 2008, David Møller Hansen wrote:
> I would really like to start such a project.

Great!

> But I wouldn't know the first thing on how to do so..

I suppose the first thing to do, is to look through the list of requirements
and check the PBC library if it qualifies. It can still be an optional
package if it doesn't qualify for inclusion in the standard library.

Then you could build an SPKG for PBC and later a Cython binding to PBC. When
you have a prototype it is time to call for a vote on PBC on [sage-devel].

> - Think I'll start by reading up on Sage dev. guide.

Please let us know about any issues you encounter, e.g. stuff that is not
documented, unclear etc.

> Do you usually keep track of projects here in the sage-devel group?

I am not quite sure what you're asking. But if people are working on a project
for Sage they usually discuss stuff here.

Cheers,
Martin

--
name: Martin Albrecht
_pgp: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x8EF0DC99
_www: http://www.informatik.uni-bremen.de/~malb

_jab: martinr...@jabber.ccc.de

[Martin Albrecht]

On Friday 14 November 2008, Burcin Erocal wrote:
> On Fri, 14 Nov 2008 05:38:03 -0800 (PST)
>
> David Møller Hansen <da...@mollerhansen.com> wrote:
> > I would really like to start such a project.
> >
> > But I wouldn't know the first thing on how to do so..
> >
> > - Think I'll start by reading up on Sage dev. guide.
> >
> > Do you usually keep track of projects here in the sage-devel group?
>
> You can get a trac account, and create an entry in trac about your
> project.
>
> http://wiki.sagemath.org/TracGuidelines
>
> As a first step in writing such a wrapper, you should create an spkg
> that installs the library in the sage environment. (i.e., with
> prefix=SAGE_LOCAL)
>
> Look in the directories under sage/interfaces/ for examples of how to go
> about calling c/c++ libraries,

Did you mean sage/libs? sage/interfaces is for pexpect interfaces while
sage/libs has some of the gory details for C/C++ integration.

> you will also need to write some python
> code to make these functions easily available to the user. I suppose
> this would be in sage/crypto/.

Or in the schemes/elliptic_curve subdirectory?

Cheers,
Martin

--
name: Martin Albrecht
_pgp: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x8EF0DC99
_www: http://www.informatik.uni-bremen.de/~malb

_jab: martinr...@jabber.ccc.de

[William Stein]

On Fri, Nov 14, 2008 at 6:05 AM, Martin Albrecht
<ma...@informatik.uni-bremen.de> wrote:
>
> On Friday 14 November 2008, David Møller Hansen wrote:
>> I would really like to start such a project.
>
> Great!
>
>> But I wouldn't know the first thing on how to do so..
>
> I suppose the first thing to do, is to look through the list of requirements
> and check the PBC library if it qualifies. It can still be an optional
> package if it doesn't qualify for inclusion in the standard library.

Just for the record, Michael Abshoff claimed that PBC does *not*
qualify due to them
switching to GPL-v3 only. So It would have to be an optional package.

> Then you could build an SPKG for PBC and later a Cython binding to PBC. When
> you have a prototype it is time to call for a vote on PBC on [sage-devel].
>
>> - Think I'll start by reading up on Sage dev. guide.
>
> Please let us know about any issues you encounter, e.g. stuff that is not
> documented, unclear etc.
>
>> Do you usually keep track of projects here in the sage-devel group?
>
> I am not quite sure what you're asking. But if people are working on a project
> for Sage they usually discuss stuff here.
>
> Cheers,
> Martin
>
> --
> name: Martin Albrecht
> _pgp: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x8EF0DC99
> _www: http://www.informatik.uni-bremen.de/~malb
> _jab: martinr...@jabber.ccc.de
>
>
> >
>

[Martin Albrecht]

On Friday 14 November 2008, William Stein wrote:
> On Fri, Nov 14, 2008 at 6:05 AM, Martin Albrecht
>
> <ma...@informatik.uni-bremen.de> wrote:
> > On Friday 14 November 2008, David Møller Hansen wrote:
> >> I would really like to start such a project.
> >
> > Great!
> >
> >> But I wouldn't know the first thing on how to do so..
> >
> > I suppose the first thing to do, is to look through the list of
> > requirements and check the PBC library if it qualifies. It can still be
> > an optional package if it doesn't qualify for inclusion in the standard
> > library.
>
> Just for the record, Michael Abshoff claimed that PBC does *not*
> qualify due to them
> switching to GPL-v3 only. So It would have to be an optional package.

He is right, I just checked. COPYING is GPLv3 and there are no copyright
statements in the source files, so COPYING definitely applies. I suppose it
is worth a shot to try to convince the authors to consider relicensing?

[William Stein]

On Fri, Nov 14, 2008 at 6:19 AM, Martin Albrecht

<ma...@informatik.uni-bremen.de> wrote:
>
> On Friday 14 November 2008, William Stein wrote:
>> On Fri, Nov 14, 2008 at 6:05 AM, Martin Albrecht
>>
>> <ma...@informatik.uni-bremen.de> wrote:
>> > On Friday 14 November 2008, David Møller Hansen wrote:
>> >> I would really like to start such a project.
>> >
>> > Great!
>> >
>> >> But I wouldn't know the first thing on how to do so..
>> >
>> > I suppose the first thing to do, is to look through the list of
>> > requirements and check the PBC library if it qualifies. It can still be
>> > an optional package if it doesn't qualify for inclusion in the standard
>> > library.
>>
>> Just for the record, Michael Abshoff claimed that PBC does *not*
>> qualify due to them
>> switching to GPL-v3 only. So It would have to be an optional package.
>
> He is right, I just checked. COPYING is GPLv3 and there are no copyright
> statements in the source files, so COPYING definitely applies. I suppose it
> is worth a shot to try to convince the authors to consider relicensing?

Yes, definitely. Alternatively, we can fork their last GPLv2 version.
(Just kidding! -- I've never even looked at the project.)

William

[David Møller Hansen]

Yeah well I guess that the project would be started as an optional
package at first and then if the code is licensed under GPLv3 it would
be, I would guess straight forward to get it into Sage.

I'll try to give some comments on the dev. guide after reading it.

/David


On 14 Nov., 15:05, Martin Albrecht <m...@informatik.uni-bremen.de>
wrote:

> _jab: martinralbre...@jabber.ccc.de

[Burcin Erocal]

On Fri, 14 Nov 2008 14:10:07 +0000
Martin Albrecht <ma...@informatik.uni-bremen.de> wrote:

>
> On Friday 14 November 2008, Burcin Erocal wrote:
> > On Fri, 14 Nov 2008 05:38:03 -0800 (PST)
> >
> > David Møller Hansen <da...@mollerhansen.com> wrote:
> > > I would really like to start such a project.
> > >
> > > But I wouldn't know the first thing on how to do so..
> > >
> > > - Think I'll start by reading up on Sage dev. guide.
> > >
> > > Do you usually keep track of projects here in the sage-devel
> > > group?
> >
> > You can get a trac account, and create an entry in trac about your
> > project.
> >
> > http://wiki.sagemath.org/TracGuidelines
> >
> > As a first step in writing such a wrapper, you should create an spkg
> > that installs the library in the sage environment. (i.e., with
> > prefix=SAGE_LOCAL)
> >
> > Look in the directories under sage/interfaces/ for examples of how
> > to go about calling c/c++ libraries,
>
> Did you mean sage/libs? sage/interfaces is for pexpect interfaces
> while sage/libs has some of the gory details for C/C++ integration.

Yes, I meant libs, and managed to write interfaces. Thanks for
the correction.

> > you will also need to write some python
> > code to make these functions easily available to the user. I suppose
> > this would be in sage/crypto/.
>
> Or in the schemes/elliptic_curve subdirectory?

I don't have any clue how a user interface to pairing should be. I just
wanted to point out that the real code is not in the libs directory.

I guess I should check my sleep deprivation levels before hitting the
send button next time. :)

Cheers,

Burcin

[David Møller Hansen]

The pairing is 'just' a bilinear mapping of two point in a curve group
into a "third" group, type depending on what pairing.

I'm wondering where pairings should go in Sage structure?

- any comments on that?

[Martin Albrecht]

On Saturday 15 November 2008, David Møller Hansen wrote:
> The pairing is 'just' a bilinear mapping of two point in a curve group
> into a "third" group, type depending on what pairing.
>
> I'm wondering where pairings should go in Sage structure?
>
> - any comments on that?

WWMD (What Would Magma Do) ;-)

http://magma.maths.usyd.edu.au/magma/htmlhelp/text1285.htm

so how about:

sage: E = EllipticCurve(...)
sage: P = E.random_element()
sage: Q = E.random_element()
sage: _ = P.tate_pairing(Q,n)

I don't work with these objects, so this might be the least desirable
solution.

Cheers,
Martin

--
name: Martin Albrecht
_pgp: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x8EF0DC99
_www: http://www.informatik.uni-bremen.de/~malb

_jab: martinr...@jabber.ccc.de

[John Cremona]

2008/11/15 Martin Albrecht <ma...@informatik.uni-bremen.de>:

>
> On Saturday 15 November 2008, David Møller Hansen wrote:
>> The pairing is 'just' a bilinear mapping of two point in a curve group
>> into a "third" group, type depending on what pairing.
>>
>> I'm wondering where pairings should go in Sage structure?
>>
>> - any comments on that?
>
> WWMD (What Would Magma Do) ;-)
>
> http://magma.maths.usyd.edu.au/magma/htmlhelp/text1285.htm
>
> so how about:
>
> sage: E = EllipticCurve(...)
> sage: P = E.random_element()
> sage: Q = E.random_element()
> sage: _ = P.tate_pairing(Q,n)
>
> I don't work with these objects, so this might be the least desirable
> solution.

That is the sort of thing I would do, but my use of pairings is not
necessarily similar to those using them in crypto. I would also
expect E.*_pairing(P,Q,n) to work, and for n to be optional with a
sensible default (depending on which pairing), say the order of P. So
if P and Q are both in E[m] then E.weil_pairing(P,Q) and need not
mention m.

John

[mabshoff]

On Nov 15, 4:50 am, Martin Albrecht <m...@informatik.uni-bremen.de>
wrote:

<SNIP>

> WWMD (What Would Magma Do) ;-)

Excellent. Have you just posted the T shirt slogan fro SD 12?

> Cheers,
> Martin

Cheers,

Michael

> --
> name: Martin Albrecht
> _pgp:http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x8EF0DC99
> _www:http://www.informatik.uni-bremen.de/~malb

> _jab: martinralbre...@jabber.ccc.de

[David Møller Hansen]

I agree John.

It could be really cool and intuitive if the pairing also was
accessible from the curve group E(F) with the whole torsion of m in
it.

In this way points P,Q would intuitively be expected to come from E
(F).

Wow.. Magma have a lot of pairings implemented..

Think I got my answer, thanks guys!

By the way..
Do not know if anyone is interested in flinging some critique my way,
I have posted my small Sage implementation of the Weil pairing and BLS
short signature scheme here under "files": http://mollerhansen.com

/David


On 15 Nov., 14:00, "John Cremona" <john.crem...@gmail.com> wrote:
> 2008/11/15 Martin Albrecht <m...@informatik.uni-bremen.de>:

> > _jab: martinralbre...@jabber.ccc.de