MPIR adapter created for .Net

[Alex Dyachenko]

Hello MPIR team,

In my fork, I have created Visual Studio solutions (for versions 2012, 2013, and 2015) to build an adapter using C++/CLI that exposes MPIR for consumption by .Net 4.5+ languages.  The basic idea is that first you build the LIB version of MPIR using one of the existing VS solutions, for your desired processor architecture.  Then you build my solution that links to that LIB and produces a .Net assembly which exposes .Net classes for ints, rationals, and floats, and forwards the methods and overloaded operators on those classes to native MPIR calls.  Similarly to the native C++ interface, most operations are implemented through overloaded operators that produce expressions with deferred-to-assignment evaluation, such that a single operation (e.g. a = b + c) results in a single MPIR call (mp*_add), while more complicated expressions will use temporaries as necessary.  (The actual syntax would be a.Value = b + c; the reason for that is that .Net does not support overloaded assignment).  This is not just a set of pinvoke signatures, but a full OO interface.

This effort is now complete with a full suite of unit tests and XML comments on the entire public interface (which feed nicely into Visual Studio intellisense), tested under both Win32 and Win64, and can of course be linked to any processor-specific assembly-optimized build of MPIR.  I had to make one small refactoring to the existing MPIR code in the raw IO area where the existing entry point was at too high a level to be reused, and I split a method in two so I could use my own memory allocation and call into the "meat" of the raw IO.  This resulted in a couple of new entry points being added to mpir.h but no existing signatures were changed.  I could have re-implemented this method entirely in my code to avoid making any changes to the MPIR proper, but that wouldn't have been in the spirit of writing an adapter whose internals are all forwarded.

I would like to create a pull request to have this addition code reviewed and eventually merged in.  I realize this may take a while, and I'll be happy to answer your questions and/or tweak the implementation based on your input.  I'm also willing to support it in future MPIR releases.  Before I do the pull request, I wanted to post this as a heads up and to get initial feedback from you as to whether you're interested in general.

Also I would like to write up a chapter for the manual, which will probably be similar in size and structure to the C++ interface chapter, however I'm not sure what tool chain you're using for that.  I have a linux VM where I'm able to edit texi sources in the doc folder and produce a PDF from them, but if I understand correctly these are dual-purpose sources that also work in some other tool that perhaps parses out the special texi instructions that don't seem to do anything for PDF output, and I wouldn't want to break that.  If you let me know the official way to edit and test the documentation, I can get started.

Cheers,

Alex

[Bill Hart]

Hi Alex,

On 15 January 2016 at 17:44, Alex Dyachenko <ady...@gmail.com> wrote:

Hello MPIR team,

In my fork, I have created Visual Studio solutions (for versions 2012, 2013, and 2015) to build an adapter using C++/CLI that exposes MPIR for consumption by .Net 4.5+ languages.  The basic idea is that first you build the LIB version of MPIR using one of the existing VS solutions, for your desired processor architecture.  Then you build my solution that links to that LIB and produces a .Net assembly which exposes .Net classes for ints, rationals, and floats, and forwards the methods and overloaded operators on those classes to native MPIR calls.  Similarly to the native C++ interface, most operations are implemented through overloaded operators that produce expressions with deferred-to-assignment evaluation, such that a single operation (e.g. a = b + c) results in a single MPIR call (mp*_add), while more complicated expressions will use temporaries as necessary.  (The actual syntax would be a.Value = b + c; the reason for that is that .Net does not support overloaded assignment).  This is not just a set of pinvoke signatures, but a full OO interface.

This sounds really great. I'm mildly surprised to hear that something like this didn't already exist for GMP. I know Peter Luschny has a GMP wrapper for C#, though I don't believe he uses delayed evaluation.

 

This effort is now complete with a full suite of unit tests and XML comments on the entire public interface (which feed nicely into Visual Studio intellisense), tested under both Win32 and Win64, and can of course be linked to any processor-specific assembly-optimized build of MPIR.  I had to make one small refactoring to the existing MPIR code in the raw IO area where the existing entry point was at too high a level to be reused, and I split a method in two so I could use my own memory allocation and call into the "meat" of the raw IO.  This resulted in a couple of new entry points being added to mpir.h but no existing signatures were changed.  I could have re-implemented this method entirely in my code to avoid making any changes to the MPIR proper, but that wouldn't have been in the spirit of writing an adapter whose internals are all forwarded.

I would like to create a pull request to have this addition code reviewed and eventually merged in.  I realize this may take a while, and I'll be happy to answer your questions and/or tweak the implementation based on your input.  I'm also willing to support it in future MPIR releases.  Before I do the pull request, I wanted to post this as a heads up and to get initial feedback from you as to whether you're interested in general.

We will definitely take a look at this. It sounds like a very valuable addition to the project. I'm not clear whether you only want to contribute the diff to MPIR or the whole project to MPIR. The former is probably better from your point of view, since you will get more credit for it.

Of course the question remains as to whether your code can be made to work on Linux too. Now that MS has made C# open source (or is it just the CLR implementation, I forget), it is now possible to run such code on Linux and other OS's.

But Visual Studio assumes Windows, no?

 

Also I would like to write up a chapter for the manual, which will probably be similar in size and structure to the C++ interface chapter, however I'm not sure what tool chain you're using for that.  I have a linux VM where I'm able to edit texi sources in the doc folder and produce a PDF from them, but if I understand correctly these are dual-purpose sources that also work in some other tool that perhaps parses out the special texi instructions that don't seem to do anything for PDF output, and I wouldn't want to break that.  If you let me know the official way to edit and test the documentation, I can get started.

I don't use a particular tool. I figured out what format the table of contents options use and just do it by hand. Silly, I know, but it's not a bottleneck in my day-to-day work routine. I rarely add new sections to the manual. I have no idea what tool free software people use. Probably some emacs module.

As for including a section in the manual, this doesn't make sense if you are only contributing the diff to MPIR and not the whole interface. In that case I recommend writing a separate manual for your project. These days I don't think you can do much better than Sphinx documentation.

Of course you are more than welcome to add a small section to the MPIR manual providing the URL of your project and a paragraph explaining what it does.

I'll leave it to you to decide whether you want to contribute the whole thing to MPIR. Of course I'd love it if you did, and provided Brian doesn't object that would be fine, but I can't help thinking you'd get less recognition for your project if you did that. Whilst we try to do as much as we can to promote individual authors, people ultimately end up associating large projects like MPIR with very few people. It's worth thinking about carefully anyway.

We will definitely provide a link to your project on the website and in our documentation at the very least. I'm really glad this exists.

Bill.

Cheers,

Alex

--
You received this message because you are subscribed to the Google Groups "mpir-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mpir-devel+...@googlegroups.com.
To post to this group, send email to mpir-...@googlegroups.com.
Visit this group at https://groups.google.com/group/mpir-devel.
For more options, visit https://groups.google.com/d/optout.

[Bill Hart]

To test the documentation you have to be on Linux (or Cygwin).

makeinfo mpir.texi

make pdf

I believe (but am not sure) that it's all part of the texinfo system, which is a GNU project. As such I'm sure it is completely portable and has been designed from the ground up to be very easy to use on Windows, Mac and other proprietary systems.

Bill.

[Alex Dyachenko]

At the time I started almost two years ago, there was a much simpler wrapper or two, searching now yields a couple, but they are both for integers only and don't use deferred execution.  At any rate, there isn't one that's officially part of MPIR, so I thought I'd make one.

Visual Studio does assume Windows; I haven't done any work in c# on Linux so that is indeed a very good question.  Since my interop is written in C++/CLI, it's most likely a no-go for Linux, but according to http://tirania.org/blog/archive/2011/Dec-19.html Mono on Linux can already consume C++ directly so I'm guessing it can just use the native MPIR C++ interface.  Mine was an effort to open MPIR to Windows .Net developers.

On Linux, "makeinfo mpir.texi" runs without error, but "make pdf" gets me a "No rule to make target 'pdf'".  However I can use "texi2dvi --pdf mpir.texi" to create a PDF; is that the same thing?

I'm confused about diff vs. whole project.  I was envisioning that my files, 95% of which are new and a handful existing MPIR files slightly modified, would become part of your repo just like Brian's VS builds now are.  Then anyone downloading an MPIR release sources from mpir.org would have the option, if they like, to build my adapter and consume it.  The other way this can work, and they can do it today, is if they clone my own fork, but I wonder if anyone in the world besides the readers of this thread knows about it.

[Bill Hart]

On 15 January 2016 at 21:01, Alex Dyachenko <ady...@gmail.com> wrote:

At the time I started almost two years ago, there was a much simpler wrapper or two, searching now yields a couple, but they are both for integers only and don't use deferred execution.  At any rate, there isn't one that's officially part of MPIR, so I thought I'd make one.

Great.

 

Visual Studio does assume Windows; I haven't done any work in c# on Linux so that is indeed a very good question.  Since my interop is written in C++/CLI, it's most likely a no-go for Linux, but according to http://tirania.org/blog/archive/2011/Dec-19.html Mono on Linux can already consume C++ directly so I'm guessing it can just use the native MPIR C++ interface.  Mine was an effort to open MPIR to Windows .Net developers.

Yeah I don't know the technical details. But MS basically open sourced a lot of their own stuff, independently of Mono, and most of it works on Linux now I think.

 

On Linux, "makeinfo mpir.texi" runs without error, but "make pdf" gets me a "No rule to make target 'pdf'".  

You have to do that in the doc directory, as with the other command.

 

However I can use "texi2dvi --pdf mpir.texi" to create a PDF; is that the same thing?

I don't know.

 

I'm confused about diff vs. whole project.  I was envisioning that my files, 95% of which are new and a handful existing MPIR files slightly modified, would become part of your repo just like Brian's VS builds now are.  

Cool.

 

Then anyone downloading an MPIR release sources from mpir.org would have the option, if they like, to build my adapter and consume it.  The other way this can work, and they can do it today, is if they clone my own fork, but I wonder if anyone in the world besides the readers of this thread knows about it.

In that case I would suggest also keeping a website somewhere for the project with your name on it prominently and a note saying that the interface is already part of MPIR version blah and later.

I'm personally happy for you to contribute this to the project, modulo us taking a look and making sure the code looks readable. But Brian will also need to agree. The fact that you have agreed to continue maintaining it will certainly make the decision whether we want to include this easier.

Bill.

[Brian Gladman]

On 15/01/2016 16:44, Alex Dyachenko wrote:

> Hello MPIR team,
>
> In my fork, I have created Visual Studio solutions (for versions 2012,
> 2013, and 2015) to build an adapter using C++/CLI that exposes MPIR for
> consumption by .Net 4.5+ languages. The basic idea is that first you
> build the LIB version of MPIR using one of the existing VS
> solutions, for your desired processor architecture. Then you build my
> solution that links to that LIB and produces a .Net assembly which
> exposes .Net classes for ints, rationals, and floats, and forwards the
> methods and overloaded operators on those classes to native MPIR calls.
> Similarly to the native C++ interface, most operations are implemented
> through overloaded operators that produce expressions with
> deferred-to-assignment evaluation, such that a single operation (e.g. a
> = b + c) results in a single MPIR call (mp*_add), while more complicated
> expressions will use temporaries as necessary. (The actual syntax would
> be a.Value = b + c; the reason for that is that .Net does not support
> overloaded assignment). This is not just a set of pinvoke signatures,
> but a full OO interface.

This sounds like a useful addition to MPIR as .Net development is very
popular on Windows.

> This effort is now complete with a full suite of unit tests and XML
> comments on the entire public interface (which feed nicely into Visual
> Studio intellisense), tested under both Win32 and Win64, and can of
> course be linked to any processor-specific assembly-optimized build of
> MPIR. I had to make one small refactoring to the existing MPIR code in
> the raw IO area where the existing entry point was at too high a level
> to be reused, and I split a method in two so I could use my own memory
> allocation and call into the "meat" of the raw IO. This resulted in a
> couple of new entry points being added to mpir.h but no existing
> signatures were changed. I could have re-implemented this method
> entirely in my code to avoid making any changes to the MPIR proper, but
> that wouldn't have been in the spirit of writing an adapter whose
> internals are all forwarded.
>
> I would like to create a pull request to have this addition code
> reviewed and eventually merged in. I realize this may take a while, and
> I'll be happy to answer your questions and/or tweak the implementation
> based on your input. I'm also willing to support it in future MPIR
> releases. Before I do the pull request, I wanted to post this as a
> heads up and to get initial feedback from you as to whether you're
> interested in general.

Your willingness to maintain and support this potential addition is very
important and counts as a big plus point for its inclusion. It would
also be nice to add a new dewveloper to the MPIR team!

> Also I would like to write up a chapter for the manual, which will
> probably be similar in size and structure to the C++ interface chapter,
> however I'm not sure what tool chain you're using for that. I have a
> linux VM where I'm able to edit texi sources in the doc folder and
> produce a PDF from them, but if I understand correctly these are
> dual-purpose sources that also work in some other tool that perhaps
> parses out the special texi instructions that don't seem to do anything
> for PDF output, and I wouldn't want to break that. If you let me know
> the official way to edit and test the documentation, I can get started.

This is a great proposal which will offer a worthwhile addition to MPIR.

If this raises any issues that need attention in my existing VS
solutions, I will be happy to take a look at these.

Brian

[Brian Gladman]

Can I assume that the changes you want in MPIR itself add extra calls
without changing the MPIR external interface as it is now. It would be
worth knowing a bit more about the changes you need here.

Do you envisage the addition of another directory witin the MPIR
directory at the same level as the build.vc and build.vc<nn>
directories? That is something like mpir/build.net

The way we are currently working is that Bill maintains the master MPIR
repository, whereas I maintain the Visual Studio build components as a
part of my own repository at https://github.com/BrianGladman/mpir

When Bill is ready to make a new release he pulls from my repository as
a part of the process. I imagine he would do the same with your
repository once your additions become a part of MPIR.

best regards,

Brian

[Bill Hart]

Hi Brian,

you probably don't get updates on pull requests to my repository, but you should be able to view the changes here [1] and comment on them.

To comment on the changes you can go to the "conversation" tab, and to see the actual changes to the files you can go to the "files changed" tab.

Bill.

[1] https://github.com/wbhart/mpir/pull/169

[Alex Dyachenko]

I added my actual source files into a new folder called mpir.net off the root of the repo.  I put the version-specific files (VS solution and project files) into a subfolder called mpir.net under each of the build.vcXX folders (excluding vc10), they all reference the same source files mentioned above.  However I also see the logic of creating my own build.vcXX subfolders within the main mpir.net, that may be a less confusing structure.  Let me know if you'd like me to arrange them that way.

Bill, you're correct that my modifications to MPIR itself were only to add a couple of additional calls without changing any of the existing interface.  This was so I could allocate my own scratch memory.  Would it help to separate those into their own pull request?

Alex

[Bill Hart]

On 17 January 2016 at 20:28, Alex Dyachenko <ady...@gmail.com> wrote:

I added my actual source files into a new folder called mpir.net off the root of the repo.  I put the version-specific files (VS solution and project files) into a subfolder called mpir.net under each of the build.vcXX folders (excluding vc10), they all reference the same source files mentioned above.  However I also see the logic of creating my own build.vcXX subfolders within the main mpir.net, that may be a less confusing structure.  Let me know if you'd like me to arrange them that way.

I think we are less likely to have accidental conflicts if we go for the second arrangement. Would you mind changing it.

I admit there is a disturbing lack of symmetry, but at least it won't be confusing to anyone.

 

Bill, you're correct that my modifications to MPIR itself were only to add a couple of additional calls without changing any of the existing interface.  This was so I could allocate my own scratch memory.  Would it help to separate those into their own pull request?

It doesn't worry me if they are left in the same pull request.

I will try to review your code as soon as I can.

Thanks,

Bill.

 

Alex

[Brian Gladman]

On 17/01/2016 19:28, Alex Dyachenko wrote:
> I added my actual source files into a new folder called mpir.net off the
> root of the repo. I put the version-specific files (VS solution and
> project files) into a subfolder called mpir.net under each of the
> build.vcXX folders (excluding vc10), they all reference the same source
> files mentioned above. However I also see the logic of creating my own
> build.vcXX subfolders within the main mpir.net, that may be a less
> confusing structure. Let me know if you'd like me to arrange them that way.

I think it will be easier if you use a build.net directory within the
mpir directory. This way people will immediately see where your build
files are located. Also, although conflicts are unlikely, it will be
easier of you manage the content of the build.net directory and I manage
the content of the build.vc and build.vc<nn> directoies.

best regards,

Brian

[Alex Dyachenko]

I have rearranged the project files as discussed.

Thanks for the positive feedback!

Alex

[Alex Dyachenko]

I have moved all the gmp-h.in changes to gmp-impl.h instead. gmp-h.in now stands unchanged, and I have added comments to the additions in gmp-impl.h, as well as in the out_raw.c and inp_raw.c sources to explain what was changed and why. Sorry, with the size of the pull request the diffs in these last two are not shown on github.

Now if I could just finish the documentation...  Have you decided if you would like a chapter in the MPIR manual or a separate document?

Alex

--
You received this message because you are subscribed to a topic in the Google Groups "mpir-devel" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/mpir-devel/WwzswnAMKSk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to mpir-devel+...@googlegroups.com.

[Bill Hart]

Hi Alex,

Thanks for doing this.

If there is a lot of documentation then I would prefer a separate document. By all means add a note in the main documentation explaining where to find the full documentation.

The reason for separating it like this is just for convenience of the user. They won't have to go looking through the manual to find what they are looking for.

Bill.

[Alex Dyachenko]

Just pushed the documentation update.  The new chapter is similar to and follows the C++ Class Interface chapter, but came out to about twice its length, 19 pages.  Can't blame it there, it's essentially an overview of all of MPIR's functions.  Should the manual link to the URL of the browsable version of the documentation (hosted in my fork's wiki)?

[Bill Hart]

Thanks Alex. Sorry it is going to take me a while to get to this. We are insanely busy at this time of year.

Yes, the manual can link to your wiki, so long as you consider that a permanent URL.

Bill.

You received this message because you are subscribed to the Google Groups "mpir-devel" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mpir-devel+...@googlegroups.com.