RE: [ExcelDna] Excel-DNA CUDA

173 views
Skip to first unread message

Govert van Drimmelen

unread,
Mar 9, 2022, 4:09:07 AM3/9/22
to exce...@googlegroups.com

--------------------------------------------------

Excel-DNA is now registered on GitHub Sponsors.

Your monthly contribution will encourage further development and support.

--------------------------------------------------

 

It’s not an area I have delved into myself, but I can at least give some pointers.

 

You might think of two issues as quite orthogonal, but complementary:

  • Doing machine learning and other computation-intensive work in .NET
  • Using Excel as a front-end interface for managing such processing, viewing inputs and results etc.

 

While not at the level of Python, .NET has good integration with various machine learning and related libraries. So you might search for and explore this on its own, without only looking for resource which also have Excel in the mix. One might make some console applications to just see how to use and interact with the computation libraries.

You might have a look at

 

Then exposing this functionality to Excel through an Excel-DNA add-in makes for a very powerful and user friendly interface. That part is not really specific to the machine learning application area, but depends on your aesthetic feeling about how stuff should work from Excel. Some people want Excel to be purely a reporting canvas for showing results, others take a more interactive approach, where Excel is used to define a machine learning pipeline.

 

As you can imagine, most of what people do in this area is proprietary work done in a commercial context.

 

One set of open source educational material that fits this topic is from Bahrudin Hrnjica – see bhrnjica (Bahrudin Hrnjica) (github.com)

In particular, he has some .NET based machine learning tools which also have .NET add-ins, under a permissive open source license.

See bhrnjica/anndotnet: ANNdotNET - deep learning tool on .NET Platform. (github.com) as a good starting point.

 

Maybe other users can share some experience or add some references to the list.

 

-Govert

 

 

 

From: exce...@googlegroups.com <exce...@googlegroups.com> On Behalf Of audeser servicios
Sent: 9 March 2022 10:27
To: Excel-DNA <exce...@googlegroups.com>
Subject: [ExcelDna] Excel-DNA CUDA

 

Is this even possible? Any hint on where to look?

 

If possible it could leverage Machine Learning on Excel a LOT, because training the sets over VBA code its quite time consuming.

 

Best regards

--
You received this message because you are subscribed to the Google Groups "Excel-DNA" group.
To unsubscribe from this group and stop receiving emails from it, send an email to exceldna+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/exceldna/3dfbacff-11d6-47fc-a46b-dc371f9e8864n%40googlegroups.com.

Message has been deleted

Govert van Drimmelen

unread,
Mar 12, 2022, 1:49:20 AM3/12/22
to exce...@googlegroups.com

By ML.NET I refer to this initiative from Microsoft: ML.NET | Machine Learning made for .NET (microsoft.com)

 

As I tried to explain in my message, I believe Excel is a good ‘front-end’ for interacting and driving machine learning libraries – kind of an alternative to the Jupyter Notebooks so commonly used. But I don’t have nice examples for this yet.

 

There are different wrappers for OpenCV into a .NET library – I believe the most Popular is EmguCV: Emgu CV: OpenCV in .NET (C#, VB, C++ and more)

 

User-defined types can be defined and used inside your .NET library, but these types are not understood in the Excel sheet. So any functions you make would need to accept Excel-compatible data types, then build up or convert to your UDTs, and vice versa to return to the sheet.

 

-Govert

 

 

From: exce...@googlegroups.com <exce...@googlegroups.com> On Behalf Of audeser servicios
Sent: 10 March 2022 07:06
To: Excel-DNA <exce...@googlegroups.com>
Subject: Re: [ExcelDna] Excel-DNA CUDA

 

Thank you for your detailed guidance.

 

I'm conscientious of this little nightmare, if to be done on Excel...

 

Until I get your guide, I thought that Microsoft ML.NET was given as a "site address", but when I did the search, "site" was down and could find nothing under archive.org... uhmmm! I blame MSFT for ditching technologies as fast as Google does. XDDD, better shut up.

 

For what I can say, there is not a lot of extensive work around ML on Excel (VBA), as the script programming language that has taken the role on this is clearly Python (some JS also; you can't tell how it will do this language on the future as web development related to JS is occupying a lot of niches). It does not seem this gonna change, as ML is evolving quite fast and with Excel+VBA pace they will no get on this on time...

I have a post on my own blog about NN, where tried to resume all the info I found on the web on this topic (and for Excel mainly) for future guidance. There are some links, if needed I'll post them here.

 

For basic ML level (classification problems), Excel has nice features that do not require learning a new language, and can be programmed inside. Other ML arquitectures may not be that easy. Transformers are out of the reach...

Given said that, David Bots and Mike Pallister amazed me some years ago; but I must say that Richard Maddison is the king on the room, has some nice implementations on CovNet and CapsuleNet entirely on Excel, and the UI is quite complex. The hell on these developments is with the training, several days of computations for very basic stuff, but considering that they were using only Excel "formulas", not that bad.

 

I also had developed some implementations on VBA for the topic, based on some spanish youtube video and info from Richard videos and website, to get better numbers, and ease the NN development on VBA -avoiding the closed commercial add-ins, I want to learn, not to produce-. Multiperceptron arquitectures are "quite" simple, and the backpropagation algorithm is easy to implement once understood (*), as it's only matrix multiplication operations and it's generalized on AI. But as I reach the point to operate with images to extract information, it will be impossible to scale up: VBA is just too slow for this tasks of image data manipulation. Even matrix multiplication is sluggish if dealing with big matrices, and thousand of rounds.

 

So, following my investigations on this .XLL thing, yesterday I did a simple performance test for a Matrix multiplication, VBA vs .XLL version. If the code has to perform a dynamic ReDim, VBA easy overcome .XLL on my tests (but that could be pointed at me, because I don't know yet how to make ReDim inside Vb.NET in a fashion way); if matrix sizes are static, the gain of performance of XLL is over 5 times (+500%... very nice figure). My last search on an easy to get performance was a "VBA vs VB5 compiled DLL" and VBA was faster than the DLL. So, right now I have discarded the VB5 compile path as XLL is clearly the fastest horse on this race.

 

Going back to CUDA topic, if the use of the GPU for computation were a possibility, as Python can achieve, it could be attained a HUGE boost on performance; did a search for CUDA on the G Group with no luck, so I opened the question with that interest. But then realized (from a post of you in this group to other user) that XLL greately operates on .NET and if a C++ project is needed, maybe XLW is better option (but that was on 2013). So deleted the post until I get more clear on this point.

 

In the meanwhile I tried to find that CUDA documents showing how to operate CUDA on Excel (the ones on NVIDIA site). And succeed, finding 2 -lets call them "kits"-, one for Excel 2007 and the other one for Excel 2010, and an archived file from the nVIDIA CodePlex site. Open any of the "kits", and you will find -among others- a XLSM file, and a XLL file, with a VS project. Have not time to put my hands on them yet, but it looks as it's possible (as they are official NVIDIA documents), have to learn more about XLL. There are other several pages (counted with the fingers of one hand) on the net, talking about Excel and CUDA, but could not get a word from the original posters when asked via email, so discarded. There is also 2 GitHub repositories on this Excel+CUDA... but I don't get myself to manage this GitHub "texty thing", I need some more visual guidance on the process, so I give up on this repository, although I may look at this other with nicer eyes.

 

Another topic I should dig on: How to get to work OpenCV with Excel (VBA)?. OpenCV functions are incompatible with VBA, and will need a wrapper for this library. Would be easy to create that kind of wrapper?, don't know, never did such a thing, but I found some promising posts (**) detailing how to create a wrapper to a C++ library using Excel-DNA... another item to investigate further on.

 

And, if permitted, I have one last topic... UDT (user defined Types) with Excel-DNA. Any suggestion?

 

 

Best regards

Ricardo Sousa Rodrigues

unread,
Mar 12, 2022, 3:18:39 AM3/12/22
to exce...@googlegroups.com
Hi there

There's Excel + Excel DNA + Libraries (. Net or C++).
So your focus should be how to make CUDA inside a C# dll and just include on the DNA project. Just that ;-)

Thanks
Ricardo 

On Sat, Mar 12, 2022, 06:38 audeser servicios <aud...@gmail.com> wrote:
Thank you for your detailed guidance.

I'm conscientious of this little nightmare, if to be done on Excel...

Until I get your guide, I thought that Microsoft ML.NET was given as a "site address", but when I did the search, "site" was down and could find nothing under archive.org... uhmmm! I blame MSFT for ditching technologies as fast as Google does. XDDD, better shut up.

For what I can say, there is not a lot of extensive work around ML on Excel (VBA), as the script programming language that has taken the role on this is clearly Python (some JS also; you can't tell how it will do this language on the future as web development related to JS is occupying a lot of niches). It does not seem this gonna change, as ML is evolving quite fast and with Excel+VBA pace they will no get on this on time...
I have a post on my own blog about NN, where tried to resume all the info I found on the web on this topic (and for Excel mainly) for future guidance. There are some links, if needed I'll post them here.

For basic ML level (classification problems), Excel has nice features that do not require learning a new language, and can be programmed inside. Other ML arquitectures may not be that easy. Transformers are out of the reach...
Given said that, David Bots and Mike Pallister amazed me some years ago; but I must say that Richard Maddison is the king on the room, has some nice implementations on CovNet and CapsuleNet entirely on Excel, and the UI is quite complex. The hell on these developments is with the training, several days of computations for very basic stuff, but considering that they were using only Excel "formulas", not that bad.

I also had developed some implementations on VBA for the topic, based on some spanish youtube video and info from Richard videos and website, to get better numbers, and ease the NN development on VBA -avoiding the closed commercial add-ins, I want to learn, not to produce-. Multiperceptron arquitectures are "quite" simple, and the backpropagation algorithm is easy to implement once understood (*), as it's only matrix multiplication operations and it's generalized on AI. But as I reach the point to operate with images to extract information, it will be impossible to scale up: VBA is just too slow for this tasks of image data manipulation. Even matrix multiplication is sluggish if dealing with big matrices, and thousand of rounds.

So, following my investigations on this .XLL thing, yesterday I did a simple performance test for a Matrix multiplication, VBA vs .XLL version. If the code has to perform a dynamic ReDim, VBA easy overcome .XLL on my tests (but that could be pointed at me, because I don't know yet how to make ReDim inside Vb.NET in a fashion way); if matrix sizes are static, the gain of performance of XLL is over 5 times (+500%... very nice figure). My last search on an easy to get performance was a "VBA vs VB5 compiled DLL" and VBA was faster than the DLL. So, right now I have discarded the VB5 compile path as XLL is clearly the fastest horse on this race.

Going back to CUDA topic, if the use of the GPU for computation were a possibility, as Python can achieve, it could be attained a HUGE boost on performance; did a search for CUDA on the G Group with no luck, so I opened the question with that interest. But then realized (from a post of you in this group to other user) that XLL greately operates on .NET and if a C++ project is needed, maybe XLW is better option (but that was on 2013). So deleted the post until I get more clear on this point.

In the meanwhile I tried to find that CUDA documents showing how to operate CUDA on Excel (the ones on NVIDIA site). And succeed, finding 2 -lets call them "kits"-, one for Excel 2007 and the other one for Excel 2010, and an archived file from the nVIDIA CodePlex site. Open any of the "kits", and you will find -among others- a XLSM file, and a XLL file, with a VS project. Have not time to put my hands on them yet, but it looks as it's possible (as they are official NVIDIA documents), have to learn more about XLL. There are other several pages (counted with the fingers of one hand) on the net, talking about Excel and CUDA, but could not get a word from the original posters when asked via email, so discarded. There is also 2 GitHub repositories on this Excel+CUDA... but I don't get myself to manage this GitHub "texty thing", I need some more visual guidance on the process, so I give up on this repository, although I may look at this other with nicer eyes.

Another topic I should dig on: How to get to work OpenCV with Excel (VBA)?. OpenCV functions are incompatible with VBA, and will need a wrapper for this library. Would be easy to create that kind of wrapper?, don't know, never did such a thing, but I found some promising posts (**) detailing how to create a wrapper to a C++ library using Excel-DNA... another item to investigate further on.

And, if permitted, I have one last topic... UDT (user defined Types) with Excel-DNA. Any suggestion?


Best regards
El miércoles, 9 de marzo de 2022 a las 10:09:07 UTC+1, Govert van Drimmelen escribió:
To view this discussion on the web visit https://groups.google.com/d/msgid/exceldna/6d02d1b4-c945-4c31-9df0-1fba215bf10dn%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages