convert mwsize * to int *

[Tomas Schiler]

Hello,
I am looking for way how to convert mwsize *type to int * because i was using mex file writen for 32 bit matlab which was used to call function of MIP solver xpress do i cannot change type in these functions. Now the code doesn't work and i am would like to find simpliest posible solution to solve problem.
Thank you.

[Bruno Luong]

"Tomas Schiler" <ex...@atlas.cz> wrote in message <h9i819$t53$1...@fred.mathworks.com>...

> Hello,
> I am looking for way how to convert mwsize *type to int * because i was using mex file writen for 32 bit matlab which was used to call function of MIP solver xpress do i cannot change type in these functions. Now the code doesn't work and i am would like to find simpliest posible solution to solve problem.
> Thank you.

Huuh? mwSize IS actually defined as INT on 32 bit platform. Here is highlight of the definition header:

#ifdef MX_COMPAT_32
typedef int mwSize;
typedef int mwIndex;
typedef int mwSignedIndex;
#else
typedef size_t mwSize; /* unsigned pointer-width integer */
typedef size_t mwIndex; /* unsigned pointer-width integer */
typedef ptrdiff_t mwSignedIndex; /* a signed pointer-width integer */
#endif

Bruno

[Tomas Schiler]

I am sorry that I didn&#8217;t express my problem clear enough. Function is written for 32 bit system but I am using 64 bit system now.

"Bruno Luong" <b.l...@fogale.findmycountry> wrote in message <h9iat2$33d$1...@fred.mathworks.com>...

[James Tursa]

"Tomas Schiler" <ex...@atlas.cz> wrote in message <h9ig2u$kuj$1...@fred.mathworks.com>...

> I am sorry that I didn&#8217;t express my problem clear enough. Function is written for 32 bit system but I am using 64 bit system now.
>

I think you will need to provide more detail, or post some code, etc. to show us exactly what the issue is, and why simple type casts will not work.

James Tursa

[James Tursa]

"James Tursa" <aclassyguy_wi...@hotmail.com> wrote in message <h9inrd$ahr$1...@fred.mathworks.com>...

P.S. Basically, if you have a function that takes an int * as an argument, it is expecting an array of ints. If you have an array of mwSize on in the calling routine, and mwSize is different from an int, then you need to allocate an int array the same size as your mwSize array, copy the mwSize elements into the int elements, and then pass the pointer to the int array to the function, and when you are done then free the allocated int array. You can't convert the mwSize * itself to an int * and expect things to work by passing this pointer to the function, because the underlying data hasn't been converted.

James Tursa

[Dan Hensley]

Tomas Schiler wrote, On 9/25/2009 7:20 AM:
> I am sorry that I didn&#8217;t express my problem clear enough. Function is written for 32 bit system but I am using 64 bit system now.

You can't run 32bit object code from a 64bit executable. It sounds like
you have linked to 3rd party functions that are 32bit. If you only have
object code/library for these functions, you will not be able to use
them with 64bit. The only solution is to obtain 64bit versions of this
library, or if you are on Windows you can wrap the code in a 32bit COM
executable that you can call from 64bit Matlab using actxserver. This
is not a trivial exercise, as I can attest from experience. It works,
but is a tremendous amount of work.

Dan

[Thomas Meinguet]

I'd like to come back on this topic. This is not a problem of running a 32bit executable on a 64bit environment but a relevant issue for which a clean solution is indispensable. There are powerful precompiled external 64bit libraries whose input are int* pointers as well. The solution I see for the moment is to cast mwIndex (size_t) values into int in new vectors like this:

const mwIndex *MWrowInd = mxGetIr(prhs[0]);
const mwIndex *MWcolPtr = mxGetJc(prhs[0]);
const int nnz = (int) MWcolPtr[n];
rowInd = new int[nnz];
colPtr = new int[n+1];
for(int i=0; i<nnz; i++) { rowInd[i] = (int)(MWrowInd[i]); }
for(int i=0; i<=n; i++) { colPtr[i] = (int)(MWcolPtr[i]); }
//
Library_run(..., rowInd, colPtr ,...);

The number nnz of element of rowInd is equal to the number of non zero elements in the matrix prhs[0] here and can be high. The manual cast results in duplicating large data arrays in the memory, in a context where the developer wants to increase the performances ... just to cancel the unsigned feature of size_t.

There's no way to work around this using the -compatibleArrayDims / -largeArrayDims of the mex command as the C++ code to pass to mex can be itself generated with a external code generator that can say this - prior to the mex compilation :

error: argument of type "const mwIndex *" is incompatible with parameter of type "const int *"

Does someone have a better workaround for this error, not involving inefficient data duplication in the memory?

[James Tursa]

"Thomas Meinguet" <thomas....@skynet.be> wrote in message <kcbl88$de3$1...@newscl01ah.mathworks.com>...

General comment: If mwIndex is a size_t and you need an int, then your above looping code will still have potential problems if the value overflows an int. What you will probably get is a negative number in your downstream code, which of course is not correct.

Specific comment: If the underlying type of the existing array is different from the expected type of a routine you are using, I don't know what answer you are looking for. You will have to convert & copy the data or use a different routine. And if there is a potential for overflow, then you will have to manually check for this during the conversion instead of blindly letting the compiler/processor wrap such values to negative values (like you do in the above code). On older versions of MATLAB (R2006a and earlier) the Ir and Jc arrays were ints, so there would be no conversion necessary. But on newer versions of MATLAB I don't see any shortcut possible if you want to use the routine that is expecting an int* as input. At the very least, even if mwIndex (size_t) and int are the same size (e.g., both 32-bit), to be robust you would have to check the values for int overflow before passing
the pointer to the routine.

James Tursa

[Thomas Meinguet]

Thank you James.

To prevent any cast issue at no operational cost I suggest to check, for sparse martices, whether nnz <= INT_MAX which in turn implies n < INT_MAX. This already allows up 2 billions of on zero elements per matrix which should do the job for most applications.

But still looking for a way to not duplicate large data arrays in the memory only for that. Maybe a good approach would be to enhance the choice of very large arrays when creating the sparse matrix in the m-code rather than as an option of the mex command?

I can see this is a popular topic :-)

[Bruno Luong]

"Thomas Meinguet" <thomas....@skynet.be> wrote in message <kcgfbp$v9$1...@newscl01ah.mathworks.com>...

>
> But still looking for a way to not duplicate large data arrays in the memory only for that.

>Maybe a good approach would be to enhance the choice of very large arrays when creating the sparse matrix in the m-code rather than as an option of the mex command?

If the library would able to handle 64-bit indexing, there even no need for such thing.

Bruno

[James Tursa]

"Thomas Meinguet" <thomas....@skynet.be> wrote in message <kcgfbp$v9$1...@newscl01ah.mathworks.com>...

> Thank you James.
>
> To prevent any cast issue at no operational cost I suggest to check, for sparse martices, whether nnz <= INT_MAX which in turn implies n < INT_MAX.

This will not work. nnz simply tells you how many non-zeros there are ... it doesn't tell you anything about their values, and it is their values that your are interested in. You could have nnz = 1 and at the same time that one index value could overflow an int.

James Tursa

[Thomas Meinguet]

Correct.

Forgot to mention the check assumed non empty lines and columns, which is reasonable or should be reasonably achievable by deleting them and keeping trace of the operation.

I agree if the library admitted size_t there won't be any problem. If Matlab had a mex-flag Classical/Large for the array size, similarly to Single/Double, Dense/Sparse or Real/Complex it would also does the job and allows the developer not to copy data for nothing when the indexing fits INT limits.

Let's ask both enhancements.

To summarize all the contributions, in the meantime, the code below should do the bridge with clean check and optional second part depending on how A is constructed.

But the topic is still open to any improvement, remark or useful finding. Thanks all!

#include "mex.h"
#include "limits.h"

void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) {
if(nrhs != 1) { mexErrMsgTxt("missing argument"); }
if(!mxIsSparse(prhs[0])) { mexErrMsgTxt("matrix A must be sparse"); }

int *rowInd, *colPtr;
const mwSize n = mxGetN(prhs[0]);
const mwSize m = mxGetM(prhs[0]);

mwIndex *MWrowInd = mxGetIr(prhs[0]);

mwIndex *MWcolPtr = mxGetJc(prhs[0]);

// If A does not contain empty rows, this prevents cast issues:
if(MWcolPtr[n] > INT_MAX) { mexErrMsgTxt("indexing exceeds INT limits"); }

const int nnz = (int) MWcolPtr[n];

rowInd = new int[nnz];
colPtr = new int[n+1];

for(int i=0; i<nnz; i++) {

// If A is allowed to contain empty rows, also check
if(MWrowInd[i] > INT_MAX) { mexErrMsgTxt("indexing exceeds INT limits"); }

rowInd[i] = (int)(MWrowInd[i]);
}

for(int i=0; i<=n; i++) { colPtr[i] = (int)(MWcolPtr[i]); }

MWrowInd = 0;
MWcolPtr = 0;

const double *a = (double*) mxGetPr(prhs[0]);

printf("n=%d\n",n);
printf("nnz=%d\n",nnz);
printf("m=%d\n",m);
for(int i=0; i<nnz; i++) { printf("a(%d)=%f\n",i,a[i]); }
for(int i=0; i<nnz; i++) { printf("rowInd(%d)=%d\n",i,rowInd[i]); }
for(int i=0; i<=n; i++) { printf("colPtr(%d)=%d\n",i,colPtr[i]); }
}

[Thomas Meinguet]

small amendments to fix the indicices of the indexation as well:

Add:
// If A is allowed to contain empty cols, check
if(n > INT_MAX) { mexErrMsgTxt("indexing exceeds INT limits"); }

Update:
if(MWrowInd[i] >= INT_MAX) { mexErrMsgTxt("indexing exceeds INT limits"); }