Problem in Matlabstan: Expected input to be of these types

[Daniel Berman]

Sorry, I forgot to change the name of the topic before I posted it. I have a model that I built in R using Rstan and now I want to implement it in matlab since I recently got matlabstan working. So I know the model works, runs and gives the correct outputs in R. However, when I tried to run it in matlab, I run into problems.

I have posted my code and the portion of my matlab code that is relevant with the workspace to run it, this appears in matlab:

Having a problem getting stan version.
This is likely a problem with Java running out of file descriptors
Trying again.
132 end
output-1.csv: Exception: mismatch in number dimensions declared and found in context; processing stage=data initialization; variable name=measurementResults; dims declared=(4,1000,16); dims found=(64,1000)
output-1.csv: Diagnostic information:
output-1.csv: Dynamic exception type: std::runtime_error
output-1.csv: std::exception::what: mismatch in number dimensions declared and found in context; processing stage=data initialization; variable name=measurementResults; dims declared=(4,1000,16); dims found=(64,1000)

When I try to print the output of the sampling function, I get:

ans =

    'This application has requested the Runtime to terminate it in an unusual way.'
    'Please contact the application's support team for more information.'
    'terminate called after throwing an instance of 'std::invalid_argument''
    '  what():  Error with header of input file in parse'

I'm a bit confused as to what to do.

 

[Aki Vehtari]

On Wednesday, July 22, 2015 at 5:44:20 AM UTC+3, Daniel Berman wrote:

output-1.csv: Exception: mismatch in number dimensions declared and found in context; processing stage=data initialization; variable name=measurementResults; dims declared=(4,1000,16); dims found=(64,1000)

 Check the dimensions of measurementResults in Matlab. If the dimensions match (4,1000,16), then check that measurementResults has correct dimensionality in temp.data.R (created by MatlabStan).

Aki

[Daniel Berman]

Sorry I over looked that. I fixed the dimensionality on my end, but when I look at temp.data.R, measurementResults is not present. It's not the only data that's missing. Another 3D array is missing. Both that are missing are 3D arrays. The matrices, vectors, and real/integer values are missing. Is there a reason for this? I've uploaded my updated code. The same workspace should work.

[Daniel Berman]

Hey,

I looked into the code and in the file rdump.m, I noticed that there are only categories for scalar, vector, and matrix. Is there a possibility of adding the option for arrays? Otherwise, it's not possible to add arrays into the code. I have posted the code from rdump below.

% https://github.com/stan-dev/rstan/search?q=stan_rdump&ref=cmdform
% struct or containers.Map
function fid = rdump(fname,content)
   if isstruct(content)
      vars = fieldnames(content);
      data = struct2cell(content);
   elseif isa(content,'containers.Map')
      vars = content.keys;
      data = content.values;
   end

   fid = fopen(fname,'wt');
   for i = 1:numel(vars)
      if isscalar(data{i})
         if any(data{i}(:) > intmax('int32'))
            fprintf(fid,'%s <- %f\n',vars{i},data{i});
         else
            fprintf(fid,'%s <- %d\n',vars{i},data{i});
         end
      elseif isvector(data{i})
         fprintf(fid,'%s <- c(',vars{i});
         if any(data{i}(:) > intmax('int32'))
            fprintf(fid,'%f, ',data{i}(1:end-1));
            fprintf(fid,'%f)\n',data{i}(end));
         else
            fprintf(fid,'%d, ',data{i}(1:end-1));
            fprintf(fid,'%d)\n',data{i}(end));
         end
      elseif ismatrix(data{i})
         fprintf(fid,'%s <- structure(c(',vars{i});
         if any(data{i}(:) > intmax('int32'))
            fprintf(fid,'%f, ',data{i}(1:end-1));
            fprintf(fid,'%f), .Dim = c(',data{i}(end));
         else
            fprintf(fid,'%d, ',data{i}(1:end-1));
            fprintf(fid,'%d), .Dim = c(',data{i}(end));
         end
         fprintf(fid,'%g,',size(data{i},1));
         fprintf(fid,'%g',size(data{i},2));
         fprintf(fid,'))\n');
      end
   end
   fclose(fid);
end

[Aki Vehtari]

By array do you mean Matlab's cell array? Cell's in cell array may have different data types and different sizes, which is not supported by Stan (ragged arrays are in the todo list). I think MatlabStan should give an error if the data is no numerical scalar, vector or matrix.

Aki

[Daniel Berman]

No I don't mean cell arrays. I mean a matrix with three or more dimensions. The function ismatrix() only returns true if it has 2 dimensions. measurementResults has three. Therefore, it is not included in the temp.data.R file. That is why he code will not run.

[Aki Vehtari]

Ok, this should be relatively easy to fix. (I've used Matlab for over 20 years, and I couldn't remember that Matlab uses also the term 'numeric array')

Aki

[Daniel Berman]

I put a temporary fix on my end so that it accepts the multidimensional arrays. However, I'm still having a problem. I'm not really sure what's going on but my code won't run. I know that the model works because I've used it in R.

Stan is sampling with 1 chains...

gst_est =

Stan is still working. You can either:
  1) Use the peek method to get partial samples
  2) Come back later, or
  3) Attach a listener to the StanFit object.
  StanFit with properties:

          model: [1x1 StanModel]
      processes: [1x1 processManager]
    output_file: {'C:\Users\daniel\Documents\QuantumComputingIndependentStudy\gst_matlab...'}
        verbose: 1
     exit_value: NaN
         loaded: NaN
            sim: []
        version: '0.8.0'

output-1.csv: Exception: mismatch in dimension declared and found in context; processing stage=data initialization; variable name=Sigma; position=0; dims declared=(4,4); dims found=(8,8)

output-1.csv: Diagnostic information:
output-1.csv: Dynamic exception type: std::runtime_error

output-1.csv: std::exception::what: mismatch in dimension declared and found in context; processing stage=data initialization; variable name=Sigma; position=0; dims declared=(4,4); dims found=(8,8)

ans =

    'This application has requested the Runtime to terminate it in an unusual way.'
    'Please contact the application's support team for more information.'
    'terminate called after throwing an instance of 'std::invalid_argument''
    '  what():  Error with header of input file in parse'

Stan is still working. You can either:
  1) Use the peek method to get partial samples
  2) Come back later, or
  3) Attach a listener to the StanFit object.

[Brian Lau]

Hi Daniel

I see that Aki opened an issue on Github to deal with the multidimensional arrays. I'm happy to take a look at this. Can you post the current version of your script that generates the problem?

Thanks,

-b

[Daniel Berman]

Hi,

Here is the workspace and the code that I'm using. I do need to address the more pressing issue of getting the entire code to work. I've attached my patched version of rdump in case you'd like to see what I did. I'm not a developer so it's probably not a great patch.
I should mention that in the GST_Bayesian.m file, lines 133-138 should be commented out. I'm exploring a way to get this to run in R because I have a short deadline and I the code to work as soon as possible.

[Aki Vehtari]

On Friday, July 24, 2015 at 1:31:31 AM UTC+3, Daniel Berman wrote:

I put a temporary fix on my end so that it accepts the multidimensional arrays. However, I'm still having a problem. I'm not really sure what's going on but my code won't run. I know that the model works because I've used it in R.

... 

output-1.csv: Exception: mismatch in dimension declared and found in context; processing stage=data initialization; variable name=Sigma; position=0; dims declared=(4,4); dims found=(8,8)

Did you check the size of Sigma?

Aki

[Daniel Berman]

My version has Sigma as a 4x4 matrix.

[Aki Vehtari]

Did you check the size of the Sigma in in temp.data.R?

Aki