Monte Carlo Post-Processor (MCPP)__Error using dataread: Buffer overflow (bufsize = 65536) while reading string from file (row 1, field 1).

[future...@gmail.com]

Hello,

I came across a weird problem when reading post-process data. The post-process data is attached here. Its size is very small, since I am doing a test with MCPP by launching 100 photons.

Thanks and Stay Safe!

Owen

------------------------------------Matlab code to read post-processed data---------------------------

outdir = 'C:\MonteCarlo\MC_v4.8.0Beta\';

dataname = 'SET1_FixMusp_g_p9_post';

MC07b = loadMCResults(outdir, char(dataname));

----------------------------------------Error Message  below---------------------------

Error using dataread

Buffer overflow (bufsize = 65536) while reading string from

file (row 1, field 1).

Error in textread (line 171)

[varargout{1:nlhs}]=dataread('file',varargin{:}); %#ok<REMFF1>

Error in loadMCResults>readAndParseJson (line 1166)

json_strings = textread(filename, '%s',  'whitespace', '', 'bufsize', 65536);

Error in loadMCResults (line 7)

json = readAndParseJson([datadir slash dataname '.txt']);

Error in test (line 12)

MC07b = loadMCResults(outdir, char(dataname));

[Carole Hayakawa]

Hi Owen,

Nice to hear from you!  Thanks for posting. 

I can reproduce your error. I fixed it by modifying line 1233 in loadMCResults.m to

json_strings = textread(filename, '%s', 'whitespace', '', 'bufsize', 131072);

I increased 'bufsize' by a factor of 2.  I think you just ran up against the maximum buffersize.  I noticed that your infile specifies

around 57 detectors.  I think you are working in a realm no other man has gone!  Please try my quick fix and let me know how

it goes for you.  If it appears to work for you, I will think about how we should fix this on our side so that it gets updated in the

next release.

Thanks for your help to improve our software and glad you are still using it.

Hope you stay safe too!

Carole

[future...@gmail.com]

Hi, Carole, 

Your quick fix works. However, the real cause for this issue may not be using 57 detectors, since the original loadMCResults.m can read another set of post-processed data (MC07b_Post_pMC.rar)  without any error. I attached it in this post.

I am very glad to hear from you! Thanks for your timely rescue on me! 

Owen

[future...@gmail.com]

Hi, Carole,

Here are My another two feedbacks about MCPP when doing post-processing with mc_post.exe

1) outname command cannot be used.

2) The database folder must be placed in directory: C:\MonteCarlo\MC_v4.8.0Beta\, or else it cannot read.

Owen

On Monday, April 27, 2020 at 8:37:07 PM UTC-4, Carole Hayakawa wrote:

[Carole Hayakawa]

Hi Owen,

Great feedback!  I will check into why loadMCResults.m needs larger buffer size.

Concerning your feedback about the MCPP.

1) So I was able to change "OutputName" in the MCPP infile and it worked. 

Are you using the command line option to change the output folder name?  I was able to do that too using

mc infile=infile_pMC_one_layer_ROfRho_DAW.txt outpath=test

This was on a windows system using the example infile.  This got fixed with Issue #24 and is in the MC_v4.8.0Beta which you are using.

If you are still having problems, please send me sample infiles for the database gen and the MCPP.

2) I can replicate this error.  I stepped through the code and the MCPP infile validation is catching the discrepancy between the inputFolder in the

infile and the inpath specified in the command line.  This is Issue #22 which I closed because my unit test on this fix passed.  As I step through the code I see now that the unit test passed erroneously because a prior test had been run and created a folder in the default folder that it was finding.  I will reopen issue and work on this.  Thanks for finding this error!

Carole

[Carole Hayakawa]

Hi Owen,

I fixed the command line directive "inpath" bug.  We created a release 4.9.0 on March 9, 2020 and so this fix will be in our next release.

Thanks again for providing feedback to us, it is very helpful.

Carole

[future...@gmail.com]

Hi, Carole,

For the first feedback, I attached infile in this post. 

Below is my commands to do post-processing:

mc_post.exe infile=C:\MonteCarlo\Infiles\infile_FixMusp_g_p9_post_pMC.txt^

outpath=C:\MonteCarlo\Results^     

outname=SET1_FixMusp_g_p9_post

......

By the way, I found you on LinkedIn!

Owen

[Carole Hayakawa]

Hi Owen,

Here is what I did.  I downloaded your infile.  Since it was for the post-processor, I used mc.exe results that I had generated previously and renamed the folder to SET1_FixMusp_g_p9_Baseline and the associated infile inside folder to same name with extension .txt.

I tried to run your command as is (with my absolute referencing) and with relative referencing i.e.

mc_post.exe infile=infile_FixMusp_g_p9_post_pMC.txt outpath=Results outname=SET1_FixMusp_g_p9_post

and both worked for me.  I'm on a linux machine and used MC_v4.8.0Beta.

Thanks for your continued help with helping us find bugs!

Carole