[ADMB Users] ADMB and MinGW-w64 users should update

31 views
Skip to first unread message

Johnoel Ancheta

unread,
Oct 27, 2014, 4:40:10 PM10/27/14
to ADMB Users, devel...@admb-project.org
An issue and been found by Jeff Laake and corrected.


Users that need buffers greater than 2GB, please download the most recent


Follow the installation procedure below


dave fournier

unread,
Oct 28, 2014, 10:30:46 AM10/28/14
to us...@admb-project.org
As I recall there were two issues involved with having a buffer size >
2GB. First was
to use an 8 byte integer type for the offset.

The second issue was that if the buffer gets full it is saved to disk
with a low level unbuffered
write. This write could not write more than 2GB for me. Of course if
the buffer is large
enough so that it never gets full that is not a problem.
_______________________________________________
Users mailing list
Us...@admb-project.org
http://lists.admb-project.org/mailman/listinfo/users

dave fournier

unread,
Nov 6, 2014, 11:00:42 AM11/6/14
to us...@admb-project.org

I did a bit of experimenting with writing large buffers to disk and it
appears that
replacing the low level writes with fwrites and open with fopen etc .
permits one
to write large buffers to disk in the code.

A more interesting solution might be to optionally add another thread
which can write
portions of the buffer to disk as they become full.

dave fournier

unread,
Nov 6, 2014, 2:43:57 PM11/6/14
to Johnoel Ancheta, ADMB Users
On 11/06/2014 11:23 AM, Johnoel Ancheta wrote:

I already did a bit of testing. test_write.cpp is the low level one and test_write2.cpp uses fopen.


Yes, this is a good idea.  I'll try changing write functions and compare speeds.
test_write.cpp
test_write2.cpp

dave fournier

unread,
Nov 6, 2014, 2:50:45 PM11/6/14
to Johnoel Ancheta, ADMB Users
On 11/06/2014 11:43 AM, Johnoel Ancheta wrote:

No they seemed to be about the same. Point was I could write the entire buffer
of  4294967304 bytes with one fwrite.  So that would fix the immediate problem.
Did you find speed improvements?

Johnoel Ancheta

unread,
Nov 17, 2014, 5:38:45 PM11/17/14
to dave fournier, ADMB Users
The first issue was corrected.  See


The second issue, I will need to look into.  MinGW-w64 does support large files,
but I will need to test this.

Johnoel Ancheta

unread,
Nov 17, 2014, 5:41:19 PM11/17/14
to dave fournier, ADMB Users
Yes, this is a good idea.  I'll try changing write functions and compare speeds.

Johnoel Ancheta

unread,
Nov 17, 2014, 5:43:43 PM11/17/14
to dave fournier, ADMB Users
Did you find speed improvements?
Reply all
Reply to author
Forward
0 new messages