HELP! Problems with ENMTools - correlations

[jen guevara]

Hi everyone,

I have just started to use ENMTools and Maxent so I'm still a bit new.
I have tried to use the Correlation tool with little luck (using the
19 bioclim layers-30s resolution for the Americas). I have tried
running it with the 5m-resolution layers and it works fine so I'm
thinking that my layers are just too massive.

This is the error message I get every time:

malloc: *** mmap(size=16777216) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
Out of memory!

I have plenty of memory available in my computer and I have tried
running the same analysis using a Mac and a PC, both times
unsuccessful.

Is there a way to get around this problem? Or is it just impossible to
run high resolution layers? Any ideas? THANK YOU so much!

Jen
-------------------------------
Jennifer Guevara
PhD candidate
Zoology Department
University of British Columbia
Vancouver, Canada

[dan.l....@gmail.com]

It does appear to just be a memory issue. The script as it's
currently written loads both rasters being compared into memory, which
means that your available RAM (i.e., what's left over after accounting
for everything else your computer is doing) needs to be at least twice
the size of your input files. I may be able to bash together a
command line script that doesn't load the files into memory, but it
will be significantly slower. I'll send you an email if I manage to
work that out.

Dan

[MikeC]

Sorry to drop in, but I'm getting the same sort of error message: Out of memory. Callback called exit at ENMTools - 3-17-2011.pl line 1186.  I'm trying to undertake niche overlap - two ascii files both about 250Mb each. CPU memory is 4Gb, and I've tried allocating 1024Mb then 2048Mb via config file. Any advice/ suggestions? Many thanks
cheers
MikeC

[dan.l....@gmail.com]

Hi Mike! If you've got Perl installed (or if you're on a Mac), you
can download this Perl script that does overlap on large files:

http://www.danwarren.net/enmtools/builds/overlap_large.pl

To run it you just drop it into a directory that has your .asc files
in it. You will also need to create a .csv file that has the names of
all of the pairs of files to be compared, like this:

file1.asc,file2.asc
file2.asc,file4.asc
file3.asc,file1.asc

et cetera.

Then go to a command line and navigate to that directory. Type "perl
overlap_large.pl LISTFILE",where LISTFILE is the .csv list you just
made. If all goes well it should spit out another .csv file ending in
"_output.csv" that has your overlaps in it. With large files this
could take quite a while, though, so be patient!

One more tip - Perl sometimes chokes on Mac line endings, so if you're
having trouble make sure that your .asc files have been saved with
Windows or Unix line endings.

Let me know if you run into any problems. Cheers!

Dan

[cava...@netspace.net.au]

Hi Dan,

many thanks for reply. I tried the address you sent via both on my PC and Mac
but got the following error message both times:

Internal Server Error

The server encountered an internal error or misconfiguration and was unable to
complete your request.

Please contact the server administrator, webm...@danwarren.net and inform them
of the time the error occurred, and anything you might have done that may have
caused the error.

More information about this error may be available in the server error log.

Additionally, a 404 Not Found error was encountered while trying to use an
ErrorDocument to handle the request.
Apache/2.2.14 (Unix) mod_ssl/2.2.14 OpenSSL/0.9.7a mod_auth_passthrough/2.1
mod_bwlimited/1.4 FrontPage/5.0.2.2635 Server at www.danwarren.net Port 80

Any ideas?

cheers
MikeC

----------------------------------

Quoting "Dan.L....@gmail.com" <dan.l....@gmail.com>:

> --
> You received this message because you are subscribed to the Google Groups
> "Maxent" group.
> To post to this group, send email to max...@googlegroups.com.
> To unsubscribe from this group, send email to
> maxent+un...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/maxent?hl=en.
>
>

------------------------------------------------------------
This email was sent from Netspace Webmail: http://www.netspace.net.au

[dan.l....@gmail.com]

Ah, I should have mentioned that you have to right click and "Save
as". It's a perl script, so your browser will actually try to execute
it if you just click it like a normal link.

[MikeC]

Hi Dan,

Many thanks for follow up. When I click and Save Link As - I get a
file called url.htm In notepad, the contents of this file are:

<HTML><HEAD>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>Redirecting</TITLE>
<META HTTP-EQUIV="refresh" content="1; url=http://www.danwarren.net/
enmtools/builds/overlap_large.pl">
</HEAD>
<BODY onLoad="location.replace('http://www.danwarren.net/enmtools/
builds/overlap_large.pl'+document.location.hash)">
Redirecting you to http://www.danwarren.net/enmtools/builds/overlap_large.pl</BODY></HTML>

If I try to tun that in perl all it does is open a browser window
which shows the original error message. There is a previous page that
pops up for about half a second that says something like its trying to
redirect to http:\dan warren etc but it dissapears and I'm left with
the error message.

I've also tried this in the perl command line I'm typing >d:\MikeC
\maxent_outs\enm_olaps\perl overlap_large.pl LISTFILE and get a
command line error message that 'd:\MikeC\maxent_outs\enm_olaps\perl'
is not recognised as an internal or externalcommand, operable program
or batch file. The default command line prompt is: C:\Documents and
Settings\cavanaghm\My Documents>

Hope this makes sense - and I dare say I'm probably missing the
obvious! Many thanks


On Feb 9, 12:39 am, "Dan.L.War...@gmail.com" <dan.l.war...@gmail.com>
wrote:

[dan.l....@gmail.com]

Well darn, your browser is definitely not letting you download it.
I'll try emailing it to you instead.

[SG]

Hi! I was having the same memory problem when trying to run the niche
overlap analysis. I tried to download the perl script but ran into
similar errors as the previous poster. Would it be possible to
download the perl script from a different place?
Any help would be appreciated!

Thanks!

Sincerely,
Shalika Gupta

[John Baumgartner]

Might be best to just rename the extension to something the server
won't try to execute, like .txt.

[dan.l....@gmail.com]

I put a renamed copy here:

http://www.danwarren.net/enmtools/builds/overlap_large.plx

Which you can download and rename to .pl. Hope that helps!

On Feb 17, 4:06 am, John Baumgartner <jbb...@gmail.com> wrote:
> Might be best to just rename the extension to something the server
> won't try to execute, like .txt.
>

[SG]

Hi, thanks for your quick response. I tried the new link on a few
different computers and still had the same trouble with the internal
server error. I right-click on the file and save linked file as/
downloaded file as a file with a .pl extension, but when I open the
script in a text editor it reads:

<HTML><HEAD>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>Redirecting</TITLE>
<META HTTP-EQUIV="refresh" content="1; url=http://www.danwarren.net/
enmtools/builds/overlap_large.pl">
</HEAD>
<BODY onLoad="location.replace('http://www.danwarren.net/enmtools/
builds/overlap_large.pl'+document.location.hash)">
Redirecting you to http://www.danwarren.net/enmtools/builds/overlap_large.pl</BODY></HTML>

Maybe I'm making an error somewhere?

Thanks!

Sincerely,
Shalika

On Feb 17, 3:55 am, "Dan.L.War...@gmail.com" <dan.l.war...@gmail.com>
wrote: