allocating memory with mac os X

[Simon]

Hi everyone,

I'm using Refine since few years (on Ubuntu & windows), so at the very begining of this year I was very happy to receive my new macbook pro with 16gb of RAM on Yosemite.

I installed refine very quickly and follow the method to allocate approximatively 8Gb of memory.

(I mean opening the pacage and update Info.plist)

Unfortunately, it seems that Refine & chrome never use more than 1GB...

Do you have any idea of what I should try ?

I'm a kind of newbie on macosx.

Thx a lot.

[Owen Stephens]

I use OpenRefine 2.6beta on Yosemite with 4Gb allocated without any issues.

Can you share the relevant lines from your Info.plist?

Thanks

[Simon]

thx Owen,

this is my info.plist : 

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

<plist version="1.0">

<dict>

<key>CFBundleName</key>

<string>Google Refine</string>

<key>CFBundleShortVersionString</key>

<string>2.5</string>

<key>CFBundleGetInfoString</key>

<string>Google Refine 2.5</string>

<key>CFBundleAllowMixedLocalizations</key>

<string>false</string>

<key>CFBundleInfoDictionaryVersion</key>

<string>6.0</string>

<key>CFBundleExecutable</key>

<string>JavaApplicationStub</string>

<key>CFBundleDevelopmentRegion</key>

<string>English</string>

<key>CFBundlePackageType</key>

<string>APPL</string>

<key>CFBundleSignature</key>

<string>????</string>

<key>CFBundleIconFile</key>

<string>google-refine.icns</string>

<key>CFBundleIdentifier</key>

<string>com.google.refine.Refine</string>

<key>Java</key>

<dict>

<key>MainClass</key>

<string>com.google.refine.Refine</string>

<key>JVMVersion</key>

<string>1.6+</string>

<key>ClassPath</key>

<array>

<string>$JAVAROOT/jdatapath-alpha2.jar</string>

<string>$JAVAROOT/jetty-6.1.22.jar</string>

<string>$JAVAROOT/jetty-util-6.1.22.jar</string>

<string>$JAVAROOT/log4j-1.2.15.jar</string>

<string>$JAVAROOT/servlet-api-2.5.jar</string>

<string>$JAVAROOT/slf4j-api-1.5.6.jar</string>

<string>$JAVAROOT/slf4j-log4j12-1.5.6.jar</string>

<string>$JAVAROOT/google-refine-2.5-r2407-server.jar</string>

</array>

<key>VMOptions</key>

<string>-Xms256M -Xmx8192M -Drefine.version=r2407</string>

<key>WorkingDirectory</key>

<string>$APP_PACKAGE/Contents/Resources</string>

<key>Properties</key>

<dict>

<key>apple.awt.textantialiasing</key>

<string>true</string>

<key>apple.laf.useScreenMenuBar</key>

<string>true</string>

<key>apple.awt.antialiasing</key>

<string>true</string>

</dict>

</dict>

</dict>

</plist>

[Owen Stephens]

Hi Simon,

I can't see anything wrong there. If I use the same setup and load a large file I see memory usage for Refine go above 1Gb (via OS X Activity Monitor), but only just. That's the limits of my knowledge here I'm afraid - memory management isn't something I know a lot about and what I've read suggests its complicated :)

As an aside I notice you are running GoogleRefine 2.5. Generally I'd recommend using OpenRefine 2.6 beta despite the 'beta' label.

Owen

[Tom Morris]

Are you using a 64-bit JVM?  What's the output of the command java -version ? Are you opening projects which are big enough to require more than 1 GB of RAM?

Tom

--
You received this message because you are subscribed to the Google Groups "OpenRefine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[K.Willson]

I followed this thread because I've been unable to load a data set into OpenRefine (I tried the Beta version but it isn't working on my machine). I installed the stable 2.5 version and followed the directions to allocate more memory. But I'm unable to save the change from 1024 to 2048M.

Do any of you know what might be the problem?

This might be moot because the program seems to have been designs for small data sets. My smallest, and the one I tried to upload, is 336M

Thoughts?

[Owen Stephens]

Hi K Willson

It looks like the problem is with saving the updated version of the plist file. Generally my first guess would be that your account on your computer doesn't have permission to edit that file, but as the message in the screenshot says you can check the Console to see what the error message was. This isn't anything related to OpenRefine directly, but some issue in saving the plist file. Console is a built in application in OS X to view log files and errors etc.

In terms of what amount of data OpenRefine can handle - it isn't just about file size, but about the number of rows/columns of data. OpenRefine can cope with say up to 1 million rows OK if there are relatively few columns, but would not work with the number of columns and rows reversed. That said, 336M does sound on the large side - how many rows/cols does this have?

Owen

(PS As a general rule it's best to start a new discussion thread when the problem you have is different to the original post - just means different issues can be discussed separately)

[Jason Perfetto]

Hey K. Wilson - I also had difficulty saving, but I found that you need to disable your System Integrity Protection in order to be able to edit and save this file. You can do so by rebooting in recovery mode, going into terminal, and entering the command crsutil disable, then restart. You should now be able to open your plist file in Xcode and save the parameters you modify. Once you have increased your upper memory bound you should immediately restore SIP by doing the reverse process but this time entering crsutil enable. System integrity protection is vital to protecting your mac from root level malware and security threats, and you should never leave it off for an extended period of time. 

[Thad Guidry]

Jason,

Could you add something about that to our Wiki section Running & Configuration specifically for Mac users ? along with your security warning note to only do what you suggest as temporary ?

Thanks !

-Thad

+ThadGuidry