Memory Requirements

[John]

Hey Kyle,

This is John from the Biostats dept at Emory. I think it's pretty cool
I'll be getting to test your masterpiece for my thesis (I'm working
with Sharath and Linda). I have a question about the memory
requirements for data with say, 100 million cell counts. Will the
application be able to handle that many events? Also sometimes the
note "Data pt Error" appears on the bottom of the applet. However, all
the data seems to be loaded in. What does this error mean?

Thanks,

John

[Kyle Thayer]

Hey John,

I'm glad to hear you will be using the program for your thesis. By all means, let me know what issues you run into and if you need help with anything. In answer to your two questions here:

1) The memory is set to a default maximum of 700 Mb, which will probably fall way short of what you need for data of 100 million cells. The memory is set by the jnlp file that launches the program, so download and edit IFCSoft.jnlp, and change the 700m in:

<j2se version="1.5+" java-vm-args="-Xmx700m"/> to whatever you want (I think you can get it highest on a 64 bit Windows 7 machine. I don't know about Mac). Make sure it IFCSoft.jnlp is saved as a ".jnlp" file and isn't a ".xml" file.

If you have that many points though, it is probably best to reduce the data in any way you can, for example:

* Using flowjo or some other program, save out only the cell populations that you are interested in (eg. remove junk data and save out only the CD8 T-cells for example)

*  Save only the dimensions that you are going to need to reduce memory usage. It may be best to use FCSExtract to first turn them into CSV files. Load those into Excel or whatever and delete unneeded columns.

*  Pick a random sample from the data you have. When you load data sets into IFCSoft, there is an option in the data set viewer to shrink the data set. You give it a percent to keep (say keep 10%) and it keep a random 10%. You can then save this back out again. You would have to do this for each set and restart the program when too many sets are loaded. If you wanted to write your own program to do this, that might be easier.

2) Data pt Error is actually a measure of how accurate the SOM is, it doesn't mean that anything broke. I should remove it from the screen and have it under some SOM statistics display.

Let me know if you run into further issues,

   Kyle