Error when run GSEA on windows command line.

2 views
Skip to first unread message

Dan Li

unread,
Nov 28, 2025, 11:51:48 AM (5 days ago) Nov 28
to gsea-help
Hello:

When the GMT file is huge (https://www.gsea-msigdb.org/gsea/msigdb/download_file.jsp?filePath=/msigdb/release/2025.1.Hs/msigdb.v2025.1.Hs.symbols.gmt), I found that the desktop GSEA_4.1.0 software always had an error: "not enough Java heap space." The limit of Java heap space is 4G and I can't increase it. 
I tried to run GSEA on the Windows command line. 
```
D:\GSEA\GSEA_4.4.0-NoJava\GSEA_4.4.0\gsea-cli.bat -Xmx16g GSEAPreranked -gmx D:\R\RNA-seq\Fanny\Project_12768_B\GSEA\msigdb.v2025.1.Hs.symbols.gmt -collapse No_Collapse -mode Max_probe -norm meandiv -nperm 1000 -rnk D:\R\RNA-seq\Fanny\Project_12768_B\DEgenes_Post_vs_Pre.rnk.txt -scoring_scheme weighted -rpt_label DEgenes_Post_vs_Pre_all -create_svgs false -include_only_symbols true -make_sets true -plot_top_x 200 -rnd_seed timestamp -set_max 500 -set_min 1 -zip_report false -out D:\R\RNA-seq\Fanny\Project_12768_B\GSEA\DEgenes_On_vs_Pre_all
```
But the program can't initiate properly:
```
D:\R\RNA-seq\Fanny\Project_12768_B>setlocal

D:\R\RNA-seq\Fanny\Project_12768_B>for %x in (D:\GSEA\GSEA_4.4.0-NoJava\GSEA_4.4.0\gsea-cli.bat) do set BatchPath=%~dpsx

D:\R\RNA-seq\Fanny\Project_12768_B>set BatchPath=D:\GSEA\GSEA_4.4.0-NoJava\GSEA_4.4.0\

D:\R\RNA-seq\Fanny\Project_12768_B>for %x in (D:\GSEA\GSEA_4.4.0-NoJava\GSEA_4.4.0\) do set BatchPath=%~dpsx

D:\R\RNA-seq\Fanny\Project_12768_B>set BatchPath=D:\GSEA\GSEA_4.4.0-NoJava\GSEA_4.4.0\

D:\R\RNA-seq\Fanny\Project_12768_B>if exist D:\GSEA\GSEA_4.4.0-NoJava\GSEA_4.4.0\\jdk (
echo "Using bundled JDK."
 set JAVA_HOME=D:\GSEA\GSEA_4.4.0-NoJava\GSEA_4.4.0\\jdk
 set JAVA_CMD=D:\GSEA\GSEA_4.4.0-NoJava\GSEA_4.4.0\\jdk\bin\javaw
)  else (
echo "Using system JDK."
 set JAVA_CMD=java
)
"Using system JDK."

D:\R\RNA-seq\Fanny\Project_12768_B>if exist "C:\Users\lidan\.gsea\java_arguments" (start java --module-path=modules -Xmx4g -Djava.awt.headless=true -Djava.util.logging.config.file=logging.properties @gsea.args @"C:\Users\lidan\.gsea\java_arguments" --module=org.gsea_msigdb.gsea/xapps.gsea.CLI  -Xmx16g GSEAPreranked -gmx D:\R\RNA-seq\Fanny\Project_12768_B\GSEA\msigdb.v2025.1.Hs.symbols.gmt -collapse No_Collapse -mode Max_probe -norm meandiv -nperm 1000 -rnk D:\R\RNA-seq\Fanny\Project_12768_B\DEgenes_Post_vs_Pre.rnk.txt -scoring_scheme weighted -rpt_label DEgenes_Post_vs_Pre_all -create_svgs false -include_only_symbols true -make_sets true -plot_top_x 200 -rnd_seed timestamp -set_max 500 -set_min 1 -zip_report false -out D:\R\RNA-seq\Fanny\Project_12768_B\GSEA\DEgenes_On_vs_Pre_all )  else (start java --module-path=modules -Xmx4g -Djava.awt.headless=true -Djava.util.logging.config.file=logging.properties @gsea.args --module=org.gsea_msigdb.gsea/xapps.gsea.CLI  -Xmx16g GSEAPreranked -gmx D:\R\RNA-seq\Fanny\Project_12768_B\GSEA\msigdb.v2025.1.Hs.symbols.gmt -collapse No_Collapse -mode Max_probe -norm meandiv -nperm 1000 -rnk D:\R\RNA-seq\Fanny\Project_12768_B\DEgenes_Post_vs_Pre.rnk.txt -scoring_scheme weighted -rpt_label DEgenes_Post_vs_Pre_all -create_svgs false -include_only_symbols true -make_sets true -plot_top_x 200 -rnd_seed timestamp -set_max 500 -set_min 1 -zip_report false -out D:\R\RNA-seq\Fanny\Project_12768_B\GSEA\DEgenes_On_vs_Pre_all )
```
Can anyonr let me know what's wrong with the command?
Thanks a lot


David Eby

unread,
Nov 28, 2025, 12:46:47 PM (5 days ago) Nov 28
to gsea...@googlegroups.com
Hello,

The Java parameters used by GSEA need to be embedded in the launcher, and need to be set in a way that will work in general on most users' computers.  These parameters limit the GSEA program to 4 GB of RAM, which is usually enough for most analyses but can run into limits for large files as you've discovered.

Unfortunately, it's difficult to allow modifications to the launcher as it needs to be signed for distribution, so we've added a mechanism for adjusting Java parameters to avoid the need to edit the launcher.  In your user home directory, add a ".gsea" subdirectory and within that create a plain-text file named "java_arguments" with the contents as follows:
-Xmx8g

To simplify things, I've attached an example java_arguments file.  That setting will allow running GSEA with 8 GB RAM, and you can adjust that as needed depending on your system's available memory.  This is written up in more detail here:  https://github.com/GSEA-MSigDB/gsea-desktop/blob/master/scripts/readme.txt

All this being said, we actually do not recommend running GSEA with the full MSigDB GMT but rather suggest using individual collections.  There is far too much redundancy when looking at all these collections together side-by-side.  We recommend instead starting with the Hallmarks collection to investigate the dataset broadly and then doing follow-up analyses using individual collections based on what that uncovers.

I hope this is useful.

Regards,
David


--
You received this message because you are subscribed to the Google Groups "gsea-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gsea-help+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/gsea-help/ae71ece8-66fe-4323-9a97-4f4f2ca9f614n%40googlegroups.com.
java_arguments
Reply all
Reply to author
Forward
0 new messages