Hi --
This shouldn't be a problem for a modern PC ... are you using a 32 bit machine right now? That might be the first issue.
If you're using a 64 bit machine now, are you sure you have the 64 bit JVM and 64 bit Cytoscape installed?
It might be helpful for you to run the Troubleshooting script (
http://cytoscape.org/troubleshooting.html) and post the results here. If you're running Windows, be sure to save the script on your disk and then run it from a command line.
Also, if you're willing to send me your network, I can try it on one of our machines.
Note, too, that Cytoscape 3.2 and later makes better use of its memory ... if you're not using latest Cytoscape (3.4), you should consider upgrading.