-Xms8192M allocates 8 GB of memory right from the start. This can help with performance. At least, it makes sure that the OS can allocate this much memory before it starts the slave. This eliminates one OOM candidate where you've configured Java for enough memory but the OS simply doesn't have it.
-Xmx8192M should fix your problem unless
1. it's not used. Look at the process table of the slave when the job is running to make sure the option was picked up (it should be but checking is better than believing).
2. you have a huge memory leak. If it always stops at this point, then check the environment variables. Maybe someone was "clever" and tried to put a huge amount of data into an environment variable.
This doesn't work because every memory leak is different. You will have to understand your own code, how your configured Jenkins and how everything works.
Use a memory profiler like YourKit or or try to disable things until the error goes away.
Regards,
Aaron Digulla