Alternatives to process isolation?

[ivan]

It seems many approaches involve sandboxing applications. Wether it's a browser process, an assembly sandbox ,or an isolated process like in singularity, so on. I was wondering if there are alternatives to sandboxing to prevent cases for example, where a virus simply modifies the executables. Even if the process is sandboxed, if it has disk permissions the sandbox doesn't prevent it from maliciously modifying files. I was thinking if there have been experiments monitoring the footprint of an application (memory, channels used, typical read writes ,etc) and seeing if the pattern changes to flag the process. 

[Deian Stefan]

There definitely has been research on this end.

1. Network intrusion detection systems that go beyond the functionality of Snort (effectively pattern-matching)  sometimes monitor features of the system (e.g., network traffic behavior) and try to detect anomalous behavior.

2. IFC systems let you enforce stronger policies than can/cannot write to disk. For example, you can make sure that if the process has read from the network (and thus may be influenced by untrusted parties) it can no longer write to disk.

An interesting paper that may be worth looking it is Dynamic Taint Analysis: Automatic Detection, Analysis, and Signature Generation of Exploit Attacks on Commodity Software. They use taint tracking to make sure that data from untrusted sources, such as the network, cannot affect trustworthy sinks, such as system call arguments.

3. Auditing systems. Among other, one example is Aeolus. The system keeps a log of all security-related event for an application. You can imagine building a log while running the app in a safe environment, and then using the log as a comparison in the wild.

4. Somewhat low-level, but probably one of the most relevant works is systrace.

[Raj Raghavan]

"I was thinking if there have been experiments monitoring the footprint of an application (memory, channels used, typical read writes ,etc) "

I am not too familiar on the research side, but just to wanted to express on how this issue is confronted practically.

In large enterprises opnet trace is extensively used to sniff network traffic, the opnet trace contains exhaustive information with all the information regarding the network anomalies. To monitor the footprint of an app on server, there are multiple tools, however the most effective one is called e-health (from CA)  which will record pretty much everything happening on the server like processor, memory, IO, page files etc. You can go back in time and see a graphical representation of how the health of the server was in the past.