[lemona commit] r431 - wiki
codesite...@google.com
Date: Thu Nov 13 16:48:09 2008
New Revision: 431
Modified:
wiki/Workshop20081031.wiki
Log:
Edited wiki page through web user interface.
Modified: wiki/Workshop20081031.wiki
==============================================================================
--- wiki/Workshop20081031.wiki (original)
+++ wiki/Workshop20081031.wiki Thu Nov 13 16:48:09 2008
@@ -9,12 +9,14 @@
---------------------------------------------------------------------------
- _*CAUTION: This page is a work in progress as long at the WIP tag has
not been removed.
- Results should not be considered solid until the final revision.*_
-
- _For the educational interest of future fellow ITEC810 students, the
original
- draft of this workshop paper is still available for online browsing and
- download ([Workshop20081017 IT-SOFT 2008 Workshop Paper Draft])_
+ _*CAUTION: This page is a work in progress as long at the WIP tag
+ has not been removed. Results should not be considered solid until
+ the final revision.*_
+
+ _For the educational interest of future fellow ITEC810 students, the
+ original draft of this workshop paper is still available for online
+ browsing and download
+ ([Workshop20081017 IT-SOFT 2008 Workshop Paper Draft])_
---------------------------------------------------------------------------
@@ -67,7 +69,11 @@
* *Logging Components*
* *Forensics Components*
* *Experimentation*
+ * *Test Configuration*
+ * *Performance & Availability Benchmarks*
+ * *Functional Tests*
* *Results*
+ * *Notes on the Lemona Results*
* *Coverage*
* *Performance*
* *Conclusion*
@@ -941,17 +947,7 @@
*Lemona* is a compound of several components (Figure
[http://lemona.googlecode.com/svn/docs/images/architecture/lemona-architecture-5.png
4] and Figure 5), separated in three categories:
-{{{ TODO: get diagram from PDF version here }}}
-
- * Monitoring Components
- * Kernel Patches
- * Loadable Driver Modules
- * Logging Components
- * Database Servers
- * Logging Servers
- * Forensics Components
- * Database-Querying Applications
- * Data Mining Tools
+http://lemona.googlecode.com/svn/docs/images/architecture/lemona-architecture-components.png
=== Monitoring Components ===
@@ -1053,18 +1049,19 @@
== Experimentation ==
-==== Test Configuration ====
+=== Test Configuration ===
The results have been gathered using the following configuration:
- * CPU: Intel(R) Pentium(R) M processor 1.80GHz
- * RAM: 1GB
- * O.S: Linux Gentoo
- * CFLAGS: -O2 -march=pentium-m -pipe -fomit-frame-pointer
- * VM: VirtualBox-2.0.4
- * Guest O.S: Linux Gentoo
- * Base Memory: 256MB
- * Video Memory: 16MB
- * Network: PCnet-FAST III (host interface, tap0)
+
+ * *CPU:* Intel(R) Pentium(R) M processor 1.80GHz
+ * *RAM:* 1GB
+ * *OS:* GNU/Linux Gentoo
+ * *CFLAGS:* -O2 -march=pentium-m -pipe -fomit-frame-pointer
+ * *VM:* VirtualBox-2.0.4
+ * *Guest OS:* Linux Gentoo
+ * *Base Memory:* 256MB
+ * *Video Memory:* 16MB
+ * *Network:* PCnet-FAST III (host interface, tap0)
There are two different batteries of tests to demonstrate the capabilities
of the *Lemona* system:
@@ -1072,7 +1069,7 @@
* Performance & Availability Benchmarks
* Functional Tests
-==== Performance & Availability Benchmarks ====
+=== Performance & Availability Benchmarks ===
These benchmarks aim to demonstrate the usability of *Lemona* in a
test environment, by proving that its performance impact does not render
@@ -1115,13 +1112,12 @@
* average and peak log file sizes
* average and total number of database entries
-==== Functional Tests ====
+=== Functional Tests ===
These tests aim to demonstrate the functional feasibility of *Lemona*
within a real-life environment. Using real case scenarios (yet to be
determined), the application will be tested to assert:
-
* *Lemona*'s capacity at collecting relevant information
Information relative to security breaches or system failures should
@@ -1145,21 +1141,81 @@
== Results ==
- NOTE: As of this writing, results are still being collected. We are
- just providing below some insights on our expectations and our
- checkpoints.
+=== Notes on the *Lemona* Results ===
+
+As of this writing, results are still being collected. We are just
+providing below some insights on our expectations and our
+checkpoints, as well as the current results we managed to collect.
+
+Currently, we have not collected the results for the functional tests
+outlined above, as *Lemona* is not feature complete from the computer
+forensics point of view, and we did not perform tests with an active
+encryption layer, as it is not yet part of our proof of concept.
+
+Regarding the benchmark comparison with *Forensix*, we would like to
+emphasize that they might not be easy to compare with a 1-to-1 ratio,
+for the following reasons:
+
+ * Coverage
+
+ *Forensix* does not record every single operation, and only focuses
+ on specific processes, whereas *Lemona* aims at monitoring the
+ complete system activity. On the other hand, *Lemona* does not
+ (yet) monitor all the _system calls_; but the ones implemented are
+ monitored for all processes.
+
+ * Hardware and Software Configurations
+
+ We do not have at our disposal the hardware resources to perform
+ tests matching these of *Forensix*. Therefore, we use different
+ hardware and system configurations, and our tests were run within a
+ _virtual machine_.
+
+ * Tests Replicability
+
+ *Lemona* and *Forensix* are compared based on similar actions, but
+ with different inputs. For instance, they use different kernel
+ sources for the performance test, which means that the kernel
+ compilation time might not be as relevant.
+
+However, we believe these benchmarks provide a valuable insight and
+indication of *Lemona*'s usability on a real system, and that the
+transparence we provide with this comparison *Forensix* might allow
+readers to determine if our solution matches their needs.
+
+=== Coverage ===
+
+We intend to cover close to 100% of a system's activity, in that we
+actually trace all existing _system calls_ and monitor memory mapped
+areas' read and write accesses. We should therefore be able to monitor
+all actions executed by all (human and machine) users logged onto a
+system as the system's core activity. We expect to be able to collect
+enough data to virtually reconstruct a complete system from a given
+checkpoint.
+
+Our proof of concept achieves a technical validation of this
+objective, by monitoring twenty-two (22) _system calls_, among which
+some of the most intensive ones, performing I/O operations.
+
+=== Performance ===
+
+==== CPU ====
=== Linux Kernel 2.6.26.3 Compilation ===
The kernel has been compiled using the configuration file found in
-arch/x86/configs/i386_defconfig that come along its sources.
+_arch/x86/configs/i386_defconfig_ that comes along with its sources.
+
+Between each test:
+
+ * the virtual machine has been reset
+ * the 'old' Linux directory has been deleted and copied over again.
-Between each test, the virtual machine have been reset, the old linux
-directory deleted and copied over again. 'make menuconfig' have been
-executed before launching the compilation, no modification has been
-made through it, we simply ask him to save the .config for us.
+For each test _make menuconfig_ has been executed before launching the
+compilation. No modification has been made to the default
+configuration, we simply have him save the _.config_ file.
-The command used was 'time make'.
+We use the _time make_ command to produce the following measurements.
|| *Test* || *User Time* || *System Time* || *Real
Time* || *CPU* ||
|| without lemona || 283s || 226s ||
519s || 98% ||
@@ -1168,6 +1224,12 @@
|| lemona + relay || 300s || 292s || 604s
(+16.37%) || 98% ||
|| lemona + relay + cat || 247s || 256s || 1028s
(+98.07%) || 49% ||
+As we predicted, we notice a significant impact on the system's
+performance when *Lemona* is enabled. However, and even though
+*Lemona* performs worse than *Forensix* (See Appendix A), our proof of
+concept remains usable inspite of the overhead.
+
+==== Network ====
=== Apache 2 Benchmark ===
@@ -1201,33 +1263,6 @@
|| lemona + relay + cat || 6.700s (+16.17%) || 1492.50 (-13.92%) ||
591.75Kb (-13.92%) ||
-=== Coverage ===
-
-We intend to cover close to 100% of a system's activity, in that we
-actually trace all existing _system calls_ and monitor memory mapped
-areas' read and write accesses. We should therefore be able to monitor
-all actions executed by all (human and machine) users logged onto a
-system as the system's core activity. We expect to be able to collect
-enough data to virtually reconstruct a complete system from a given
-checkpoint.
-
-=== Performance ===
-
-==== CPU ====
-
-Though results are not available yet, we expect a significant impact
-on the monitored system's performance. The related projects have
-experienced similar difficulties, but remained usable. However they
-did not use the same accuracy and granularity as *Lemona*. Our reference
-test will be similar to the one presented by *Forensix* `[9]`, which uses
-a web-server with medium load to produce benchmarks. We expect to have
-performance impacts double as aggressive than *Forensix*. This will of
-course also depend on the system's settings, and the activation of
-optional modules (for instance, the encryption of the reported
-traces).
-
-==== Network ====
-
As for the CPU consumption, the network resources take a significant
hit if *Lemona* is configured to transmit its traces over the
network. Considering *Lemona* will be transmitting all the traces over a
@@ -1409,6 +1444,14 @@
= Appendices =
+
+== Appendix A ==
+
+Performance Results published by *Forensix* `[9]` for a
+
+|| || Auditing Off || Auditing on Network Off || Auditing on
Network On ||
+|| total time || 232.2s || 247.1s (6%) || 252.0s
(8%) ||
+|| system Time|| 14.0s || 26.3 ||
30.7s ||
---------------------------------------------------------------------------