Load pcap file in Hogzilla

[Nada Essaouini]

Hello,

I' m trying to use hogzilla to evaluate it using data in pcap format.

I know that Hogzilla takes sflows as input using sflowtools. Can I input my pcap files instead?

Thanks for your answers.

Regards,

[Paulo Angelo]

Hi Nada,

In this case, I suggest you to replay the pcap files and collect the sflows to approximate to a real scenario. There are some tools used to replay traffic, one is  tcpreplay, but there are other alternatives.

I also recommend you to make use of known available datasets. In this direction, I mention the datasets below.

• CICIDS2017
• CTU-13
• ISCX2012

best regards,

Paulo Angelo

[Nada Essaouini]

Hi Paulo

Thank you for your reply. I plan to use CIC-IDS 2017 data for the evaluation.
I m using Host Sflows to turn PCAP into sflows and it works very well.

hbase(main):002:0> count 'hogzilla_sflows'
Current count: 1000, row: 10.0.3.15.1529570011.0.1898                                                                                                                                                                   
Current count: 2000, row: 10.0.3.15.1529570011.0.548                                                                                                                                                                    
2500 row(s) in 1.5680 seconds

=> 2500

However I get failed status in /tmp/log/hogzilla when runing hogzilla:

18/06/21 10:16:28 INFO yarn.Client: Application report for application_1529568791673_0001 (state: FINISHED)
18/06/21 10:16:28 INFO yarn.Client:
     client token: N/A
     diagnostics: N/A
     ApplicationMaster host: 192.168.21.89
     ApplicationMaster RPC port: 0
     queue: default
     start time: 1529568884675
     final status: FAILED
     tracking URL: http://HOGZILLA:8088/proxy/application_1529568791673_0001/
     user: hogzilla
Exception in thread "main" org.apache.spark.SparkException: Application application_1529568791673_0001 finished with failed status
    at org.apache.spark.deploy.yarn.Client.run(Client.scala:1132)
    at org.apache.spark.deploy.yarn.Client$.main(Client.scala:1175)
    at org.apache.spark.deploy.yarn.Client.main(Client.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:736)
    at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:185)
    at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:210)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:124)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
18/06/21 10:16:28 INFO util.ShutdownHookManager: Shutdown hook called
18/06/21 10:16:28 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-1953bdf3-53bd-4733-aecc-94b49c5aa336