Error building topologies

[joeyl...@gmail.com]

Good morning everyone.  My employer has asked me to look into deploying OpenSOC in our NOC as part of a new security driven initiative for our business.  On my local dev machine (running linux mint as the host), I have downloaded the OpenSOC-Vagrant source from GitHub, as well as OpenSOC-Streaming and OpenSOC-UI.  I have been successful at getting MOST of the OpenSOC project to build using Vagrant, with the topologies being the one exception.  Here is what I have done so far, and some relevant information about my build machine:

Java 1.8 installed on the host machine I am building from.

Installed Vagrant 1.7.3 from the Vagrant website (the one in the linux mint repo was too old).

Maven version is 3.0.5

Fabric version is 1.8.2

Vagrant is building VirtualBox VM's.

jre-7u79-linux-x64.rpm located in the OpenSOC-Vagrant resources folder (also tried the latest version).

Changed the version of Apache Hive in the common.sh file from "1.2.0" to "1.2.1" to fix a broken mirror link.

I had to add a few lines to the POM file in OpenSOC-Streaming to explicitly tell Maven to build with java 1.7 (it was defaulting to 1.3), this fixed a few of the earlier build errors.

Attempting to run 'fab vagrant quickstart' or 'fab vagrant postsetup' with this configuration will build everything successfully until it comes to the topologies.  The build process for the topologies throws an error complaining that Apache Thrift cannot be found.  I attempted to resolve this error by adding libthrift 0.9.2 as a dependency to the topologies POM file.  This allows the build to continue a little bit further, but it now throws an error complaining that it cannot access TUnion.

TopologyRunner.java:[483,16] error: cannot access TUnion.

Can anyone offer any advice on resolving this issue?  I have managed to get OpenSOC-UI running as well, so this is really the only part holding me back from getting this all running on my dev machine.  Once I am able to successfully build and run this project, I would be happy to contribute the details of my progress to a setup guide, which I notice doesn't seems to exist yet for this project.

[james....@gmail.com]

Hi, which beta version are you trying to build?  Try building from the beta 6 branch

[joeyl...@gmail.com]

I am currently using the BETA 6 build.  I was able to resolve the issue by removing that Thrift dependency that I manually added to the OpenSOC-Streaming POM file (see my previous post), and instead changed the storm version from 0.9.2-incubating to 0.9.4 in the same file.  Everything built fine with vagrant/fabric after that, however now I am running into another issue.  I am very new with both storm and kafka, so im not entirely sure how to describe this issue, but in the storm UI, you can see there is an error involving the kafka spout for each of the topologies.  I have attached a screenshot below to show you what I am talking about.  The screenshot is only showing the kafka spout for the pcap topology, but this error is the same for all topologies on the kafka spout.  I could definitely use some guidance on this.

[Martin Andreoni]

I think I've solved this issue.
The problem is the different version between opensoc-streaming and opensoc-vagrant.
Basically I updated the opensoc-streaming pom.xml with the same version that opensoc-vagrant\common.sh. Even I created "0.98.13-hadoop2", for flume and zookeeper as well, in opensoc-streaming/pom.xml.
Then I updated all pom.xml in the different opensoc-streaming folders. 
for ex: in opensoc-streaming/OpenSOC-Pcap_Service/pom.xml line 13, I updated with ${global_flume_version} created in opensoc-streaming/pom.xml
then I did: fab vagrant quickstart, and the topologies are running in the storm-ui.
The problem now is that topologies are not gathering any data.