Unable to install Bulk Extractor in Ubuntu 20.04

1,412 views
Skip to first unread message

PC Houseboy

unread,
Jun 17, 2021, 12:20:26 AM6/17/21
to bulk_extractor-users
I tried both "bulk_extractor-1.5.5" and "bulk_extractor-rec-master". "./configure" no errors, but "make" or "sudo make" give error below. Also, "dpkg --list" does not show installed. Thank you.

ake[2]: *** [Makefile:1243: bulk_extractor_api.o] Error 1
make[2]: Leaving directory '/home/MyUserName/Software/bulk_extractor-1.5.5/src'
make[1]: *** [Makefile:485: all-recursive] Error 1
make[1]: Leaving directory '/home/MyUserName/Software/bulk_extractor-1.5.5'
make: *** [Makefile:424: all] Error 2

"

Simson Garfinkel

unread,
Jun 17, 2021, 7:40:01 AM6/17/21
to bulk_extra...@googlegroups.com
Hi. Thanks for your bug report.
Some questions:

1 - Where did you get 1.5.5 from? Did you install it from github, or download the installer?
2 - Please post the error message. You clearly are missing a lot of the make output.
3 - We are developing BulkExtractor 2.0 with Ubuntu 20.04 as one of the target deployment environments. Would you be willing to test or even contribute to the development effort?


--
You received this message because you are subscribed to the Google Groups "bulk_extractor-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bulk_extractor-u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bulk_extractor-users/8eeb74f9-4429-4a04-8290-9a882fd46897n%40googlegroups.com.

PC Houseboy

unread,
Jun 18, 2021, 1:00:31 AM6/18/21
to bulk_extractor-users
Thank you for your prompt reply.

1. I used http://digitalcorpora.org/downloads/bulk_extractor/  "bulk_extractor-1.5.5.tar.gz"
2. I will be on my Ubuntu20.04 tomorrow and I will recreate the "make" and "sudo make install" for the full error
3. I would like to test BulkExtractor 2.0 on my Ubuntu 20.04. I am not a programmers so I would be unable to contribute to the development

I also tried "sudo apt-get -y install gcc g++ flex libewf-dev libssl-dev" before the
./configure
make
make install

I may also try installing https://github.com/bitcurator/bitcurator-distro-installer, but I would like to install the BulkExtractor 2.0 first.

Simson Garfinkel

unread,
Jun 18, 2021, 1:05:02 AM6/18/21
to bulk_extra...@googlegroups.com
Thanks for your email. Given your situation, I recommend using Ubuntu 18.04 and waiting until bulk_extractor 2.0 is available before trying to use bulk_extractor on Ubuntu 20.04.

PC Houseboy

unread,
Jun 18, 2021, 9:56:12 PM6/18/21
to bulk_extractor-users
Good advice; I will wait for  bulk_extractor 2.0.

Derrick Karpo

unread,
Jun 19, 2021, 4:16:15 PM6/19/21
to bulk_extra...@googlegroups.com
For what it's worth, the latest master branch of bulk_extractor 1.6.0
from Github (https://github.com/simsong/bulk_extractor) compiles clean
under the latest Debian unstable so it should compile clean under
Ubuntu 20.04. You might want to give that version a shot in the
interim.

The release-2.0-dev branch doesn't compile as it appears some of the
dig.*, findopts.*, and be13_api stuff is in transition, likely for the
bulk_extractor 2.0 build that Simpson is working on!
> To view this discussion on the web visit https://groups.google.com/d/msgid/bulk_extractor-users/58c9fe29-1481-49ee-a67b-d26530a42c16n%40googlegroups.com.

Simson Garfinkel

unread,
Jun 19, 2021, 4:33:37 PM6/19/21
to bulk_extra...@googlegroups.com, Derrick Karpo
The transition is huge and I’m now slowly getting getting BE2.0 operational. 
It’s easier to follow the slg-dev branch.
I’m almost at the point that non-developer testers are useful, but not quite.

be13_api is now covered 57% by unit tests (it’s also been highly refactored)

I’ll push from slg-dev to the release-2.0-dev branch shortly.
We’re also now using GitHub actions for CI.
Today’s big challenge is to get jpeg carving working again.

The goal of the rewrites are here:

Goals for the 2.0 release

  • Production quality. Version 1.x of bulk_extractor was a research tool that also found usefuleness in operational settings. Verison 2.x is a production tool. As such:

    • Research scanners have been removed from the master branch. They can still be researched by making them shared-libraries and using the bulk_extractor plug-in system.

    • Unit tests have been added.

  • Improved software development practices.

    • Continious integration is employed to validate each commit.

    • Development will take place in feature branches which will be added to the master branch only if CI tests pass.

  • Sensible defaults for production operation. With the undertanding that most users do not understand command-line options, bulk_extractor now runs with fewer command-line options.

  • Standards-compliant. Where possible, we are adopting C++14 features that are now widely available.

  • Experimental features have been removed. Experiments are now conducted with plugin-s.

  • BE2.0 will be released as a pure command-line tool. The user interface with the windows installer (and embedded CLI) will be released afterwards.

  • SQL will be turned on by default and the program will provide the user with instructions on how to use it. Performance will be analyzed to determine the fastest way to create the text feature files, the SQLite3 database, and the histograms.

  • Include other easy-to-output feature files by default, such as collect all email messages.

  • Integration with The Sleuth Kit for file enumeration


PC Houseboy

unread,
Jun 19, 2021, 9:59:17 PM6/19/21
to bulk_extractor-users
I will try 1.6.0; Newer releases can man a big difference.

PC Houseboy

unread,
Jun 24, 2021, 6:28:51 PM6/24/21
to bulk_extractor-users
I got Bulk Extractor 1.6.0 to install on my Ubuntu 20.04. It ran great creating a WordList on a one TB Win Server 2016 image. I will be on the lookout for BE 2.0. Thanks again for your input.

$ git clone --recursive https://github.com/simsong/bulk_extractor.git
$ cd bulk_extractor
~/bulk_extractor$ sh bootstrap.sh
~/bulk_extractor$ ./configure
~/bulk_extractor$ make
~/bulk_extractor$ make install

Simson Garfinkel

unread,
Jun 24, 2021, 9:43:50 PM6/24/21
to bulk_extra...@googlegroups.com
Great!


----
Sent from my phone.

On Jun 24, 2021, at 6:28 PM, PC Houseboy <pchou...@gmail.com> wrote:



PC Houseboy

unread,
Oct 26, 2021, 12:32:59 AM10/26/21
to bulk_extra...@googlegroups.com

Scenario: Win Server 2019 using bulk_extractor V 1.6.0; Ubuntu 20.04 using bulk_extractor V Beta 2.0.0. Image is Averatech IDF .dd 80 GB total size
No file level comparison yet due to time constraints

AveratechIDF Win server 2019 V1.6.0. Same .dd 80 GB total size
No Folders.Files listed not 0 bytes

Files (10):
domain.txt        2.9 KB
domain_histogram.txt    348 Bytes
elf.txt        630.4 KB
Report.xml        15 KB
rfc822.txt        552 Bytes
url.txt        6.3 KB
url_histogram.txt    1.1 KB
url_services.txt    345 Bytes
windirs.txt        5.8 KB
winpe.txt        55.1 KB

Averatec Ubuntu 20.04 Beta 2.0.0
No Folders or Files listed not 0 bytes
Folders (2):    winpe_carved/000 with 10 files (DOS/Windows executable (application- n/x-ms-dos-executable)
                       Program (application/octet-stream)

Files (11):
domain.txt        2.9 KB
domain_histogram.txt    336 Bytes
elf.txt        632.1 KB
report.xml        17.3 KB       
rfc822.txt        542 Bytes
url.txt        6.3 KB
url_histogram.txt    1.1 KB
url_services.txt    354 Bytes
windirs.txt        5.8 KB
winpe.txt        55.1 KB
winpe_carved.txt    2.7 KB

===============================================

Simson Garfinkel

unread,
Oct 29, 2021, 11:35:37 PM10/29/21
to PC Houseboy, bulk_extra...@googlegroups.com
Hi. I’m now able to focus on this a bit more. Are you reporting a problem?

Message has been deleted

Simson Garfinkel

unread,
Oct 30, 2021, 2:34:00 PM10/30/21
to bulk_extra...@googlegroups.com
Ah, okay. There is a program called bulk_diff in the python directory that is designed to help you in this situation. It does not do performance comparisons, but does compare differences in the feature files.



----
Sent from my phone.

On Oct 30, 2021, at 2:31 PM, SteveAreno <pchou...@gmail.com> wrote:

No, I am not reporting a problem.  My subject line "Version 1.6.0 Verse Beta 2.0.0 Bake Off" was a comparison test between V 1.6.- and V Beta 2.0.0. The upshot is the Beta 2.0.0 pulled more results, including two folders, which V 1.6.0 did not. Beta 2.0.0 also had more scanner files with contents. My impression is Beta 2.0.0 is an improved product. I listed the two OSs (Win Server 2019 and Linux 20.04) as detail. Perhaps I should have started a new conversation for this.
Reply all
Reply to author
Forward
Message has been deleted
0 new messages