Bulk Extractor

53 views
Skip to first unread message

Rodney Swaner

unread,
May 10, 2016, 11:11:42 AM5/10/16
to BitCurator Users
Kam or Cal, 

I ran Bulk Extractor on a disk image about 88 GB. I started the process around 3pm yesterday and left it to run over night. When I came in this AM and it was only %40 done. Is that typical? At a visual glance I noticed just wave files on the hard drive. I guess lessening the amount of scanners would make it run faster? Thanks
Screenshot from 2016-05-10 10_15_34.png

Donald Mennerich

unread,
May 10, 2016, 11:37:25 AM5/10/16
to bitcurat...@googlegroups.com
If your running BC in a VM with a minimal amount of CPU/RAM, then no it's not completely surprising. BE was designed to run on multicore processors, there is a performance boost that is noticeable when you run it on dedicated hardware. What is your setup like?

Donald R. Mennerich, digital archivist
New York University Libraries

On Tue, May 10, 2016 at 11:11 AM, Rodney Swaner <rsw...@utah.gov> wrote:
Kam or Cal, 

I ran Bulk Extractor on a disk image about 88 GB. I started the process around 3pm yesterday and left it to run over night. When I came in this AM and it was only %40 done. Is that typical? At a visual glance I noticed just wave files on the hard drive. I guess lessening the amount of scanners would make it run faster? Thanks

--
You received this message because you are subscribed to the Google Groups "BitCurator Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bitcurator-use...@googlegroups.com.
To post to this group, send email to bitcurat...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bitcurator-users/da506203-b65b-4bbd-95ed-56f974aab070%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Rodney Swaner

unread,
May 10, 2016, 12:24:59 PM5/10/16
to bitcurator-users
I am running BC in a VM and I did not adjust any settings. We are trying to write up a workflow using BitCurator and finding that some processes are time consuming. I installed BC in its own environment but found problems with that type of install too. Thanks 

Rodney Swaner
Digital Archivist 
B.S, M.S. and SAA DAS Certification
The State Archives hours of operations are Monday-Friday, 8:00 a.m. to 5:00 p.m.

To view this discussion on the web visit https://groups.google.com/d/msgid/bitcurator-users/CAD067EnR%2BEoiBNj4fpMpf5k0%2BaEg%3D01P02zdYSSQKKFULn6YXw%40mail.gmail.com.

Kam Woods

unread,
May 10, 2016, 12:50:06 PM5/10/16
to bitcurat...@googlegroups.com
That's your problem, then. The VM is distributed with 1575MB RAM and 1 processor assigned. This is to allow people using completely outdated hardware to try out the environment. It should never, ever be used in production settings this way. 

Please see the QuickStart guide for the same justification and a step-by-step guide that will show you how to adjust the settings.

Hardware-wise, if the VM is a better fit for you than the installed environment, and you would like to process larger materials, I would suggest running it on a machine with 64GB RAM or more, everything on solid state drives, and at least four processor cores assigned to the VM.

Kam



To view this discussion on the web visit https://groups.google.com/d/msgid/bitcurator-users/CABE3DHcw9LaP_jK%3Dqt9oNOeDe1n44UOxVTYa_-D5iTq1MwhAcg%40mail.gmail.com.

Rodney Swaner

unread,
May 10, 2016, 1:11:09 PM5/10/16
to bitcurator-users
Thanks Kam. Now I just have to get that approved for the next budget. That is what I thought. 

Rodney Swaner
Digital Archivist 
B.S, M.S. and SAA DAS Certification
The State Archives hours of operations are Monday-Friday, 8:00 a.m. to 5:00 p.m.

To view this discussion on the web visit https://groups.google.com/d/msgid/bitcurator-users/CAAOjFxCFHNn_xEXPgpYd6gLXFU972aXYgcuHH93u09-2gtkFVw%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages