Need help troubleshooting four program installs (EyeWitness/theHarvester/pipenv/kazam) for Fedora based OSINT VM
Chrome
#######################################################################
# EyeWitness Setup #
#######################################################################
[Error]: \S is not supported by this setup script.
[user@OSINT-Template setup]$
2. Fix theHarvester
WARNING: Running pip install with root privileges is generally not a good idea. Try `python3 -m pip install --user` instead.
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)")': /simple/aiodns/
ERROR: Could not find a version that satisfies the requirement aiodns==2.0.0 (from -r requirements/base.txt (line 1)) (from versions: none)
ERROR: No matching distribution found for aiodns==2.0.0 (from -r requirements/base.txt (line 1))
[user@OSINT-Template theHarvester]$ sudo -H python3 -m pip install -r requirements.txt
WARNING: Running pip install with root privileges is generally not a good idea. Try `python3 -m pip install --user` instead.
Collecting aiodns==2.0.0
ERROR: HTTP error 502 while getting https://files.pythonhosted.org/packages/da/01/8f2d49b441573fd2478833bdba91cf0b853b4c750a1fbb9e98de1b94bb22/aiodns-2.0.0-py2.py3-none-any.whl#sha256=aaa5ac584f40fe778013df0aa6544bf157799bd3f608364b451840ed2c8688de (from https://pypi.org/simple/aiodns/)
ERROR: Could not install requirement aiodns==2.0.0 from https://files.pythonhosted.org/packages/da/01/8f2d49b441573fd2478833bdba91cf0b853b4c750a1fbb9e98de1b94bb22/aiodns-2.0.0-py2.py3-none-any.whl#sha256=aaa5ac584f40fe778013df0aa6544bf157799bd3f608364b451840ed2c8688de (from -r requirements/base.txt (line 1)) because of error 502 Server Error: Gateway Error for url: https://files.pythonhosted.org/packages/da/01/8f2d49b441573fd2478833bdba91cf0b853b4c750a1fbb9e98de1b94bb22/aiodns-2.0.0-py2.py3-none-any.whl
ERROR: Could not install requirement aiodns==2.0.0 from https://files.pythonhosted.org/packages/da/01/8f2d49b441573fd2478833bdba91cf0b853b4c750a1fbb9e98de1b94bb22/aiodns-2.0.0-py2.py3-none-any.whl#sha256=aaa5ac584f40fe778013df0aa6544bf157799bd3f608364b451840ed2c8688de (from -r requirements/base.txt (line 1)) because of HTTP error 502 Server Error: Gateway Error for url: https://files.pythonhosted.org/packages/da/01/8f2d49b441573fd2478833bdba91cf0b853b4c750a1fbb9e98de1b94bb22/aiodns-2.0.0-py2.py3-none-any.whl for URL https://files.pythonhosted.org/packages/da/01/8f2d49b441573fd2478833bdba91cf0b853b4c750a1fbb9e98de1b94bb22/aiodns-2.0.0-py2.py3-none-any.whl#sha256=aaa5ac584f40fe778013df0aa6544bf157799bd3f608364b451840ed2c8688de (from https://pypi.org/simple/aiodns/)
[user@OSINT-Template theHarvester]$
WARNING: Running pip install with root privileges is generally not a good idea. Try `python3 -m pip install --user` instead.
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ReadTimeoutError("HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15)")': /simple/aiodns/
ERROR: Could not find a version that satisfies the requirement aiodns==2.0.0 (from -r requirements/base.txt (line 1)) (from versions: none)
ERROR: No matching distribution found for aiodns==2.0.0 (from -r requirements/base.txt (line 1))
[user@OSINT-Template theHarvester]$ sudo -H python3 -m pip install -r requirements.txt
WARNING: Running pip install with root privileges is generally not a good idea. Try `python3 -m pip install --user` instead.
Collecting aiodns==2.0.0
ERROR: HTTP error 502 while getting https://files.pythonhosted.org/packages/da/01/8f2d49b441573fd2478833bdba91cf0b853b4c750a1fbb9e98de1b94bb22/aiodns-2.0.0-py2.py3-none-any.whl#sha256=aaa5ac584f40fe778013df0aa6544bf157799bd3f608364b451840ed2c8688de (from https://pypi.org/simple/aiodns/)
ERROR: Could not install requirement aiodns==2.0.0 from https://files.pythonhosted.org/packages/da/01/8f2d49b441573fd2478833bdba91cf0b853b4c750a1fbb9e98de1b94bb22/aiodns-2.0.0-py2.py3-none-any.whl#sha256=aaa5ac584f40fe778013df0aa6544bf157799bd3f608364b451840ed2c8688de (from -r requirements/base.txt (line 1)) because of error 502 Server Error: Gateway Error for url: https://files.pythonhosted.org/packages/da/01/8f2d49b441573fd2478833bdba91cf0b853b4c750a1fbb9e98de1b94bb22/aiodns-2.0.0-py2.py3-none-any.whl
ERROR: Could not install requirement aiodns==2.0.0 from https://files.pythonhosted.org/packages/da/01/8f2d49b441573fd2478833bdba91cf0b853b4c750a1fbb9e98de1b94bb22/aiodns-2.0.0-py2.py3-none-any.whl#sha256=aaa5ac584f40fe778013df0aa6544bf157799bd3f608364b451840ed2c8688de (from -r requirements/base.txt (line 1)) because of HTTP error 502 Server Error: Gateway Error for url: https://files.pythonhosted.org/packages/da/01/8f2d49b441573fd2478833bdba91cf0b853b4c750a1fbb9e98de1b94bb22/aiodns-2.0.0-py2.py3-none-any.whl for URL https://files.pythonhosted.org/packages/da/01/8f2d49b441573fd2478833bdba91cf0b853b4c750a1fbb9e98de1b94bb22/aiodns-2.0.0-py2.py3-none-any.whl#sha256=aaa5ac584f40fe778013df0aa6544bf157799bd3f608364b451840ed2c8688de (from https://pypi.org/simple/aiodns/)
[user@OSINT-Template theHarvester]$
3. Fix pipenv
[user@OSINT-Template theHarvester]$ sudo -H python3 -m pip install pipenv
WARNING: Running pip install with root privileges is generally not a good idea. Try `python3 -m pip install --user` instead.
Collecting pipenv
ERROR: HTTP error 502 while getting https://files.pythonhosted.org/packages/c1/a3/d266421362565864f130cb97f55f70c763b843c9a67311d215d75b7ec464/pipenv-2021.5.29-py2.py3-none-any.whl#sha256=d180f5be4775c552fd5e69ae18a9d6099d9dafb462efe54f11c72cb5f4d5e977 (from https://pypi.org/simple/pipenv/) (requires-python:>=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*)
ERROR: Could not install requirement pipenv from https://files.pythonhosted.org/packages/c1/a3/d266421362565864f130cb97f55f70c763b843c9a67311d215d75b7ec464/pipenv-2021.5.29-py2.py3-none-any.whl#sha256=d180f5be4775c552fd5e69ae18a9d6099d9dafb462efe54f11c72cb5f4d5e977 because of error 502 Server Error: Gateway Error for url: https://files.pythonhosted.org/packages/c1/a3/d266421362565864f130cb97f55f70c763b843c9a67311d215d75b7ec464/pipenv-2021.5.29-py2.py3-none-any.whl
ERROR: Could not install requirement pipenv from https://files.pythonhosted.org/packages/c1/a3/d266421362565864f130cb97f55f70c763b843c9a67311d215d75b7ec464/pipenv-2021.5.29-py2.py3-none-any.whl#sha256=d180f5be4775c552fd5e69ae18a9d6099d9dafb462efe54f11c72cb5f4d5e977 because of HTTP error 502 Server Error: Gateway Error for url: https://files.pythonhosted.org/packages/c1/a3/d266421362565864f130cb97f55f70c763b843c9a67311d215d75b7ec464/pipenv-2021.5.29-py2.py3-none-any.whl for URL https://files.pythonhosted.org/packages/c1/a3/d266421362565864f130cb97f55f70c763b843c9a67311d215d75b7ec464/pipenv-2021.5.29-py2.py3-none-any.whl#sha256=d180f5be4775c552fd5e69ae18a9d6099d9dafb462efe54f11c72cb5f4d5e977 (from https://pypi.org/simple/pipenv/) (requires-python:>=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*)
4. Fix kazam
[user@OSINT-Template kazam-1.4.4]$ sudo python3 setup.py install
Traceback (most recent call last):
File "setup.py", line 8, in <module>
from DistUtilsExtra.command import *
ModuleNotFoundError: No module named 'DistUtilsExtra'
[user@OSINT-Template kazam-1.4.4]$
unman
>
> Good Morning again,
>
> I got the OSINT VM about 95% setup. Thank you all for your help on it. I
> ran into problems during the install of a few recommended tools in the Mike
> Bazzell OSINT manual. These programs are as follows: EyeWitness,
> theHarvester, pipenv, and kazam.
>
> Kazam seems like something I don't actually need but I'd still like all
> tools set up. Below is the terminal text I saved in a "todo" txt file. Any
> help understanding the error messages and what I need to do to resolve them
> would be appreciated. I recognize everyone on here is quite busy but this
> n00b would definitely appreciate the help. Thank you.
Standalone, or template based qube.
>
> =========================================================================
> 1. Fix Eyewitness
> #######################################################################
> # EyeWitness Setup #
> #######################################################################
>
> [Error]: \S is not supported by this setup script.
not supported.
>
> [user@OSINT-Template setup]$
>
> 2. Fix theHarvester
You have repeated error 502 - this is "Bad Gateway"- change your routing
to these sites.
You do not have right python installed - the requirement is specific -
>=2.7, but less than 3.0.
Install and configure your python(s) as necessary.
> 4. Fix kazam
> [user@OSINT-Template kazam-1.4.4]$ sudo python3 setup.py install
> Traceback (most recent call last):
> File "setup.py", line 8, in <module>
> from DistUtilsExtra.command import *
> ModuleNotFoundError: No module named 'DistUtilsExtra'
> [user@OSINT-Template kazam-1.4.4]$
>
install python-distutils-extra or python3-distutils-extra , depending on
your python version.
Chrome
Chrome
Floyd: If you see this, let me know if you have any feedback. It would definitely be appreciated since you commented on my posts before and have done something similar to what I'm trying to do re: OSINT VM's on Fedora. I know you settled on debian but I just want to finally call this setup complete and get on with it, ya know? Thanks again all.
Floyd
--
I found my notes - basically a print out of the install document Bazzell provided on his website, IIRC. I printed it from an editor that provided line numbers (notepad++ ?). I also used info from the 7th edition of his book, which is earlier than the one you're using IIRC.
I don't see any notes regarding issues with python3-pip or
installing Python in general. I used the Qubes generic Debian 10
template and cloned it before making the mods.
I do see a note about needing a git account (which I do not have,
I don't think) for EyeWitness. My notes indicate I reached a
section of Bazzells notes that involved git cloning - Sublist3r,
Photon, theHarvester - when I apparently stopped. Huh. My template
shows EyeWitness installed, but it does not run. Maybe that one
did not need an account. Looking at some of the time stamps in the
template it's been over a year since I worked with the template
and almost as long with the test case VM. In fact, there are
enough of the apps I did install that now do not work that leads
me to think I may have an issue with the template. I use Qubes as
my daily driver for everything unless I absolutely need a
proprietary Windows app. Everything else is working.
My notes indicate I stopped after the install of "amass" - whatever that is.
I also did not do any of the steps in his Updates section. Based solely on the numbered lines of the printout, there are approximately 100 install steps, of which I did ~80% before stopping. The steps do NOT include anything related to installing an actual OS (Ubuntu in this case), but does include tweaking Firefox. It does not include any steps from the Updates: section. In my 7th Ed version, there do appear to be some apps that are actually initially installed in the Updates: section - Skiptracer, Sherlock and a couple of crawlers.
I found the Firefox stuff and his scripts to be useful when
working the test case. I was working with known current data, so I
had some insight. I also had data dumps from commercial databases
as well for comparison. As I stated, it has been awhile since I
played with the OSINT VM, but these tools require more hands on
that what I have given them. I also did not have any occasion to
use, other than to test, many of the tools Bazzell recommended.
That said, these are useful tools and OSINT has a place in an
investigators toolbox. Bazzell has done an excellent job of
sharing his knowledge and experience.
My *nix background is limited, especially in the desktop area. I dealt with servers, hypervisors and appliances. Some of the replies from far more knowledgeable and experienced Qubes users have been on point, detailed and educational. Qubes is a natural for OSINT I think, but most of the world runs on standalone installs or relatively uncomplicated virtualization solutions that mostly just work.
You have sparked my interest in acquiring the most current edition of his book. In my experience, Bazzell really puts a lot of work into his editions. His Privacy books are excellent. It's also timely in that my upcoming OSINT CE (continuing education) course begins in a couple of weeks. Be interesting for me to compare and contrast.
Regarding browsers, Firefox is mostly sufficient, but I prefer
others, and they are not based on Chrome. It's a question of whom
do you trust. And what browser supported the required extensions.
Regarding video CODECs, my issue is that the Firefox in Fedora didn't have them installed. Firefox ESR in Debian plays everything I've needed without installing any CODECS. In fact, I've been using a Debian VM a lot more for daily work because it just works. YMMV.
I'm giving serious thought to a fresh install of Qubes on much
larger drives. Space has become an issue and I'd like to do a
couple of things different this time.
Floyd
--
Regarding Debian vs Fedora, one consideration I made that I had
forgotten about is that Ubuntu is a Debian derivative.