There’s a bug in collect_system_data_files that causes it to collect the parent folder of the folder it’s supposed to collect, which in some cases means trying to include most of your operating system. And it looks like there is a second bug in the Qt hooks if '/usr' if being passed to collect_system_data_files which would again drag in huge parts of your filesystem, even without the help if the 1st bug.
The 1st bug is fixed in the development version of PyInstaller and was introduced between the 3.6 and 4.0 releases so you can either upgrade:
pip install https://github.com/pyinstaller/pyinstaller/archive/develop.zip
or downgrade:
pip install pyinstaller==3.6
The second bug I’ll need to look into. Chances are no-one’s noticed.