Transfer source displays nothing with large directories

[Joseph Collard]

Under the transfer tab, I have a directory that shows no items to transfer. If edit the location to one of the subdirectories, it works fine.  My theory is that the root directory is too large (almost 2 terabytes), causing a timeout (logs below)  I don't believe it is a permission error, as the directory, subdirectory, and all files are set to 775, and the archivematica user is part of the group that is owner of the directory. Anyone have an idea how to deal with this? Thanks.

==> /var/log/archivematica/dashboard/dashboard.log <==
  File "/usr/share/archivematica/virtualenvs/archivematica/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/share/archivematica/virtualenvs/archivematica/lib/python3.6/site-packages/urllib3/connectionpool.py", line 756, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/usr/share/archivematica/virtualenvs/archivematica/lib/python3.6/site-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/share/archivematica/virtualenvs/archivematica/lib/python3.6/site-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/usr/share/archivematica/virtualenvs/archivematica/lib/python3.6/site-packages/urllib3/connectionpool.py", line 706, in urlopen
    chunked=chunked,
  File "/usr/share/archivematica/virtualenvs/archivematica/lib/python3.6/site-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/share/archivematica/virtualenvs/archivematica/lib/python3.6/site-packages/urllib3/connectionpool.py", line 337, in _raise_timeout
    self, url, "Read timed out. (read timeout=%s)" % timeout_value
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='127.0.0.1', port=8000): Read timed out. (read timeout=5.0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/archivematica/virtualenvs/archivematica/lib/python3.6/site-packages/django/core/handlers/exception.py", line 41, in inner
    response = get_response(request)
  File "/usr/share/archivematica/virtualenvs/archivematica/lib/python3.6/site-packages/django/core/handlers/base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/usr/share/archivematica/virtualenvs/archivematica/lib/python3.6/site-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/share/archivematica/dashboard/components/filesystem_ajax/views.py", line 125, in directory_children_proxy_to_storage_server
    response = storage_service.browse_location(location_uuid, path)
  File "/usr/lib/archivematica/archivematicaCommon/storageService.py", line 275, in browse_location
    response = _storage_api_session().get(url, params=params)
  File "/usr/share/archivematica/virtualenvs/archivematica/lib/python3.6/site-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/share/archivematica/virtualenvs/archivematica/lib/python3.6/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/share/archivematica/virtualenvs/archivematica/lib/python3.6/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/archivematica/archivematicaCommon/storageService.py", line 75, in send
    return super(HTTPAdapterWithTimeout, self).send(*args, **kwargs)
  File "/usr/share/archivematica/virtualenvs/archivematica/lib/python3.6/site-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPConnectionPool(host='127.0.0.1', port=8000): Read timed out. (read timeout=5.0)

[Joseph Collard]

I figured out the issue.  I'm guessing that due to network lag, the dashboard timeout when it didn't get object counts quickly enough.  Enabling the " Object counting in spaces is disabled?" stops this from happening. On the plus side, I can see all the directories and objects.  The downside is that I no longer see the object count.