Re: Cloud Resource Manager Forseti Error

[Henry Chang]

Hi JS, this seems to be related to when Forseti is trying to inventory G Suite groups.  Has this been working for you before?  Or is this a new issue?

If this is a new issue, I would suggest that you review your gsuite configuration as below.

https://forsetisecurity.org/docs/latest/configure/inventory/gsuite.html

There might be something else happening as well, with the API connection, but I like to verify that your configuration is working correctly first.

On Thu, Mar 26, 2020 at 9:31 AM JS <jina...@outlook.com> wrote:

Hello All!

Has anyone seen the following error? It appears to be with the "Cloud Resource Manager API" and the Python file. Looks like it's not closing the file properly or something. 

Forseti Version: 2.24.1

Terraform Module Version: "~>5.1.0"

Error:

2020-03-26 15:41:03,028 ERROR [forseti-security][2.24.1] google.cloud.forseti.services.inventory.base.resources(accept): Exception raised processing ResourceManagerOrganization<data="{"creationTime": "2019-04-15T19:18:29.527Z", "displayName": "test001-at-gmail.com.a.ongcp.co", "lifecycleState": "ACTIVE", "name": "organizations/308943652241", "owner": {"directoryCustomerId": "C00lspva"}}", parent_resource_type="organization", parent_resource_id="308943652241">: [Errno 9] Bad file descriptor
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/forseti_security-2.24.1-py3.6.egg/google/cloud/forseti/services/inventory/base/resources.py", line 364, in accept
    for resource in yielder.iter():
  File "/usr/local/lib/python3.6/dist-packages/forseti_security-2.24.1-py3.6.egg/google/cloud/forseti/services/inventory/base/resources.py", line 2346, in iter
    self.resource['owner']['directoryCustomerId']):
  File "/usr/local/lib/python3.6/dist-packages/forseti_security-2.24.1-py3.6.egg/google/cloud/forseti/services/inventory/base/gcp.py", line 2284, in iter_gsuite_groups
    result = self.ad.get_groups(gsuite_id)
  File "/usr/local/lib/python3.6/dist-packages/forseti_security-2.24.1-py3.6.egg/google/cloud/forseti/common/gcp_api/admin_directory.py", line 217, in get_groups
    paged_results, 'groups')
  File "/usr/local/lib/python3.6/dist-packages/forseti_security-2.24.1-py3.6.egg/google/cloud/forseti/common/gcp_api/api_helpers.py", line 87, in flatten_list_results
    for page in paged_results:
  File "/usr/local/lib/python3.6/dist-packages/forseti_security-2.24.1-py3.6.egg/google/cloud/forseti/common/gcp_api/repository_mixins.py", line 55, in list
    verb_arguments=arguments):
  File "/usr/local/lib/python3.6/dist-packages/forseti_security-2.24.1-py3.6.egg/google/cloud/forseti/common/gcp_api/_base_repository.py", line 467, in execute_paged_query
    response = self._execute(request)
  File "/usr/local/lib/python3.6/dist-packages/forseti_security-2.24.1-py3.6.egg/google/cloud/forseti/common/util/replay.py", line 169, in replay_wrapper
    return f(self, request, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/forseti_security-2.24.1-py3.6.egg/google/cloud/forseti/common/util/replay.py", line 82, in record_wrapper
    return f(self, request, *args, **kwargs)
  File "/home/ubuntu/forseti-security/.eggs/retrying-1.3.3-py3.6.egg/retrying.py", line 49, in wrapped_f
    return Retrying(*dargs, **dkw).call(f, *args, **kw)
  File "/home/ubuntu/forseti-security/.eggs/retrying-1.3.3-py3.6.egg/retrying.py", line 212, in call
    raise attempt.get()
  File "/home/ubuntu/forseti-security/.eggs/retrying-1.3.3-py3.6.egg/retrying.py", line 247, in get
    six.reraise(self.value[0], self.value[1], self.value[2])
  File "/usr/local/lib/python3.6/dist-packages/six.py", line 703, in reraise
    raise value
  File "/home/ubuntu/forseti-security/.eggs/retrying-1.3.3-py3.6.egg/retrying.py", line 200, in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "/usr/local/lib/python3.6/dist-packages/forseti_security-2.24.1-py3.6.egg/google/cloud/forseti/common/gcp_api/_base_repository.py", line 541, in _execute
    num_retries=self._num_retries)
  File "/home/ubuntu/forseti-security/.eggs/google_api_python_client-1.7.10-py3.6.egg/googleapiclient/_helpers.py", line 130, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/home/ubuntu/forseti-security/.eggs/google_api_python_client-1.7.10-py3.6.egg/googleapiclient/http.py", line 850, in execute
    method=str(self.method), body=self.body, headers=self.headers)
  File "/home/ubuntu/forseti-security/.eggs/google_api_python_client-1.7.10-py3.6.egg/googleapiclient/http.py", line 164, in _retry_request
    resp, content = http.request(uri, method, *args, **kwargs)

For example, I've had a look at the file:

/usr/local/lib/python3.6/dist-packages/forseti_security-2.24.1-py3.6.egg/google/cloud/forseti/services/inventory/base/resources.py

but I can't seem to find anything obvious. 

Forseti Config:

module "forseti" {
	source                   = "terraform-google-modules/forseti/google"
	version                  = "~>5.1.0"
	#  forseti_version          = "v2.24.0"
	gsuite_admin_email       = var.gsuite_admin_email
	domain                   = var.domain
	project_id               = var.project_id
	org_id                   = var.org_id
	storage_bucket_location  = var.region
	bucket_cai_location      = var.region
	client_region            = var.region
	client_private           = true
	server_region            = var.region
	server_private           = true
	cloudsql_region          = var.region
	cloudsql_private         = true
	network                  = var.forseti_network
	subnetwork               = var.forseti_subnetwork
	network_project          = var.project_id
	config_validator_enabled = true
	cscc_violations_enabled  = true
	cscc_source_id           = var.cscc_source_id
	forseti_run_frequency    = var.forseti_run_frequency
	manage_rules_enabled     = false
	}
	module "real_time_enforcer_roles" {
	source  = "terraform-google-modules/forseti/google//modules/real_time_enforcer_roles"
	version = "5.0.0"
	org_id  = var.org_id
	suffix  = module.forseti.suffix
	}
	module "real_time_enforcer_organization_sink" {
	source            = "terraform-google-modules/forseti/google//modules/real_time_enforcer_organization_sink"
	version           = "5.0.0"
	pubsub_project_id = var.project_id
	org_id            = var.org_id
	}
	module "real_time_enforcer" {
	source                     = "terraform-google-modules/forseti/google//modules/real_time_enforcer"
	version                    = "5.0.0"
	project_id                 = var.project_id
	org_id                     = var.org_id
	enforcer_instance_metadata = var.instance_metadata
	topic                      = module.real_time_enforcer_organization_sink.topic
	enforcer_viewer_role       = module.real_time_enforcer_roles.forseti-rt-enforcer-viewer-role-id
	enforcer_writer_role       = module.real_time_enforcer_roles.forseti-rt-enforcer-writer-role-id
	client_region              = var.region
	enforcer_region            = var.region
	network                    = var.forseti_network
	subnetwork                 = var.forseti_subnetwork
	network_project            = var.project_id
	storage_bucket_location    = var.region
	enforcer_instance_private  = var.enforcer_instance_private
	suffix                     = module.forseti.suffix
	}

Thank You in Advance!

JS

--
You received this message because you are subscribed to the Google Groups "Forseti Security Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@forsetisecurity.org.
To view this discussion on the web visit https://groups.google.com/a/forsetisecurity.org/d/msgid/discuss/34a4f8a3-ff2a-446e-9104-711206ac1276%40forsetisecurity.org.

[JS]

Hi Henry

Yes, it used to work before. Apart from introducing VPC Service Controls on the infrastructure, we have changed any of the Forseti code etc.

So it's quite strange.

Regards

Jinal