forseti cli not found in client vm

[yukie sato]

Installed rc2, and everything looked OK during installation, except when I ssh into the client vm from cloud shell and tried 'forseti' command, I get command not found error.

What could go wrong during client vm setup?

[Lukas Karlsson]

I got the same issue.  I noticed in the /tmp/deployment.log that there was some sort of failure during the install.

I did a gcloud compute instances reset <foreseti-client-vm-####> and when the host came back online a few moments later, I was able to SSH in and run the client.

I had the same issue on the server.  My client was not able to connect to the server for some reason.  I SSHed into the server and noticed the service was not running and there were some errors in the /tmp/deployment.log.  After another gcloud compute instances reset, the server was working as well.

Then, when I logged into the client I was able to use the forseti cli to create a new inventory successfully.

One thing I would recommend if you have a very large GCP environment with tens of thousands of resources, you may get a lot of errors during the scan, such as inability to retrieve bucket IAM permissions, and so when you run the "inventory list" command it may be useful to grep out the warnings:

$ forseti inventory list | grep -v warnings:

id: "2018-04-18T04:57:38.735183"

schema_version: 1

count_objects: 158737

status: "CREATED"

id: "2018-04-18T06:36:17.896636"

schema_version: 1

count_objects: 158999

status: "CREATED"

id: "2018-04-18T08:36:17.636273"

schema_version: 1

count_objects: 158241

status: "CREATED"

id: "2018-04-18T10:36:17.832025"

schema_version: 1

count_objects: 157999

status: "CREATED"

id: "2018-04-18T12:36:18.619962"

schema_version: 1

count_objects: 158885

status: "CREATED"

id: "2018-04-18T13:35:23.344416"

schema_version: 1

count_objects: 156022

status: "CREATED"

id: "2018-04-18T14:36:18.146662"

schema_version: 1

count_objects: 161343

status: "CREATED"

$ forseti inventory get 2018-04-18T14:36:18.146662 | grep -v warnings:

inventory {

  id: "2018-04-18T14:36:18.146662"

  schema_version: 1

  count_objects: 161343

  status: "CREATED"

}

Otherwise, I was just paging through thousands of warnings. 

On Wed, Apr 18, 2018 at 11:54 AM, yukie sato <yukie.a...@gmail.com> wrote:

Installed rc2, and everything looked OK during installation, except when I ssh into the client vm from cloud shell and tried 'forseti' command, I get command not found error.

What could go wrong during client vm setup?

--
You received this message because you are subscribed to the Google Groups "Forseti Security Beta Testers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beta-testers+unsubscribe@forsetisecurity.org.
To post to this group, send email to beta-testers@forsetisecurity.org.
To view this discussion on the web visit https://groups.google.com/a/forsetisecurity.org/d/msgid/beta-testers/394b1a1b-d329-4773-83e7-4d2d0aa13774%40forsetisecurity.org.

--

Lukas Karlsson, Cloud Architect / Developer Advocate

Broad Institute of MIT and Harvard
75 Ames Street, 11129, Cambridge, Massachusetts 02142
karl...@broadinstitute.org, +1.6177147142

[Joe Cheuk]

Hi Yukie and Lukas,

The problem occurred because the newest version of pip (10.0.0) is not backward compatible with the version we were using before (9.0.3).

The issue has been addressed here:
https://github.com/GoogleCloudPlatform/forseti-security/pull/1432

Can you try doing a reset on your VM instance and see if that resolves the problem?

Instructions on how to reset your VM instance:
https://cloud.google.com/compute/docs/instances/restarting-an-instance

Hope that helps, please let us know if you have more questions!

Thanks,
Joe

To unsubscribe from this group and stop receiving emails from it, send an email to beta-testers...@forsetisecurity.org.
To post to this group, send email to beta-t...@forsetisecurity.org.

To view this discussion on the web visit https://groups.google.com/a/forsetisecurity.org/d/msgid/beta-testers/394b1a1b-d329-4773-83e7-4d2d0aa13774%40forsetisecurity.org.

--

Lukas Karlsson, Cloud Architect / Developer Advocate

Broad Institute of MIT and Harvard
75 Ames Street, 11129, Cambridge, Massachusetts 02142
karl...@broadinstitute.org, +1.6177147142

--

You received this message because you are subscribed to the Google Groups "Forseti Security Beta Testers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to beta-testers...@forsetisecurity.org.
To post to this group, send email to beta-t...@forsetisecurity.org.
To view this discussion on the web visit https://groups.google.com/a/forsetisecurity.org/d/msgid/beta-testers/CADT05SQwDJ6_bfb___z7%2BxwcqLk4EUFYVzXwf0EjGDBXfBSQiw%40mail.gmail.com.

[yukie sato]

Hi Lukas and Joe,

Thanks for reply.  VM instance reset helped somewhat.  forseti cli is there now, but I am getting Import Error on google.apputils now.

Traceback (most recent call last):  File "/usr/local/bin/forseti", line 11, in <module>    load_entry_point('forseti-security==2.0.0', 'console_scripts', 'forseti')()  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 480, in load_entry_point    return get_distribution(dist).load_entry_point(group, name)  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2693, in load_entry_point    return ep.load()  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2324, in load    return self.resolve()  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2330, in resolve    module = __import__(self.module_name, fromlist=['__name__'], level=0)  File "/usr/local/lib/python2.7/dist-packages/forseti_security-2.0.0-py2.7.egg/google/cloud/forseti/stubs.py", line 26, in <module>    from google.apputils import run_script_moduleImportError: No module named apputils

I do see apputils found in /tmp/deployment.log

Using /usr/local/lib/python2.7/dist-packagesSearching for google-apputils==0.4.2Best match: google-apputils 0.4.2Adding google-apputils 0.4.2 to easy-install.pth file

Thanks,

-Yukie

[Joe Cheuk]

Hi Yukie,

Thanks for the updates! 

I would like to doublecheck with you - did you reset the VM before or after my email response? I merged in the fix right before sending out the email so if you did it before it might not contain the updates I put in. When trying on my new build, everything seems to be working, so it could just be the timing issue.

Best regards,

Joe

To view this discussion on the web visit https://groups.google.com/a/forsetisecurity.org/d/msgid/beta-testers/46814aea-a594-4442-a315-cf9dcfcb9eee%40forsetisecurity.org.

[yukie sato]

HI Joe,

So, I reset the VM before and after your reply.  I also manually updated the startup-script of the VM to match the change you guys made for installing pip 9.0.3.

When I check the version of the pip, it does say 9.0.3

also, there is google/apputils under /usr/local/lib/python2.7/dist-packages

Maybe I should just delete the VMs and start from scratch at this point?

[Lukas Karlsson]

Is there a trick for deleting the vm and recreating it with deployment manager?  

I find myself commenting out the lines related to the vm, running the "deployment-manager deployment update" to delete the vm and then uncommenting the lines related to the vm and running "deployment-manager deployment update" once again to recreate it.

Is there a more elegant way to do that?

/l

[Joe Cheuk]

Hi Yukie,

Thanks for the quick response! 

Start from scratch is the best option in your scenario (with branch "2.0-rc2"). If that doesn't work, we can always schedule a meeting to go through the issues that you are experiencing.

Lukas,

I don't think there is any tricks in deployment manager that allows you to delete some of the resources but not all, commenting the resource out might be the only option here. I am thinking if there is a good use case maybe we can split the deployment manager template into 2, one for compute engine (VM instance) and one for everything else (cloudsql instance and bucket). What do you think?

Best regards,

Joe

--

You received this message because you are subscribed to the Google Groups "Forseti Security Beta Testers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beta-testers...@forsetisecurity.org.
To post to this group, send email to beta-t...@forsetisecurity.org.

To view this discussion on the web visit https://groups.google.com/a/forsetisecurity.org/d/msgid/beta-testers/CADT05SRZW%3DLe8UnJo8N9jmOnEyZm8raVsmc-_fQuMT_Yod_v_A%40mail.gmail.com.

[Lukas Karlsson]

I mean, in the end of the day this is a deployment-manager issue not a forseti issue. 

But in general, as someone who uses deployment manager, it would be nice if there was an easy way to "recreate this name resource" instead of 1) delete resource, 2) update the deployment config file, 3) run an update, 4) update config file again, 5) run an update.  

/l

On Wed, Apr 18, 2018 at 3:01 PM, Joe Cheuk <joec...@google.com> wrote:

Lukas,

I don't think there is any tricks in deployment manager that allows you to delete some of the resources but not all, commenting the resource out might be the only option here. I am thinking if there is a good use case maybe we can split the deployment manager template into 2, one for compute engine (VM instance) and one for everything else (cloudsql instance and bucket). What do you think?

Best regards,

Joe

[yukie sato]

Hi Joe,

So, I did complete re-install of forseti, now I can run forseti command.

When I run forseti inventory create, It runs, but I see several warnings: 2, and when I run forseti inventory list, I get following message:

id: "2018-04-18T19:58:02.432502"

schema_version: 1

count_objects: 1847

status: "SUCCESS"

warnings: "unauthorized_client: Client is unauthorized to retrieve access tokens using this method.\n\nunauthorized_client: Client is unauthorized to retrieve access tokens using this method. n\n"

Is client's service account missing something?

[Joe Cheuk]

Hi Lukas,

That makes a lot of sense! Please let us know if there is anything we can do on the Forseti side to make the process easier!

Thanks,

Joe

[Joe Cheuk]

Hi yukie,

Did you set up the Domain-Wide Delegation on the gsuite service account and provision the service account with permission admin.directory.group.readonly and admin.directory.user.readonly?

If not, you can follow the steps here to do that:

https://forsetisecurity.org/docs/howto/configure/gsuite-group-collection.html

That should be the main reason why you were getting the unauthorized warnings. 

Thanks,

Joe

--
You received this message because you are subscribed to the Google Groups "Forseti Security Beta Testers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beta-testers...@forsetisecurity.org.
To post to this group, send email to beta-t...@forsetisecurity.org.

To view this discussion on the web visit https://groups.google.com/a/forsetisecurity.org/d/msgid/beta-testers/a332613e-fef4-4916-84bd-d2064c0c31db%40forsetisecurity.org.

[yukie sato]

Hi Joe,

I had wrong service account enabled for Domain-Wide delegation. After fixing that, it works now.

Thanks for all the help.

[Joe Cheuk]

Hi Yukie,

It's my pleasure! Please don't hesitate to reach out again if you have more questions!

Best regards,

Joe

--
You received this message because you are subscribed to the Google Groups "Forseti Security Beta Testers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beta-testers...@forsetisecurity.org.
To post to this group, send email to beta-t...@forsetisecurity.org.

To view this discussion on the web visit https://groups.google.com/a/forsetisecurity.org/d/msgid/beta-testers/4e20bdfa-830b-48cc-96aa-9eb68ed416be%40forsetisecurity.org.