model creation failure

[yukie sato]

Hi,

I am getting following error when I try to create a model (removed actual group name and user names from the message).  

looks like it trying to add same group(or user?) multiple times and getting error on Primary key?

We do have several nested groups, but not sure if that has anything to do with it.

forseti model create inventory --id 2018-04-19T16:32:18.054460 test_model2

Traceback (most recent call last):

  File "/usr/local/lib/python2.7/dist-packages/forseti_security-2.0.0-py2.7.egg/google/cloud/forseti/services/cli.py", line 1031, in <module>

    main(sys.argv[1:], ENV_CONFIG)

  File "/usr/local/lib/python2.7/dist-packages/forseti_security-2.0.0-py2.7.egg/google/cloud/forseti/services/cli.py", line 1013, in main

    services[config.service](client, config, output, config_env)

  File "/usr/local/lib/python2.7/dist-packages/forseti_security-2.0.0-py2.7.egg/google/cloud/forseti/services/cli.py", line 694, in run_model

    actions[config.action]()

  File "/usr/local/lib/python2.7/dist-packages/forseti_security-2.0.0-py2.7.egg/google/cloud/forseti/services/cli.py", line 670, in do_create_model

    config.background)

  File "/usr/local/lib/python2.7/dist-packages/forseti_security-2.0.0-py2.7.egg/google/cloud/forseti/services/client.py", line 219, in new_model

    background=background))

  File "/usr/local/lib/python2.7/dist-packages/grpc/_channel.py", line 487, in __call__

    return _end_unary_response_blocking(state, call, False, deadline)

  File "/usr/local/lib/python2.7/dist-packages/grpc/_channel.py", line 437, in _end_unary_response_blocking

    raise _Rendezvous(state, None, None, deadline)

grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with (StatusCode.UNKNOWN, Exception calling application: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (_mysql_exceptions.IntegrityError) (1062, "Duplicate entry 'group/ GROUP NAME REMOVED HERE' for key 'PRIMARY'") [SQL: u'INSERT INTO `2cc28387d8da6c6278954677fc2478d0_members` (name, type, member_name) VALUES (%s, %s, %s)'] [parameters: (('user/  HERE LIST OF USERS '))] (Background on this error at: http://sqlalche.me/e/gkpj))>

Thanks.

[Joe Cheuk]

Hi Yukie,

Thanks for reporting the issue! This is a known issue on our end and we are working on addressing it.

We have a pull request opened for this issue:

https://github.com/GoogleCloudPlatform/forseti-security/pull/1440

I would also like to address another issue that's related to building the data model before patching the 2.0-rc2 branch:

https://github.com/GoogleCloudPlatform/forseti-security/issues/1442

I hope to have both of these issues fixed early next week and we will keep you posted on the progress.

Best regards,

Joe

--
You received this message because you are subscribed to the Google Groups "Forseti Security Beta Testers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beta-testers...@forsetisecurity.org.
To post to this group, send email to beta-t...@forsetisecurity.org.
To view this discussion on the web visit https://groups.google.com/a/forsetisecurity.org/d/msgid/beta-testers/d5b47ec8-1324-41d6-bd21-ae833cc63fb3%40forsetisecurity.org.

[yukie sato]

Hi Joe,

Thanks for the info.  I'll wait for the update then.

[yukie sato]

Hi Joe,

Another question about model.

I installed forseti in smaller organization so I don't run into the duplicate error during model creation. 

And, now it creates model, but the status says 'PARTIAL_SUCCESS'.  

When I take a look at the model, it has some warnings about role missing permissions, and Role reference in iam policy not found (not sure if that's related to being partial success).

How do I find out what's 'PARTIAL' about it?

Thanks.

[Joe Cheuk]

Hi Yukie,

The reason why you are seeing those warnings is because the Google Cloud API doesn't return permissions for some of the roles (e.g. roles in alpha versions) and we log that as a warning when building the data model for future reference. If there is any warnings in the model, the status will become 'PARTIAL_SUCCESS' so there is nothing to worry about in this case.

Thanks,

Joe

--
You received this message because you are subscribed to the Google Groups "Forseti Security Beta Testers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beta-testers...@forsetisecurity.org.
To post to this group, send email to beta-t...@forsetisecurity.org.

To view this discussion on the web visit https://groups.google.com/a/forsetisecurity.org/d/msgid/beta-testers/0c073532-238c-4773-bd8f-9a0225808008%40forsetisecurity.org.

[Joe Cheuk]

Hi Yukie,

The 2.0-rc2 branch has been patched with a fix to the model creation issue, can you try resetting the server VM and run the model creation command again to see if you still experience the same error?

Instructions on how to reset the VM:

​

You can find more details on how to reset an instance here: https://cloud.google.com/compute/docs/instances/restarting-an-instance

Thanks,

Joe