Verify that v2 inventory data is consistent with v1.

[Lukas Karlsson]

In the section about testing the sql data, there is a query that it suggests:

-- In v2.0.

-- Get the latest inventory_index_id.

SET @inventory_index_id = (SELECT id FROM inventory_index ORDER BY id DESC LIMIT 1);

-- Number of projects inventoried.

SELECT Count(*) number_of_projects

FROM gcp_inventory

WHERE gcp_inventory.index = @inventory_index_id

AND gcp_inventory.type = 'project'

AND gcp_inventory.type_class = 'resource';

This doesn't work for me at all. Is that what's intended?

In v2, there is no field called index, no field called type and no field called type_class as far as I can tell.

When I changed it to this, it worked better:

SELECT count(*) number_of_projects

FROM gcp_inventory

WHERE gcp_inventory.inventory_index_id = '2018-04-18T04:57:38.735183'

AND gcp_inventory.resource_type = 'project'

AND gcp_inventory.category = 'resource';

Also, for me, my most recent index did not show any projects but when I picked an earlier one I got back the number I expected.

/l

--

Lukas Karlsson, Cloud Architect / Developer Advocate

Broad Institute of MIT and Harvard
75 Ames Street, 11129, Cambridge, Massachusetts 02142
karl...@broadinstitute.org, +1.6177147142

[Lukas Karlsson]

And in the section with:

Confirm that G Suite is configured correctly

Run the following SQL commands to make sure that gsuite_group and gsuite_users were correctly inventoried.

-- Check for G Suite groups. SELECT * FROM gcp_inventory WHERE type = 'gsuite_group'; -- Check for G Suite users. SELECT * FROM gcp_inventory WHERE type = 'gsuite_user';

Again, there is no "type" field as far as I can tell.  These queries worked better for me to test that this was working:

-- Check number of projects 
SELECT Count(*) number_of_projects, 
       inventory_index_id 
FROM   gcp_inventory 
WHERE  resource_type = 'project' 
       AND category = 'resource' 
GROUP  BY inventory_index_id; 

-- Check number of service accounts 
SELECT Count(*) number_of_service_accounts, 
       inventory_index_id 
FROM   gcp_inventory 
WHERE  resource_type = 'serviceaccount' 
       AND category = 'resource' 
GROUP  BY inventory_index_id; 

-- Check G Suite Groups 

SELECT Count(*) AS count, 

       inventory_index_id 

FROM   gcp_inventory 

WHERE  resource_type = 'gsuite_group' 

GROUP  BY inventory_index_id; 

-- Check G Suite Users 

SELECT Count(*) AS count, 

       inventory_index_id 

FROM   gcp_inventory 

WHERE  resource_type = 'gsuite_user' 

GROUP  BY inventory_index_id; 

 /l

[Joe Cheuk]

Hi Lukas,

This section of the testing guide is outdated, sorry for the inconvenience.

I think the reason why you were not getting any results from querying the most recent index is because you were running the query against the inventory with status "CREATED" and not "SUCCESS". Since the GCP organization you are running Forseti on is very big, the default 2 hours (from the cron job) might not be enough to finish the inventory process. Can you try decreasing the frequency of the cron job to have it run twice a day and see if you are able get more inventory with status = "SUCCESS"? The cron job is scheduled under user ubuntu so you will need to switch user to ubuntu to modify the frequency.

Best regards,

Joe

--
You received this message because you are subscribed to the Google Groups "Forseti Security Beta Testers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beta-testers...@forsetisecurity.org.
To post to this group, send email to beta-t...@forsetisecurity.org.
To view this discussion on the web visit https://groups.google.com/a/forsetisecurity.org/d/msgid/beta-testers/CADT05SR5o5QkfDx1WMQA0Fy8v-sP9FGEyUVe25UKHAr-VfDGYw%40mail.gmail.com.

[Lukas Karlsson]

Okay, sounds good.  Yes, I didn't realized that CREATED wasn't the best thing to be seeing there. Makes sense.

So when a new inventory runs 2 hours later it kills the previously running one instead of skipping itself?

Changed the crontab to every 12 hours instead of ever 2 hours:

sudo env EDITOR=vim crontab -u ubuntu -e

/l

To unsubscribe from this group and stop receiving emails from it, send an email to beta-testers+unsubscribe@forsetisecurity.org.
To post to this group, send email to beta-testers@forsetisecurity.org.

To view this discussion on the web visit https://groups.google.com/a/forsetisecurity.org/d/msgid/beta-testers/CADT05SR5o5QkfDx1WMQA0Fy8v-sP9FGEyUVe25UKHAr-VfDGYw%40mail.gmail.com.