Forseti Validator failing the testing

[Alex Kamalov]

Greetings,

This is Forseti policy validator question.  Wondering if anyone can point out my mistakes? 

I wrote a policy that is supposed to validate if machines have their secure boot enabled. 

File: test_gcp_compute_secure_boot_vm_test.rego

package templates.gcp.TESTGCPComputeSecureBootVM

import data.validator.test_utils as test_utils

import data.test.fixtures.TEST_gcp_compute_secure_boot_vm.assets as fixture_instances
import data.test.fixtures.TEST_gcp_compute_secure_boot_vm.constraints as fixture_constraints

# Find all violations on our test cases
find_violations[violation] {
	instance := data.instances[_]
	constraint := data.test_constraints[_]

	issues := deny with input.asset as instance
		 with input.constraint as constraint

	total_issues := count(issues)

	violation := issues[_]
}


test_forbid_non_secure_boot_no_instances {
	found_violations := find_violations with data.instances as []

	count(found_violations) = 0
}


test_forbid_non_secure_boot_no_constraints {
	found_violations := find_violations with data.instances as fixture_instances
		 with data.constraints as []

	count(found_violations) = 1
}

violations_with_empty_parameters[violation] {
	constraints := [fixture_constraints.forbid_non_secure_boot_default]

	found_violations := find_violations with data.instances as fixture_instances
		 with data.test_constraints as constraints

	violation := found_violations[_]
}

test_forbid_non_secure_boot_default {
	found_violations := violations_with_empty_parameters

	count(found_violations) = 1
}

Above, I am requesting that:

- if there arer no intances, there would be no violations
- if no constraints have been applied, raise a flag
- if an instance running without secure_boot parameters applied, raise a flag

Asset: data.json

[{
  "id": "858084573258323786",
  "creationTimestamp": "2020-06-12T09:13:26.329-07:00",
  "name": "test-server",
  "tags": {
    "items": [
      "cloudsec",
      "forseti"
    ],
    "fingerprint": "XXXXXXXXX"
  },
  "machineType": "https://www.googleapis.com/compute/v1/projects/test-project/zones/us-west4-c/machineTypes/n1-standard-1",
  "status": "RUNNING",
  "zone": "https://www.googleapis.com/compute/v1/projects/test-project/zones/us-west4-c",
  "networkInterfaces": [
    {
      "network": "https://www.googleapis.com/compute/v1/projects/test-project/global/networks/vpc-test-001",
      "subnetwork": "https://www.googleapis.com/compute/v1/projects/test-project/regions/us-west4/subnetworks/sub-uswest4-test",
      "networkIP": "192.168.1.9",
      "name": "nic0",
      "fingerprint": "1MDILfmwZDY=",
      "kind": "compute#networkInterface"
    }
  ],
  "disks": [
    {
      "type": "PERSISTENT",
      "mode": "READ_WRITE",
      "source": "https://www.googleapis.com/compute/v1/projects/test-project/zones/us-west4-c/disks/test-server",
      "deviceName": "persistent-disk-0",
      "index": 0,
      "boot": true,
      "autoDelete": true,
      "licenses": [
        "https://www.googleapis.com/compute/v1/projects/centos-cloud/global/licenses/centos-7"
      ],
      "interface": "SCSI",
      "guestOsFeatures": [
        {
          "type": "UEFI_COMPATIBLE"
        },
        {
          "type": "SECURE_BOOT"
        }
      ],
      "diskSizeGb": "100",
      "kind": "compute#attachedDisk"
    }
  ],
  "metadata": {
    "fingerprint": "DmKF_vwHnkg=",
    "items": [
      {
        "key": "enable-oslogin",
        "value": "TRUE"
      }
    ],
    "kind": "compute#metadata"
  },
  "serviceAccounts": [
    {
      "email": "test...@test-project.iam.gserviceaccount.com",
      "scopes": [
        "https://www.googleapis.com/auth/cloud-platform"
      ]
    }
  ],
  "selfLink": "https://www.googleapis.com/compute/v1/projects/test-project/zones/us-west4-c/instances/test-server1",
  "scheduling": {
    "onHostMaintenance": "MIGRATE",
    "automaticRestart": true,
    "preemptible": false
  },
  "cpuPlatform": "Intel Skylake",
  "labelFingerprint": "42WmSpB8rSM=",
  "startRestricted": false,
  "deletionProtection": false,
  "shieldedInstanceConfig": {
    "enableSecureBoot": true,
    "enableVtpm": true,
    "enableIntegrityMonitoring": true
  },
  "shieldedInstanceIntegrityPolicy": {
    "updateAutoLearnPolicy": true
  },
  "fingerprint": "lqzrbVbMZA4=",
  "kind": "compute#instance"
},
{
  "name": "//compute.googleapis.com/projects/test-project/zones/us-east1-c/instances/vm-cant-forward",
  "asset_type": "compute.googleapis.com/Instance",
  "resource": {
    "version": "v1",
    "discovery_document_uri": "https://www.googleapis.com/discovery/v1/apis/compute/v1/rest",
    "discovery_name": "Instance",
    "parent": "//cloudresourcemanager.googleapis.com/projects/68478495408",
    "data": {
      "canIpForward": false,
      "cpuPlatform": "Intel Haswell",
      "creationTimestamp": "2018-01-18T12:16:22.261-08:00",
      "deletionProtection": false,
      "disk": [
        {
          "autoDelete": true,
          "boot": true,
          "deviceName": "persistent-disk-0",
          "guestOsFeature": [
            {
              "type": "VIRTIO_SCSI_MULTIQUEUE"
            }
          ],
          "index": 0,
          "interface": "SCSI",
          "license": [
            "https://www.googleapis.com/compute/v1/projects/debian-cloud/global/licenses/debian-9-stretch"
          ],
          "mode": "READ_WRITE",
          "source": "https://www.googleapis.com/compute/v1/projects/test-project/zones/us-east1-c/disks/vm-no-ip",
          "type": "PERSISTENT"
        }
      ],
      "id": "8987947392482197114",
      "labelFingerprint": "42WmSpB8rSM=",
      "machineType": "https://www.googleapis.com/compute/v1/projects/test-project/zones/us-east1-c/machineTypes/g1-small",
      "name": "vm-no-ip",
      "networkInterfaces": [
        {
          "fingerprint": "+QCnSman4bQ=",
          "ipAddress": "10.1.0.2",
          "name": "nic0",
          "network": "https://www.googleapis.com/compute/v1/projects/test-project/global/networks/default",
          "subnetwork": "https://www.googleapis.com/compute/v1/projects/test-project/regions/us-east1/subnetworks/default-us-east1"
        }
      ],
      "scheduling": {
        "automaticRestart": true,
        "onHostMaintenance": "MIGRATE",
        "preemptible": false
      },
      "selfLink": "https://www.googleapis.com/compute/v1/projects/test-project/zones/us-east1-c/instances/vm-no-ip",
      "serviceAccount": [
        {
          "email": "66666666666-compute@developer.gserviceaccount.com",
          "scope": [
            "https://www.googleapis.com/auth/cloud-platform"
          ]
        }
      ],
      "startRestricted": false,
      "status": "RUNNING",
      "tags": {
        "fingerprint": "42WmSpB8rSM="
      },
      "zone": "https://www.googleapis.com/compute/v1/projects/test-project/zones/us-east1-c"
    }
  }
}
]

Here is the policy constraint file:

File: test_gcp_compute_secure_boot_vm.yaml

apiVersion: constraints.gatekeeper.sh/v1alpha1
kind: TESTGCPComputeSecureBootVM
metadata:
  name: wmt_gcp_compute_secure_boot_vm
  annotations:
    description: Trigger violation if non-secure boot VMs are used
spec:
  severity: high
  match:
    target: ["organization/*"]
    except: [] # Policy folder exceptions here
  parameters: {}
Validator test contraint file - data.yaml

apiVersion: constraints.gatekeeper.sh/v1alpha1
kind: TESTGCPCOMPSecureBootVM
metadata:
  name: forbid_non_secure_boot_default
spec:
  severity: high
  parameters: {}

Now, the test rego file supposed to find one violation for the non-compliance in "instances/vm-cant-forward" running VM. Except, it can't.

Running 'make test' shows the following:

$ make test

...

data.templates.gcp.TESTGCPComputeSecureBootVM.test_forbid_non_secure_boot_no_constraints: FAIL (152.549µs)
data.templates.gcp.TESTGCPComputeSecureBootVM.test_forbid_non_secure_boot_default: FAIL (391.163µs)

$
It looks like it is the logic to identify violations is wrong, because a secondary instance in data.json is violating the policy.

Any thoughts?

I tried to open a ticket under Forseti Github page, but I didn't get any replies, unfortunately.

Thanks so much!!!