Deleted duplicate index pattern, visualization now broken

[exe]

Hello There,

yes this may sound dumb but we somehhow had 2 exact same index patterns called wazuh-alerts-* and we deleted the duplicate. Now our visualizations dont work anymore (kinda obvious) because most of them were using the duplicate and not the default. Is there a way to save this and change it so it takes up the default indext pattern? 

I saw on the bottom that there are references for the index patterns, but i couldnt find the id for the default one. 

As always, thank you for your help!

Here the error:

There is a problem with this saved object

The index pattern associated with this object no longer exists.

If you know what this error means, go ahead and fix it — otherwise click the delete button above.

Proceed with caution!

Modifying objects is for advanced users only. Object properties are not validated and invalid objects could cause errors, data loss, or worse. Unless someone with intimate knowledge of the code told you to be in here, you probably shouldn’t be.

[Awwal Ishiaku]

Hello, send this request in the indexer dev tools to get the ID of all indices

GET .kibana/_search
{
  "_source": ["index-pattern.title"],
  "query": {
    "term": {
      "type": "index-pattern"
    }
  }
}

And then we can figure out a workaround for changing the index pattern.

How many visualizations do you have in your dashboard?

We can export the dashboard and all visualizations in it, and edit the index ID in the text file before re-importing.

You can navigate to Dashboard management -> Saved Objects

Search for your dashboard, select it, and export all objects

After this, we can edit the exported files and import.

[exe]

Hello Awwal,

thanks for the quick answer, when i did:

GET .kibana/_search
{
  "_source": ["index-pattern.title"],
  "query": {
    "term": {
      "type": "index-pattern"
    }
  }
}

I got 

{

  "error": "Not Found",
  "message": "404 - Not Found"
}

I went to Server Management -> Dev Tools, or am i on the wrong one?

I have 16 Visualizations and 3 Dashboards.

Thank you!

[Awwal Ishiaku]

In that case, let's search without the string kibana

GET /_search

[exe]

Thanks for the response:

Here the output

{
  "took": 39,
  "timed_out": false,
  "_shards": {
    "total": 458,
    "successful": 458,
    "skipped": 84,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 13,
      "relation": "eq"
    },
    "max_score": 0.8507761,
    "hits": [
      {
        "_index": ".kibana_1",
        "_id": "index-pattern:wazuh-monitoring-*",
        "_score": 0.8507761,
        "_source": {
          "index-pattern": {
            "title": "wazuh-monitoring-*"
          }
        }
      },
      {
        "_index": ".kibana_1",
        "_id": "index-pattern:wazuh-states-inventory-networks-*",
        "_score": 0.8507761,
        "_source": {
          "index-pattern": {
            "title": "wazuh-states-inventory-networks-*"
          }
        }
      },
      {
        "_index": ".kibana_1",
        "_id": "index-pattern:wazuh-states-inventory-packages-*",
        "_score": 0.8507761,
        "_source": {
          "index-pattern": {
            "title": "wazuh-states-inventory-packages-*"
          }
        }
      },
      {
        "_index": ".kibana_1",
        "_id": "index-pattern:wazuh-states-inventory-processes-*",
        "_score": 0.8507761,
        "_source": {
          "index-pattern": {
            "title": "wazuh-states-inventory-processes-*"
          }
        }
      },
      {
        "_index": ".kibana_1",
        "_id": "index-pattern:wazuh-states-inventory-protocols-*",
        "_score": 0.8507761,
        "_source": {
          "index-pattern": {
            "title": "wazuh-states-inventory-protocols-*"
          }
        }
      },
      {
        "_index": ".kibana_1",
        "_id": "index-pattern:wazuh-states-inventory-hotfixes-*",
        "_score": 0.8507761,
        "_source": {
          "index-pattern": {
            "title": "wazuh-states-inventory-hotfixes-*"
          }
        }
      },
      {
        "_index": ".kibana_1",
        "_id": "index-pattern:wazuh-states-inventory-*",
        "_score": 0.8507761,
        "_source": {
          "index-pattern": {
            "title": "wazuh-states-inventory-*"
          }
        }
      },
      {
        "_index": ".kibana_1",
        "_id": "index-pattern:wazuh-states-inventory-system-*",
        "_score": 0.8507761,
        "_source": {
          "index-pattern": {
            "title": "wazuh-states-inventory-system-*"
          }
        }
      },
      {
        "_index": ".kibana_1",
        "_id": "index-pattern:wazuh-states-inventory-hardware-*",
        "_score": 0.8507761,
        "_source": {
          "index-pattern": {
            "title": "wazuh-states-inventory-hardware-*"
          }
        }
      },
      {
        "_index": ".kibana_1",
        "_id": "index-pattern:wazuh-statistics-*",
        "_score": 0.8507761,
        "_source": {
          "index-pattern": {
            "title": "wazuh-statistics-*"
          }
        }
      }
    ]
  }
}

[exe]

Any Update here?

Thanks!

[Awwal Ishiaku]

Looks like the wazuh-alerts-* index doesn't exist in your indexer.

There are no duplicates to recover from.

Do you have any snapshots?

You can check with this command:

GET _snapshot/_all

[exe]

No snapshots, we do a backup with Veeam though.

If we can access the Server via the Backup, what would be the steps to take to get back the Data? Or to be more precise, can i just export the Dashboard and import it in the current Server?

[Awwal Ishiaku]

Since you have a Veeam backup, what matters is whether it captured the OpenSearch/Wazuh indexer data directory (not just configs). The indices will be in the /var/lib/wazuh-indexer/ directory.

Spin up a temporary restore node. In this case, you will restore the indexer data directory from Veeam to a separate VM/server
And start the indexer service there.

Then on this temporary node, verify the indices by executing the following on the indexer to verify that you have the alerts indices:
GET _cat/indices/wazuh-alerts-*?v

Once it's verified that the indices are present, you can porceed to reindex the alerts remotely to your current clusted by runner the following

POST _reindex
{
  "source": {
    "remote": {
        "host": "https://<restored-node-ip>:9200"

        "username":"YOUR_USERNAME",
         "password":"YOUR_PASSWORD"
    },
    "index": "wazuh-alerts-*"
  },
  "dest": {
    "index": "wazuh-alerts"
  }
}

Guides that may help:
https://documentation.wazuh.com/current/user-manual/wazuh-indexer/re-indexing.html

https://docs.opensearch.org/latest/im-plugin/reindex-data/#reindex-from-a-remote-cluster

[Awwal Ishiaku]

Update:

Your destination alerts index will look like

 "dest": {
    "index": "wazuh-alerts-*"