I think that's a very reasonable request: the page tells you that there are N objects with the tag, but doesn't let you find those items. I don't see an existing FR on github for this, so I suggest you create one.
I imagine it would be a variation on SearchView since it could find objects of mixed types.
As a temporary workaround, here's a SQL query you can use that will at least show you the object types and ids. (Once you know the object type you are looking for, it's also easy to filter by tag in the UI)
select app_label,model,object_id from extras_tag