Event-based remote write vs scraping

[Elliott Balsley]

Hi all.  I'm looking for advice.

I've written several multi-target prometheus exporters to collect data from audio/video devices in our screening rooms, using various arcane and proprietary protocols provided by each manufacturer.  This is generally working well with Grafana Alloy on a 1m scrape interval, but some devices offer an event-based protocol, and here it would be nice to remote-write the data as soon as it's received, so we can see faster changes that may be hidden between scrapes.  I see that prometheus added support for this a few years ago, but the docs have this warning:

This is not considered an efficient way of ingesting samples. Use it with caution for specific low-volume use cases. It is not suitable for replacing the ingestion via scraping and turning Prometheus into a push-based metrics collection system.

In what scenarios would this be a good idea?  How low is 'low volume'?  I'm expecting around 150K series in total.  The event frequency varies a lot — perhaps bursting up to 500 per minute, and sometimes going for several hours with nothing happening.  I would probably bound this by writing data every minute when nothing is happening.  My prometheus is hosted by Grafana Cloud, so I believe it's using mimir under the hood.

Thanks,

Elliott