Duplicated timeseries in CollectorRegistry

1,227 views
Skip to first unread message

Barry Al-Jawari

unread,
Jun 25, 2021, 10:05:19 AM6/25/21
to Prometheus Users
Im currently facing an issue where I do get an error that is saying:

   ` ValueError: Duplicated timeseries in CollectorRegistry: {'scraper_request_count_created', 'scraper_request_count_total', 'scraper_request_count'}`

I have two scripts which we can call file1.py and file2.py

**file1.py**:

```py
import time
import requests
from lib.prometheus import REQUEST_COUNT
    
def from_page(url):
   while True:
     with requests.get(url) as rep:
         REQUEST_COUNT().labels(store="stackoverflow", http_status=rep.status_code).inc()
         print("Response: ", rep.status_code)
         time.sleep(60)

if __name__ == '__main__':
   from_page("https://stackoverflow.com")
```

**file2.py**
```py
import time
import requests
from lib.prometheus import REQUEST_COUNT
    
def from_page(url):
   while True:
     with requests.get(url) as rep:
         REQUEST_COUNT().labels(store="google", http_status=rep.status_code).inc()
         print("Response: ", rep.status_code)
         time.sleep(60)

if __name__ == '__main__':
   from_page("https://google.com")
```

As you can see they both call the `lib.prometheus import REQUEST_COUNT` which is:
```py
    from prometheus_client import Counter, CollectorRegistry
    
    registery = CollectorRegistry()
    
    def REQUEST_COUNT():
        return Counter(
            namespace="scraper",
            name="request_count",
            documentation="Count the total requests",
            labelnames=['store', 'http_status'],
            registry=registery
        )
```

The problem is that if I run this script simultaneously then I will get the error `ValueError: Duplicated timeseries in CollectorRegistry: {'scraper_request_count_created', 'scraper_request_count_total', 'scraper_request_count'}` and I wonder what can I do be able to push the data even if its in duplicated timeseries?

Ian Billett

unread,
Jun 28, 2021, 12:44:45 PM6/28/21
to Barry Al-Jawari, Prometheus Users
Hey Barry,

I'm no expert, but I would think you are somehow implicitly using a global CollectorRegistry object in that code? 

Anyway, I would recommend raising this issue with prometheus/client_python repo who are the experts.

Best,

Ian



--
You received this message because you are subscribed to the Google Groups "Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/00d236e4-6ca2-46a6-ab6b-92bd3f771f8en%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages