Can I distinguish devices by number in API search?

[Jan Helebrant]

Dear colleagues

Due to the large number of bGeigie Nano and CzechRad devices we operate, it is not possible to manually check which device we have data from from the user and which we do not (we lend the devices free of charge, the logistics are handled by someone other than data processing and uploading to the API).

I created the "SAFECAST API checker for Windows":

https://github.com/juhele/CzechRad/tree/main/SCRIPTS#script-06---safecast-api-checker-for-windows

which is using API search links in JSON and lists the latest uploaded data to the output CSV. Can process hundreds of links in a few minutes but I am facing a following problem.

Example:

- we have CzechRads from 0001 to 1000

- I want to know if there are some data from devices 0003, 0003 and 0005

- I use the API links:

device,device_number,project,comment,api_link
CzechRad,0001,IMPAKT,,https://api.safecast.org/en-US/bgeigie_imports?by_user_id=6776&format=json&order=created_at+desc&q=0003
CzechRad,0002,IMPAKT,,https://api.safecast.org/en-US/bgeigie_imports?by_user_id=6776&format=json&order=created_at+desc&q=0004
CzechRad,0003,IMPAKT,,https://api.safecast.org/en-US/bgeigie_imports?by_user_id=6776&format=json&order=created_at+desc&q=0005

device,device_number,project,comment,last_data_uploaded,measured_by,data_info,api_link
CzechRad,0003,IMPAKT,,2025-04-04T07:43:12.120Z,Gymta.cz (for SURO.cz),Tábor - walk,https://api.safecast.org/en-US/bgeigie_imports?by_user_id=6776&format=json&order=created_at+desc&q=0003
CzechRad,0004,IMPAKT,,2025-06-17T13:06:56.802Z,Gymta.cz (for SURO.cz),Tábor - walk,https://api.safecast.org/en-US/bgeigie_imports?by_user_id=6776&format=json&order=created_at+desc&q=0004
CzechRad,0005,IMPAKT,,2025-06-17T13:27:35.999Z,Gymta.cz (for SURO.cz),Tábor - walk,https://api.safecast.org/en-US/bgeigie_imports?by_user_id=6776&format=json&order=created_at+desc&q=0005

Unfortunately, that is wrong because these are all data from device 0500 but the search finds the number "inside" the data filename:

05000330_upload_walk_l.LOG

05000421_upload_walk_l.LOG

05000526_upload_walk_l.LOG

Do you have any idea how to solve this? I was thinking about adding some condition related to 4 left numbers in filename but that makes it complicated... Or just add the filename to the output CSV and add a comment like "device number mismatch" so my colleagues can check it manually...

regards

Jan

[Rob Oudendijk]

Jan,

Are all the files names consistently renamed, like CzechRad 0003 will be 0003*...log?

Regards

Rob

 

Regards,

Rob Oudendijk Yuka Hayashi

https://yr-design.biz   https://oudendijk.biz  https://about.me/robouden

tel +81 80-22605966  Skype: robouden  Facebook:robouden linkedin:robouden  

--
You received this message because you are subscribed to the Google Groups "Safecast Device Discussions and Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to safecast-devic...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/safecast-devices/d0f580af-7df9-441c-b599-83a208aec969n%40googlegroups.com.

[Jan Helebrant]

Dear Rob,

yes, I am trying to always preserve the original LOG name and just add something after that like "03060828_upload_bike-walk_l.LOG". 

regards

Jan

Dne čtvrtek 4. září 2025 v 13:05:23 UTC+2 uživatel oudend...@gmail.com napsal:

[Rob Oudendijk]

Jan,

You should be able to filter out some condition related to 4 left numbers in filename and get those ID's. I think.

Regards

Rob

Regards,

Rob Oudendijk Yuka Hayashi

https://yr-design.biz   https://oudendijk.biz  https://about.me/robouden

tel +81 80-22605966  Skype: robouden  Facebook:robouden linkedin:robouden  

To view this discussion visit https://groups.google.com/d/msgid/safecast-devices/d62ac264-94ed-4ee5-9a18-d6b9b4354afcn%40googlegroups.com.

[Jan Helebrant]

Dear Rob,

tried to find some more about the API but not much luck... And quick test with wildcards like:

https://api.safecast.org/en-US/bgeigie_imports?by_user_id=6776&format=json&order=created_at+desc&q=0003####

did not work.

is it possible to search within the "source" attribute? Search for "/0003" might solve this problem as the field looks like:

"source":{"url":"https://safecastapi-imports-production-us-west-2.s3.amazonaws.com/uploads/bgeigie_import/source/67249/05000330_upload_walk_l.LOG"

And I do not think there might be other items containing "/0003" within  then data from detector no 0003.

regards

Jan

 

Dne čtvrtek 4. září 2025 v 13:46:59 UTC+2 uživatel oudend...@gmail.com napsal:

[Rob Oudendijk]

Jan,

Did you try regular expectations in the search? Like https://regex101.com/r/3tl0sK/1?

Or something like this should work:

Request:

https://api.safecast.org/bgeigie_imports.json?q=0003&order=source+asc

Client-side filter (Python):

url = "https://api.safecast.org/bgeigie_imports.json"
params = {"q": "0003", "order": "source asc", "per_page": 100}
resp = requests.get(url, params=params, timeout=30)
resp.raise_for_status()
imports = resp.json()

prefix = "0003"
filtered = [i for i in imports if (i.get("source") or "").startswith(prefix)]
import_ids = [i["id"] for i in filtered]
print(import_ids)

Regards

Rob

Regards,

Rob Oudendijk Yuka Hayashi

https://yr-design.biz   https://oudendijk.biz  https://about.me/robouden

tel +81 80-22605966  Skype: robouden  Facebook:robouden linkedin:robouden  

To view this discussion visit https://groups.google.com/d/msgid/safecast-devices/7fece1be-9d58-467c-bf33-4917ab1c048en%40googlegroups.com.

[Jan Helebrant]

Hi,

the regular expressions do not seem to work here... 

My only idea is to use the "url" field in the JSON:

which seems to always be like:

"url":"https://safecastapi-imports-production-us-west-2.s3.amazonaws.com/uploads/bgeigie_import/source/69190/03740913_upload2_walk_l.LOG"

and although I apparently cannot filter out "wrong" LOG imports using API search filter it should be possible to do a search in the JSON like:

look for number combination "0374" in "url" field in first 4 characters after 7th "/" character

- that could do it... I hope

I'll try to get some free time and try to modify the code like this.

regards

Jan

PS: shouldn't the  "&per_page=100"  parameter used in the url list 100 records? It always lists only 24:

https://api.safecast.org/en-US/bgeigie_imports?by_user_id=6776&format=json&order=created_at+desc&q=v%C3%ADt+kulh%C3%A1nek&status=all&per_page=100

Dne pátek 5. září 2025 v 0:37:14 UTC+2 uživatel oudend...@gmail.com napsal:

[Rob Oudendijk]

Jan,

Looking forward to seeing your findings.

Hugs

Rob

Regards,

Rob Oudendijk Yuka Hayashi

https://yr-design.biz   https://oudendijk.biz  https://about.me/robouden

tel +81 80-22605966  Skype: robouden  Facebook:robouden linkedin:robouden  

To view this discussion visit https://groups.google.com/d/msgid/safecast-devices/fef4064e-ca12-480c-8bfb-3477f7bb3f51n%40googlegroups.com.