Statistics not showing info received from Manage
9 views
Skip to first unread message
Domingos Gonçalves
Jun 5, 2019, 7:23:11 AM6/5/19
to OpenConext Community
Hi,
we perform an upgrade on Statistics and we notice some info from Manage is not display on the interface.
For instance:
- on the Filter section the services and Institutions are empty on each Combo Box
- the Display Name for the SPs and IDPs defined on Manage are not displayed, we see only the Entity ID
You can see more detail on the figure attach (Statistics_GUI.png).
This is on the development environment, we don't have many authentication there, low records on InfluxDB database. Is it matters?
Looking to the logs we can see the messages saying (main INFO Retrieved 18 sp from Manage with mock is False). I believe statistics can to Manage normally, but Stats it's not showing the entries he received.
Jun 5 11:50:22 stats01 Apache-STATS: 193.137.196.185 - "" [05/Jun/2019:11:50:22 +0100] "GET /health HTTP/1.0" 200 16 "-" "-" "-"
Jun 5 11:50:24 stats01 Apache-STATS: 193.137.196.185 - a4bea01c-d265-361a-bc6f-6be06a13e5b8@oidc.dev.rctsaai.pt/ [05/Jun/2019:11:50:24 +0100] "GET /api/users/me HTTP/1.1" 200 165 "https://stats.dev.rctsaai.pt/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:67.0) Gecko/20100101 Firefox/67.0" "193.136.44.97"
Jun 5 11:50:24 stats01 STATISTICS: 2019-06-05 11:50:24,733 main INFO Retrieved 18 sp from Manage with mock is False
Jun 5 11:50:24 stats01 STATISTICS: 2019-06-05 11:50:24,736 main INFO Retrieved 7 idp from Manage with mock is False
Jun 5 11:50:24 stats01 influxd: ts=2019-06-05T10:50:24.736523Z lvl=info msg="Executing query" log_id=0FPlsnVl000 service=query query="SHOW TAG VALUES ON log_logins WITH KEY = sp_entity_id WHERE (_name = 'ebauth') AND (_tagKey = 'sp_entity_id')"
Jun 5 11:50:24 stats01 influxd: [httpd] 127.0.0.1 - stats [05/Jun/2019:11:50:24 +0100] "GET /query?db=log_logins&q=show+tag+values+from+ebauth+with+key+%3D+sp_entity_id HTTP/1.1" 200 57 "-" "python-requests/2.20.0" b91663f5-877f-11e9-80bb-92979c032433 1078
Jun 5 11:50:24 stats01 influxd: ts=2019-06-05T10:50:24.739336Z lvl=info msg="Executing query" log_id=0FPlsnVl000 service=query query="SHOW TAG VALUES ON log_logins WITH KEY = idp_entity_id WHERE (_name = 'ebauth') AND (_tagKey = 'idp_entity_id')"
Jun 5 11:50:24 stats01 influxd: [httpd] 127.0.0.1 - stats [05/Jun/2019:11:50:24 +0100] "GET /query?db=log_logins&q=show+tag+values+from+ebauth+with+key+%3D+idp_entity_id HTTP/1.1" 200 57 "-" "python-requests/2.20.0" b916d875-877f-11e9-80bc-92979c032433 803
Jun 5 11:50:24 stats01 Apache-STATS: 193.137.196.185 - a4bea01c-d265-361a-bc6f-6be06a13e5b8@oidc.dev.rctsaai.pt/ [05/Jun/2019:11:50:24 +0100] "GET /api/stats/identity_providers HTTP/1.1" 200 3 "https://stats.dev.rctsaai.pt/live" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:67.0) Gecko/20100101 Firefox/67.0" "193.136.44.97"
Jun 5 11:50:24 stats01 Apache-STATS: 193.137.196.185 - a4bea01c-d265-361a-bc6f-6be06a13e5b8@oidc.dev.rctsaai.pt/ [05/Jun/2019:11:50:24 +0100] "GET /api/stats/service_providers HTTP/1.1" 200 3 "https://stats.dev.rctsaai.pt/live" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:67.0) Gecko/20100101 Firefox/67.0" "193.136.44.97"
Jun 5 11:50:24 stats01 STATISTICS: 2019-06-05 11:50:24,798 main INFO select count(user_id) as count_user_id from ebauth where 1=1 and time >= 1559645424s and time < 1559775600s and state = 'prodaccepted' group by time(1m)
Jun 5 11:50:24 stats01 influxd: ts=2019-06-05T10:50:24.801881Z lvl=info msg="Executing query" log_id=0FPlsnVl000 service=query query="SELECT count(user_id) AS count_user_id FROM log_logins.autogen.ebauth WHERE 1 = 1 AND time >= 1559645424s AND time < 433271h AND state = 'prodaccepted' GROUP BY time(1m)"
Jun 5 11:50:24 stats01 influxd: [httpd] 127.0.0.1 - stats [05/Jun/2019:11:50:24 +0100] "GET /query?db=log_logins&epoch=ms&q=select+count%28user_id%29+as+count_user_id+from+ebauth+where+1%3D1+and+time+%3E%3D+1559645424s+and+time+%3C+1559775600s+and+state+%3D+%27prodaccepted%27+group+by+time%281m%29 HTTP/1.1" 200 57 "-" "python-requests/2.20.0" b9205d75-877f-11e9-80bd-92979c032433 977
Jun 5 11:50:24 stats01 Apache-STATS: 193.137.196.185 - a4bea01c-d265-361a-bc6f-6be06a13e5b8@oidc.dev.rctsaai.pt/ [05/Jun/2019:11:50:24 +0100] "GET /api/stats/public/login_time_frame?from=1559645424&to=1559775600&include_unique=true&scale=minute&epoch=ms&state=prodaccepted& HTTP/1.1" 200 15 "https://stats.dev.rctsaai.pt/live" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:67.0) Gecko/20100101 Firefox/67.0" "193.136.44.97"Best regards,
Domingos Gonçalves
Okke Harsta
Jun 5, 2019, 9:01:23 AM6/5/19
to openc...@googlegroups.com
Hi,
Could you troubleshoot and open the stats application in Chrome with the DevTools open, look for the http requests:
Anything strange there? The recent changes in stats were bug fixes for small anomalies and I would be surprised if they were related. And if you Group By Services and look at the response of:
Do the sp_entity_id's corresponds with entity_id's from https://stats.test2.surfconext.nl/api/stats/service_providers?
Cheers,
Okke
--
OpenConext - Open For Collaboration
---
You received this message because you are subscribed to the Google Groups "OpenConext Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openconext+...@googlegroups.com.
To post to this group, send email to openc...@googlegroups.com.
Visit this group at https://groups.google.com/group/openconext.
To view this discussion on the web visit https://groups.google.com/d/msgid/openconext/0f9fbd56-308a-4575-9654-d69de2d7e7d3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
<Statistics_GUI.png>
Domingos Gonçalves
Jun 5, 2019, 10:06:51 AM6/5/19
to OpenConext Community
Hi,
I performed tests you suggest and you can find some info bellow.
The impression I have is Stats is not showing what he receive from Manage. The communication between Stats and Manage is fine, but when it should present the content it's empty.
IDP HTTP request
Log generated:
Jun 5 14:36:30 stats01 STATISTICS: 2019-06-05 14:36:30,258 main INFO Retrieved 7 idp from Manage with mock is False
Jun 5 14:36:30 stats01 influxd: ts=2019-06-05T13:36:30.262305Z lvl=info msg="Executing query" log_id=0FPlsnVl000 service=query query="SHOW TAG VALUES ON log_logins WITH KEY = idp_entity_id WHERE (_name = 'ebauth') AND (_tagKey = 'idp_entity_id')"
Jun 5 14:36:30 stats01 influxd: [httpd] 127.0.0.1 - stats [05/Jun/2019:14:36:30 +0100] "GET /query?db=log_logins&q=show+tag+values+from+ebauth+with+key+%3D+idp_entity_id HTTP/1.1" 200 57 "-" "python-requests/2.20.0" ed00fb7e-8796-11e9-80cf-92979c032433 3908
Jun 5 14:36:30 stats01 Apache-STATS: 193.137.196.185 - "" [05/Jun/2019:14:36:30 +0100] "GET /api/stats/identity_providers HTTP/1.1" 200 3 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36" "172.16.10.44"
Jun 5 14:36:30 stats01 Apache-STATS: 193.137.196.185 - "" [05/Jun/2019:14:36:30 +0100] "GET /favicon.ico HTTP/1.1" 200 5430 "https://stats.dev.rctsaai.pt/api/stats/identity_providers" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36" "172.16.10.44"SP HTTP request
Log generated:
Jun 5 14:28:46 stats01 Apache-STATS: 193.137.196.185 - "" [05/Jun/2019:14:28:46 +0100] "GET /health HTTP/1.0" 200 16 "-" "-" "-"
Jun 5 14:28:49 stats01 STATISTICS: 2019-06-05 14:28:49,918 main INFO Retrieved 18 sp from Manage with mock is False
Jun 5 14:28:49 stats01 influxd: ts=2019-06-05T13:28:49.921581Z lvl=info msg="Executing query" log_id=0FPlsnVl000 service=query query="SHOW TAG VALUES ON log_logins WITH KEY = sp_entity_id WHERE (_name = 'ebauth') AND (_tagKey = 'sp_entity_id')"
Jun 5 14:28:49 stats01 influxd: [httpd] 127.0.0.1 - stats [05/Jun/2019:14:28:49 +0100] "GET /query?db=log_logins&q=show+tag+values+from+ebauth+with+key+%3D+sp_entity_id HTTP/1.1" 200 57 "-" "python-requests/2.20.0" da9e8e6a-8795-11e9-80ce-92979c032433 1302
Jun 5 14:28:49 stats01 Apache-STATS: 193.137.196.185 - "" [05/Jun/2019:14:28:49 +0100] "GET /api/stats/service_providers HTTP/1.1" 200 3 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36" "172.16.10.44"
Jun 5 14:28:49 stats01 Apache-STATS: 193.137.196.185 - "" [05/Jun/2019:14:28:49 +0100] "GET /favicon.ico HTTP/1.1" 200 5430 "https://stats.dev.rctsaai.pt/api/stats/service_providers" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36" "172.16.10.44"Grouping By Services and look at the response
SP HTTP request: https://stats.dev.rctsaai.pt/api/stats/public/login_aggregated?period=2018&include_unique=true&group_by=sp_id,&state=prodaccepted
Log generated:
Jun 5 14:39:39 stats01 STATISTICS: 2019-06-05 14:39:39,607 main INFO select * from sp_pa_users_year where 1=1 and year = '2018'
Jun 5 14:39:39 stats01 influxd: ts=2019-06-05T13:39:39.612678Z lvl=info msg="Executing query" log_id=0FPlsnVl000 service=query query="SELECT * FROM log_logins.autogen.sp_pa_users_year WHERE 1 = 1 AND year = '2018'"
Jun 5 14:39:39 stats01 influxd: [httpd] 127.0.0.1 - stats [05/Jun/2019:14:39:39 +0100] "GET /query?db=log_logins&q=select+%2A+from+sp_pa_users_year+where+1%3D1+and+year+%3D+%272018%27 HTTP/1.1" 200 267 "-" "python-requests/2.20.0" 5ddd8c49-8797-11e9-80d0-92979c032433 4758
Jun 5 14:39:39 stats01 influxd: ts=2019-06-05T13:39:39.620635Z lvl=info msg="Executing query" log_id=0FPlsnVl000 service=query query="SELECT * FROM log_logins.autogen.sp_pa_users_year_unique WHERE 1 = 1 AND year = '2018'"
Jun 5 14:39:39 stats01 influxd: [httpd] 127.0.0.1 - stats [05/Jun/2019:14:39:39 +0100] "GET /query?db=log_logins&q=select+%2A+from+sp_pa_users_year_unique+where+1%3D1+and+year+%3D+%272018%27 HTTP/1.1" 200 277 "-" "python-requests/2.20.0" 5ddec17d-8797-11e9-80d1-92979c032433 1904
Jun 5 14:39:39 stats01 Apache-STATS: 193.137.196.185 - "" [05/Jun/2019:14:39:39 +0100] "GET /api/stats/public/login_aggregated?period=2018&include_unique=true&group_by=sp_id,&state=prodaccepted HTTP/1.1" 200 1795 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36" "172.16.10.44"
Jun 5 14:39:39 stats01 Apache-STATS: 193.137.196.185 - "" [05/Jun/2019:14:39:39 +0100] "GET /favicon.ico HTTP/1.1" 200 5430 "https://stats.dev.rctsaai.pt/api/stats/public/login_aggregated?period=2018&include_unique=true&group_by=sp_id,&state=prodaccepted" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36" "172.16.10.44"quarta-feira, 5 de Junho de 2019 às 14:01:23 UTC+1, Okke Harsta escreveu:
Hi,Could you troubleshoot and open the stats application in Chrome with the DevTools open, look for the http requests:Anything strange there? The recent changes in stats were bug fixes for small anomalies and I would be surprised if they were related. And if you Group By Services and look at the response of:Do the sp_entity_id's corresponds with entity_id's from https://stats.test2.surfconext.nl/api/stats/service_providers?Cheers,
Okke
To unsubscribe from this group and stop receiving emails from it, send an email to openc...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages