Dashboard

[Domingos Gonçalves]

Hi,

After some upgrades we performed on OpenConext services our Dashboard stop sending emails when we Activate or Deactivate a service.

The only message we have on logs are from Apache with the following (/var/log/messages):

Nov 14 10:27:17 engine Apache-dashboard: 127.0.0.1 - - [14/Nov/2019:10:27:17 +0000] "POST /dashboard/api/services/disconnect HTTP/1.1" 403 - "https://dashboard.dev.rctsaai.pt/apps/21/saml20_sp/how_to_connect" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:70.0) Gecko/20100101 Firefox/70.0" "193.136.44.97, 127.0.0.1"
Nov 14 10:27:17 engine.dev.rctsaai.pt haproxy[2584]: 193.136.44.97:32005 [14/Nov/2019:10:27:17.640] ssl_ip~ dashboard_be/java 33/0/0/176/209 403 350 - - --VU 1/1/0/0/0 0/0 {Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:70.0) Gecko/20100101 Firefox/70.0} "POST /dashboard/api/services/disconnect HTTP/1.1"

No error appears on Dasboard log.

Even so, changing the log to DEBUG we can see some messages that you can find in the file attach (dashboard-activate-service.log)

So, when pressing the button "Activate service" we have the message saying "Something went wrong with your request" and no email is sent and no errors on dashboard log. That activate and deactivate services was working before 

Our services release at the moment:

   Service          [ Version ]
-------------------------------
   Engine           [ 5.11.0  ]
   Engine-API       [ 5.11.0  ]
   AA               [ 2.0.16  ]
   AUTHZ_ADMIN      [ 2.0.14  ]
   AUTHZ_SERVER     [ 1.3.16  ]
   MANAGE           [ 4.0.11  ]
   PDP              [ 2.0.5   ]
   TEAMS            [ 8.3.0   ]
   VOOT             [ 3.0.7   ]
   DASHBOARD        [ 9.0.9   ]

Our features on /opt/dashboard/application.properties

dashboard.feature.shibboleth=true
dashboard.feature.sab=false
dashboard.feature.manage=true
dashboard.feature.jira=false
dashboard.feature.consent=true
dashboard.feature.pdp=true
dashboard.feature.statistics=true
dashboard.feature.mail=true
dashboard.feature.oidc=false

That symptom appears after upgrading some other components, like EB, Voot, Teams, etc...

Is it missing something on our Apache virtual-host for the dashboard back-end?

Any idea?

Best regards,

Domingos

[Okke Harsta]

Hi,

Dashboard is currently on release 10.0.5. The main difference with the 9.x.x releases is a public landinlage. There were some bugs related to strict security in the 9.x.x releases and the 403 hints that that is the problem. When you upgrade to 10.0.5 ensure to check the differences made in https://github.com/OpenConext/OpenConext-deploy/blob/master/roles/dashboard-gui/templates/dashboard.conf.j2#L50.

Best regards,

Okke

--
OpenConext - Open For Collaboration
---
You received this message because you are subscribed to the Google Groups "OpenConext Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openconext+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openconext/3eec1730-4143-417b-8488-2e9b8cbce95b%40googlegroups.com.
<dashboard-activate-service.log>

[Domingos Gonçalves]

Hi,

we checked the differences o the Dashboard virtual-host you mention.

Performing that changes, we didn't solve the send email problem but, we create another one. When accessing Dashboard, a non-authenticated access didn't forward the browser to the proxy wayf to proceed to the authentication, but give us a direct access to the Services tab. You can tested right now if you want to see.

On the virtual-host config /etc/httpd/conf.d/dashboard.conf we had before changing to the configs:

Listen *:610
<Virtualhost *:610>
    # General setup for the virtual host, inherited from global configuration
    ServerName https://dashboard.rctsaai.pt

    ErrorLog  "|/usr/bin/logger -p local3.err  -t 'Apache-dashboard'"
    CustomLog "|/usr/bin/logger -p local3.info  -t 'Apache-dashboard'" combined

    RewriteEngine on
    RewriteCond %{REQUEST_URI} !\.html$
    RewriteCond %{REQUEST_URI} !\.js$
    RewriteCond %{REQUEST_URI} !\.svg$
    RewriteCond %{REQUEST_URI} !\.css$
    RewriteCond %{REQUEST_URI} !\.png$
    RewriteCond %{REQUEST_URI} !\.ico$
    RewriteCond %{REQUEST_URI} !\.woff$
    RewriteCond %{REQUEST_URI} !\.woff2$
    RewriteCond %{REQUEST_URI} !\.ttf$
    RewriteCond %{REQUEST_URI} !\.eot$
    RewriteCond %{REQUEST_URI} !dashboard
    RewriteCond %{REQUEST_URI} !health
    RewriteCond %{REQUEST_URI} !info
    RewriteCond %{REQUEST_URI} !fonts
    RewriteRule (.*) /index.html [L]

    ProxyPreserveHost On
    ProxyPass /Shibboleth.sso !
    ProxyPass /dashboard/api http://localhost:9394/dashboard/api retry=0
    ProxyPassReverse /dashboard/api http://localhost:9394/dashboard/api
    ProxyPass /health http://localhost:9394/health retry=0
    ProxyPass /info http://localhost:9394/info retry=0

    DocumentRoot /var/www/dashboard/current

    <Directory /var/www/dashboard/current>
      Order       allow,deny
      Allow       from all
      Options     -Indexes
    </Directory>

    <Location ~ "/dashboard/api/home">
      allow from all
      satisfy any
    </Location>

    <Location ~ "/dashboard/api/css/application.min.css">
      allow from all
      satisfy any
    </Location>

    <Location ~ "/(health|info)">
      allow from all
      satisfy any
    </Location>

    <Location />
      AuthType shibboleth
      ShibUseHeaders On
      ShibRequestSetting applicationId dashboard
      ShibRequireSession On
      ShibRequestSetting REMOTE_ADDR X-Forwarded-For
      require valid-user
    </Location>

</VirtualHost>

Changing the configs we found on https://github.com/OpenConext/OpenConext-deploy/blob/master/roles/dashboard-gui/templates/dashboard.conf.j2

Listen *:610
<Virtualhost *:610>
    # General setup for the virtual host, inherited from global configuration
    ServerName https://dashboard.dev.rctsaai.pt

    ErrorLog  "|/usr/bin/logger -p local3.err   -t 'Apache-dashboard'"
    CustomLog "|/usr/bin/logger -p local3.info  -t 'Apache-dashboard'" combined

    RewriteEngine on
    RewriteCond %{REQUEST_URI} !\.html$
    RewriteCond %{REQUEST_URI} !\.js$
    RewriteCond %{REQUEST_URI} !\.svg$
    RewriteCond %{REQUEST_URI} !\.css$
    RewriteCond %{REQUEST_URI} !\.png$
    RewriteCond %{REQUEST_URI} !\.ico$
    RewriteCond %{REQUEST_URI} !\.woff$
    RewriteCond %{REQUEST_URI} !\.woff2$
    RewriteCond %{REQUEST_URI} !\.ttf$
    RewriteCond %{REQUEST_URI} !\.eot$
    RewriteCond %{REQUEST_URI} !dashboard
    RewriteCond %{REQUEST_URI} !health
    RewriteCond %{REQUEST_URI} !info
    RewriteCond %{REQUEST_URI} !login
    RewriteCond %{REQUEST_URI} !startSSO
    RewriteCond %{REQUEST_URI} !fonts
    RewriteRule (.*) /index.html [L]

    ProxyPreserveHost On
    ProxyPass /Shibboleth.sso !
    ProxyPass /dashboard/api http://localhost:9394/dashboard/api retry=0
    ProxyPassReverse /dashboard/api http://localhost:9394/dashboard/api
    ProxyPass /health http://localhost:9394/health retry=0
    ProxyPass /info http://localhost:9394/info retry=0
    ProxyPass /login http://localhost:9394/login retry=0
    ProxyPass /startSSO http://localhost:9394/startSSO retry=0

    DocumentRoot /var/www/dashboard/current

    <Directory /var/www/dashboard/current>
      Order       allow,deny
      Allow       from all
      Options     -Indexes
    </Directory>

    <Location ~ "^/(?!startSSO)">
      allow from all
      satisfy any
    </Location>

    <Location ~ "/(health|info)">
      allow from all
      satisfy any
    </Location>

    <Location />
      AuthType shibboleth
      ShibUseHeaders On
      ShibRequestSetting applicationId dashboard
      ShibRequireSession On
      ShibRequestSetting REMOTE_ADDR X-Forwarded-For
      require valid-user
    </Location>
</VirtualHost>

After that change any user accessing Dashboard from a browser can see the Services tab without authentication.

Looking to the browser we can see the request and response with the 403 code.

Looking to the logs we don't have any explicit error, only 403.

File: /var/log/haproxy-traffic.log

Nov 20 15:12:15 engine.dev.rctsaai.pt haproxy[2584]: 193.137.196.185:33380 [20/Nov/2019:15:12:15.144] ssl_ip~ manage_be/java 2/0/0/10/12 200 5224 - - --NI 2/2/0/0/0 0/0 {Java/1.8.0_212} "POST /manage/api/internal/search/saml20_idp HTTP/1.1"
Nov 20 15:12:15 engine.dev.rctsaai.pt haproxy[2584]: 193.136.44.97:13337 [20/Nov/2019:15:12:15.098] ssl_ip~ dashboard_be/java 34/0/0/33/67 403 350 - - --VU 1/1/0/0/0 0/0 {Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:70.0) Gecko/20100101 Firefox/70.0} "POST /dashboard/api/services/connect HTTP/1.1"

File: /var/log/messages

Nov 20 11:01:15 engine Apache-Manage: 127.0.0.1 - - [20/Nov/2019:11:01:15 +0000] "GET /manage/api/health HTTP/1.0" 200 15 "-" "-" "127.0.0.1"
Nov 20 11:01:15 engine.dev.rctsaai.pt haproxy[2584]: 193.137.196.185:60534 [20/Nov/2019:11:01:15.867] ssl_ip~ manage_be/java 4/0/0/9/13 200 5224 - - --NI 1/1/0/0/0 0/0 {Java/1.8.0_212} "POST /manage/api/internal/search/saml20_idp HTTP/1.1"
Nov 20 11:01:15 engine Apache-Manage: 127.0.0.1 - - [20/Nov/2019:11:01:15 +0000] "POST /manage/api/internal/search/saml20_idp HTTP/1.1" 200 4646 "-" "Java/1.8.0_212" "193.137.196.185, 127.0.0.1"
Nov 20 11:01:15 engine.dev.rctsaai.pt haproxy[2584]: 193.136.44.97:10001 [20/Nov/2019:11:01:15.757] ssl_ip~ dashboard_be/java 84/0/0/53/137 403 350 - - --VU 0/0/0/0/0 0/0 {Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:70.0) Gecko/20100101 Firefox/70.0} "POST /dashboard/api/services/connect HTTP/1.1"
Nov 20 11:01:15 engine Apache-dashboard: 127.0.0.1 - urn:collab:person:fccn.pt:dgoncalves [20/Nov/2019:11:01:15 +0000] "POST /dashboard/api/services/connect HTTP/1.1" 403 - "https://dashboard.dev.rctsaai.pt/apps/43/saml20_sp/how_to_connect" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:70.0) Gecko/20100101 Firefox/70.0" "193.136.44.97, 127.0.0.1"

File: /var/log/dashboard/dashboard.log

2019-11-20 15:12:15,138 DEBUG [http-nio-9394-exec-8] d.s.ShibbolethPreAuthenticatedProcessingFilter:113 - Checking secure context token: org.springframework.security.web.authentication.preauth.PreAuthenticatedAuthenticationToken@cf2d2e30: Principal: CoinUser{uid=urn:collab:person:fccn.pt:dgoncalves, displayName=Domingos Gonçalves, schacHomeOrganization=fccn.pt, idp=IdentityProvider{id='https://idp.fccn.pt/idp/shibboleth', institutionId='fccn.pt'}, institutionIdps=[IdentityProvider{id='https://idp.fccn.pt/idp/shibboleth', institutionId='fccn.pt'}], institutionId=fccn.pt, email=domingos goncalves at fccn.pt, grantedAuthorities=[CoinAuthority{authority=ROLE_DASHBOARD_SUPER_USER}, CoinAuthority{authority=ROLE_DASHBOARD_ADMIN}], attributeMap={Shib_DisplayName=[Domingos Gonçalves], Shib_EduPersonPN=[dgoncalves at fccn.pt], Shib_MemberOf=[urn:collab:group:dev.rctsaai.pt:servico:dashboard-admin, urn:collab:group:dev.rctsaai.pt:servico:dashboard-super-user, urn:collab:group:dev.rctsaai.pt:servico:dashboard-viewer], Shib_Email=[domingos goncalves at fccn.pt], Shib_Uid=[dgoncalves], Shib_GivenName=[Domingos], Shib_EduPersonAffiliation=[member], Shib_SurName=[Gonçalves], Shib_EduPersonScopedAffiliation=[member at fccn.pt], Shib_HomeOrg=[fccn.pt]}, switchedToIdp=IdentityProvider{id='https://idp.fccn.pt/idp/shibboleth', institutionId='fccn.pt'}}; Credentials: [PROTECTED]; Authenticated: true; Details: org.springframework.security.web.authentication.WebAuthenticationDetails@957e: RemoteIpAddress: 127.0.0.1; SessionId: null; Granted Authorities: CoinAuthority{authority=ROLE_DASHBOARD_SUPER_USER}

2019-11-20 15:12:15,139 DEBUG [http-nio-9394-exec-8] dashboard.manage.UrlResourceManage:215 - Fetching IDP metadata entries from https://manage.dev.rctsaai.pt with body {}

Best regards,

Domingos Gonçalves

sexta-feira, 15 de Novembro de 2019 às 10:28:41 UTC, Okke Harsta escreveu:

Hi,

Dashboard is currently on release 10.0.5. The main difference with the 9.x.x releases is a public landinlage. There were some bugs related to strict security in the 9.x.x releases and the 403 hints that that is the problem. When you upgrade to 10.0.5 ensure to check the differences made in https://github.com/OpenConext/OpenConext-deploy/blob/master/roles/dashboard-gui/templates/dashboard.conf.j2#L50.

Best regards,

Okke

To unsubscribe from this group and stop receiving emails from it, send an email to openc...@googlegroups.com.

[Okke Harsta]

Hi,

My bad. I did mention it in my previous mail: “The main difference with the 9.x.x releases is a public landinlage", but probably better if I had explained it more thorough. Unauthenticated users can see the services and to some level also the details. This is intentionally and was on the product backlog of Dashboard. We of course don’t know if this is also what you guys want. On the CC the Product Owner of Dashboard who perhaps could be persuaded to introduce a feature toggle if desired;-)

Regards,

Okke

On 20 Nov 2019, at 16:43, Domingos Gonçalves <domingos....@gmail.com> wrote:

Hi,

we checked the differences o the Dashboard virtual-host you mention.

Performing that changes, we didn't solve the send email problem but, we create another one. When accessing Dashboard, a non-authenticated access didn't forward the browser to the proxy wayf to proceed to the authentication, but give us a direct access to the Services tab. You can tested right now if you want to see.

To unsubscribe from this group and stop receiving emails from it, send an email to openconext+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openconext/d3a408c5-9d03-403d-b463-6ee5e3a4bb7f%40googlegroups.com.
<Dashboard-services-tab-direct-access.png><Dashboard-error-pressing-button.png>

[Raoul Teeuwen]

Hi all.

 

As current product manager for the IdP dashboard, i apologize i did not involve the OpenConext community before. In hind sight, i guess i should have done this as soon as we started considering this change to the IdP dashboard. To be honest, until now, i have never thought about involving the OpenConext list, and this brings up a discussion in general about how and when to involve the community... Maybe we need to discuss this at some moment.

 

Anyway, in the latest version of the IdP Dashboard, we have chosen to show all services connected to our platform without the need to sign in to the dashboard. Also, anyone from an institution can sign in to get some more info. And an institution can configure in the IdP dashboard whether ‘anyone from their institution’ can see the statistics page in the IdP dashboard, and whether ‘anyone from their institution’ can see contact info of SURFconext managers in that institution.

 

We have implemented this to have an up to date public page that lists services connected to our platform, as over time we often had people asking about such a list. Also, we know often people like from a library within an institution want to know what services are available as well as what are connected to their institution, as well as to have info on service usage info. So instead of having to implement one or more roles, we decided to solve it as described.

 

In the future, i will try to better weigh when to involve the OpenConext list. Also, if you guys have a reason to update to the latest version, but having this info public poses a problem, please let us know. As Okke suggested, we can look into a feature toggle, for instance. Good to know: for this year we don’t have money for more IdP dashboard work, i’m sorry to say. But feel free to chip in and hire Okke ;-).

 

With kindest regards.

Raoul

SURF

On Thursday, November 21, 2019 at 9:46:24 AM UTC+1, Okke Harsta wrote:

Hi,

My bad. I did mention it in my previous mail: “The main difference with the 9.x.x releases is a public landinlage", but probably better if I had explained it more thorough. Unauthenticated users can see the services and to some level also the details. This is intentionally and was on the product backlog of Dashboard. We of course don’t know if this is also what you guys want. On the CC the Product Owner of Dashboard who perhaps could be persuaded to introduce a feature toggle if desired;-)

Regards,

Okke

To view this discussion on the web visit https://groups.google.com/d/msgid/openconext/d3a408c5-9d03-403d-b463-6ee5e3a4bb7f%40googlegroups.com.
<Dashboard-services-tab-direct-access.png><Dashboard-error-pressing-button.png>