How To Tell when Configuration Files have Changed

62 views
Skip to first unread message

Sherry Lake

unread,
Feb 15, 2019, 11:49:31 AM2/15/19
to Dataverse Users Community


UVa uses Shibboleth only log it.

Works fine, but after upgrading (4.9.4) we realized that the idpselect_config.js file contents had changed. Not sure when it had changed.

We have configured the idpselect_config.js  file, as the file comments say - "adjust the following to fit into your local configuration". So our practice has been that after each upgrade of production dataverse, we put a copy of "UVa's"  idselect_config.js file on the the server.

After upgrading to 4.9.4 AND adding a new home page, I noticed that logins were not going back (redirecting) to the new home page. After reviewing the idpselect_config file, I found new lines that we our modified file didn't have (which make the login go back to the home page). 

We have updated our idpselect_config file and have production now redirecting as expected.  So we are good....

But was wondering how do I keep up with changing config files? If I have a modified config file for local mods, how do I know if the next release has changed the config file?

Just trying to keep up with you guys and all the changes is hard. 😇

Thanks
Sherry Lake
UVA

Philip Durbin

unread,
Feb 15, 2019, 12:54:01 PM2/15/19
to dataverse...@googlegroups.com
Hi Sherry,

I feel the same way about Solr. So many releases. It's hard to keep up. :)

We should probably back up to why you need to change idpselect_config.js in the first place. I know there are a lot of clever hackers who run Dataverse, but I didn't know people like you were hacking on this file. Can you please open a GitHub issue about the fact that you need more configuration options? I'm not sure what you're changing or why. I'm sure that "Adjust the following to fit into your local configuration" came with the file when we downloaded it from the Shibboleth project. We can remove that message if it's confusing. :)

For your more general question of how to tell when config files of changed, you should check Dataverse release notes for any changes you need to be aware of. But again, idpselect_config.js is a file we expect that you'll leave alone. I'm sure you're changing it for a perfectly valid reason and I'm looking forward to hearing more.

I hope this helps!

Phil

p.s. idpselect_config.js was last modified for Dataverse 4.7 and we mentioned it in the release notes: https://github.com/IQSS/dataverse/releases/tag/v4.7

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/992955b4-0d86-4d8c-8530-a809da1101cb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


--

Sherry Lake

unread,
Feb 15, 2019, 3:33:49 PM2/15/19
to Dataverse Users Community
Hi Phil,

Thanks for the info. I thought that the idpselect_config file was probably updated during the homepage addition.

As to why the file was changed... I have no idea, UVa was [one of] the 1st to use Shibboleth in production. The file was changed during our very 1st install. Maybe our system admin saw the comment "adjust the following..."?
Our shibboleth config, only allow UVa users to log in.

What does Harvard have on the line: "this.dataSource"  ?
UVa has the following:
    this.dataSource = '/resources/dev/uva-shib-identities.json';

Which, is why my sysadmin has been replacing the whole idpselect_config file. Our UVa workflow for upgrading Dataverse includes putting a copy of the uva-shib-identities.json file in the directory /resource/dev. But maybe we just need to update this one line (this.dataSource) in the file?

--
Sherry


On Friday, February 15, 2019 at 12:54:01 PM UTC-5, Philip Durbin wrote:
Hi Sherry,

I feel the same way about Solr. So many releases. It's hard to keep up. :)

We should probably back up to why you need to change idpselect_config.js in the first place. I know there are a lot of clever hackers who run Dataverse, but I didn't know people like you were hacking on this file. Can you please open a GitHub issue about the fact that you need more configuration options? I'm not sure what you're changing or why. I'm sure that "Adjust the following to fit into your local configuration" came with the file when we downloaded it from the Shibboleth project. We can remove that message if it's confusing. :)

For your more general question of how to tell when config files of changed, you should check Dataverse release notes for any changes you need to be aware of. But again, idpselect_config.js is a file we expect that you'll leave alone. I'm sure you're changing it for a perfectly valid reason and I'm looking forward to hearing more.

I hope this helps!

Phil

p.s. idpselect_config.js was last modified for Dataverse 4.7 and we mentioned it in the release notes: https://github.com/IQSS/dataverse/releases/tag/v4.7

On Fri, Feb 15, 2019 at 11:49 AM Sherry Lake <shla...@gmail.com> wrote:


UVa uses Shibboleth only log it.

Works fine, but after upgrading (4.9.4) we realized that the idpselect_config.js file contents had changed. Not sure when it had changed.

We have configured the idpselect_config.js  file, as the file comments say - "adjust the following to fit into your local configuration". So our practice has been that after each upgrade of production dataverse, we put a copy of "UVa's"  idselect_config.js file on the the server.

After upgrading to 4.9.4 AND adding a new home page, I noticed that logins were not going back (redirecting) to the new home page. After reviewing the idpselect_config file, I found new lines that we our modified file didn't have (which make the login go back to the home page). 

We have updated our idpselect_config file and have production now redirecting as expected.  So we are good....

But was wondering how do I keep up with changing config files? If I have a modified config file for local mods, how do I know if the next release has changed the config file?

Just trying to keep up with you guys and all the changes is hard. 😇

Thanks
Sherry Lake
UVA

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.

Philip Durbin

unread,
Feb 15, 2019, 5:58:38 PM2/15/19
to dataverse...@googlegroups.com
Huh, well, the answer to your first question is easy. Harvard Dataverse has this...

    this.dataSource = window.location.hostname === 'localhost' ? '/resources/dev/sample-shib-identities.json' : '/Shibboleth.sso/DiscoFeed'; // where to get the data from (dev vs. prod)



... which is to say that we don't modify idpselect_config.js

I added that weird looking "dev/sample-shib-identities.json" option back during development. It's only for "localhost", for development. In production, the "dataSource", the list of Identity Providers (IdP), is supposed to come from "/Shibboleth.sso/DiscoFeed" ... from (for example) https://dataverse.lib.virginia.edu/Shibboleth.sso/DiscoFeed . UVa only has one IdP in the "DiscoFeed" list. (Mike, if you're reading this, it's for "discovery feed" and has nothing to do with Disco Stu*.) Other installations of Dataverse have multiple IdPs in this list.

I'm not sure if this background helps or not. I guess I'm mostly just saying that the "stock" idpselect_config.js file works fine in production for Harvard Dataverse and I'm not sure why a tweak is necessary for UVa.

Phil




To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.

To post to this group, send email to dataverse...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/7f71b2b1-b4c6-4af2-ba9d-d8547f8d3746%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

danny...@g.harvard.edu

unread,
Feb 15, 2019, 6:21:42 PM2/15/19
to Dataverse Users Community
Hey Sherry,

Specific Shibboleth issue aside, you're 100% correct there are lots of configuration options to keep up with and lots of new features introduced with each release. The Community members helping with #dataverse2019 planning have expressed this concern as well, and have suggested a workshop at the meeting to do a better job with communications around the different options in Dataverse. Keep an eye out for the workshop schedule, and if you have questions in the meantime please check in here or through one of the other communication channels. 

Have a good weekend! 

- Danny

Sherry Lake

unread,
Feb 22, 2019, 1:28:02 PM2/22/19
to dataverse...@googlegroups.com
Hi Phil,

Getting back to Shibb configuration. Where is this file: https://dataverse.lib.virginia.edu/Shibboleth.sso/DiscoFeed

I've looked, but cannot find it. I want to do some testing on our test box for DiscoFeed content, but cannot find the file to edit. Or is it created via a script?

Thanks.
Sherry

To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CABbxx8GPAZysLo4f%2B3xiELXMYRsMqz4M_-kAOLQUod40dsjP1g%40mail.gmail.com.

Don Sizemore

unread,
Feb 22, 2019, 1:37:52 PM2/22/19
to dataverse...@googlegroups.com
Hi Sherry,

This is generated by Shibboleth:

I hope you are well!
Donald


To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CADL9p-Ug79%3D%3D%3DyhEwrq8839u23Pf7i_Up5LbLzLvxpkQfAx%2Bzg%40mail.gmail.com.

Philip Durbin

unread,
Feb 25, 2019, 1:57:31 PM2/25/19
to dataverse...@googlegroups.com
Yep, the only way to edit the DiscoFeed JSON that I know of is to edit the underlying XML file(s) for your IdP(s). In the guides[1], we storing one or more IdPs (Identity Providers) in a single XML file at /etc/shibboleth/dataverse-idp-metadata.xml but Don showed me (years ago now) that one can cleanly put each IdP in its own XML file but I haven't gotten around to updating the guides. Anyway, since you only have one IdP, Sherry, if you're still using the suggested file location at /etc/shibboleth/dataverse-idp-metadata.xml you should be able to edit that XML file, restart shibd, and see changes in the DiscoFeed JSON.

The DiscoFeed for Harvard Dataverse and UNC Dataverse changes all the time because rather than using local XML files, we are subscribed to the InCommon feed of IdPs. So as new IdPs are added to InCommon (in the Research & Scholarship category), the appear in the DiscoFeed and the dropdown in the UI. In the guides we talk about this under "Identity Federation".

I hope this helps,

Phil




To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/CAPfMOawifP-tm%2BFhGPzC3aZeq5XgHLaMEk3u1kxxn64YEso5UQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

Sherry Lake

unread,
Jun 11, 2019, 3:00:57 PM6/11/19
to Dataverse Users Community
Hey Danny,

I've been combing through release "issues" to try to determine changes (such as there is no longer a piwikAnalyticsHost setting - didn't know, as of 4.11).

So I am really interested in a workshop like you mention below. Is there one next week?

Thanks.
Sherry

Danny Brooke

unread,
Jun 12, 2019, 6:00:54 PM6/12/19
to Dataverse Users Community
Hi Sherry,

We're nailing down the workshop schedule and it should be finalized by the end of the week, but this will for sure be a topic. 

We can definitely do a better job of calling attention to new features and changes, but things like the analytics changes that you mention will be called out in the release notes for the version in which they occur (4.11 here). We usually keep the message around in the following release as well, just to be sure it's not missed (like in 4.12). There shouldn't be a need to go through individual issues to determine the necessary changes, as they should be captured in the release notes.

See you next week!

- Danny
Reply all
Reply to author
Forward
0 new messages