Adding other Oauth2 providers

73 views
Skip to first unread message

Ruben Andreassen

unread,
Nov 16, 2017, 5:07:05 AM11/16/17
to Dataverse Users Community
Hi, this is my first question here.

Is it possible to add other Oauth providers to Dataverse?

I work as a developer at a University in Norway, and we have a national login portal that we want to use:

Would it be possible without to much hassle?

Philip Durbin

unread,
Nov 16, 2017, 7:39:18 AM11/16/17
to dataverse...@googlegroups.com
Hi! Welcome! Are you part of https://dataverse.no ? They're using Feide and Shibboleth/SAML.

If you were to follow the model of the existing OAuth providers (ORCID, GitHub, and Google), I don't think it would be too difficult to add Twitter or Facebook or any other OAuth provider.

I don't mean to encourage forks, but would you plan to fork the code (and maintain that fork over time) or are you thinking you'd like to make a pull request?

Phil

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/00c23b72-ebb1-4bf3-abdf-8337e3d4b561%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--

Ruben Andreassen

unread,
Nov 16, 2017, 9:33:48 AM11/16/17
to Dataverse Users Community
Yes. dataverse.no is a running on a local VM here at the University and are using Feide and Shibboleth.

We are now having a project to rig Dataverse in a cloud provider and at the same time use S3 as storage and OAuth as autentication.

If we need to implement our national OAuth provider our self (sounds like it) we could do a merge request. We are not aiming to maintain a seperat fork of Dataverse.


-Ruben


On Thursday, November 16, 2017 at 1:39:18 PM UTC+1, Philip Durbin wrote:
Hi! Welcome! Are you part of https://dataverse.no ? They're using Feide and Shibboleth/SAML.

If you were to follow the model of the existing OAuth providers (ORCID, GitHub, and Google), I don't think it would be too difficult to add Twitter or Facebook or any other OAuth provider.

I don't mean to encourage forks, but would you plan to fork the code (and maintain that fork over time) or are you thinking you'd like to make a pull request?

Phil
On Thu, Nov 16, 2017 at 5:07 AM, Ruben Andreassen <rube...@gmail.com> wrote:
Hi, this is my first question here.

Is it possible to add other Oauth providers to Dataverse?

I work as a developer at a University in Norway, and we have a national login portal that we want to use:

Would it be possible without to much hassle?

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse...@googlegroups.com.

Philip Durbin

unread,
Nov 16, 2017, 10:53:59 AM11/16/17
to dataverse...@googlegroups.com
Cool, I'm glad you know each other. Say hi to Phillip and Obi for me. :)

We at IQSS certainly do not have the cycles to develop the OAuth provider for Norway for you. We can certainly help you get up to speed with the code. I guess I was thinking that someone would come along offering to implement support for logging in to Dataverse via Twitter or Facebook but it makes sense that there's demand for being able to login to a national OAuth provider.

Can you please create an issue at https://github.com/IQSS/dataverse/issues about what you're proposing? That way we can start tracking the work and I can link to it from the "Dev Efforts by the Dataverse Community" spreadsheet I introduced at https://groups.google.com/d/msg/dataverse-community/X2diSWYll0w/ikp1TGcfBgAJ

We do all of our estimation by talking about specific GitHub issues, so in a future backlog grooming meeting, perhaps the IQSS team can talk about how much work on our part it would be. To start we would mostly be guiding you around the code and documentation until you're comfortable. Then there's code review and QA, as represented on our kanban board at https://waffle.io/IQSS/dataverse . I'm not sure how we'd even test your code without you giving us credentials to your national OAuth provider. I get slightly nervous about having code in our source tree that we aren't sure works or aren't sure will work after we refactor related code in the future.

Speaking of code, you're welcome to review the pull request at https://github.com/IQSS/dataverse/pull/4280 where we're introducing support for the latest ORCID OAuth API (from v1.2 to v2.0). It will help you see some of the moving parts for OAuth.

Since you mentioned S3, you might be interested to know that Harvard Dataverse is moving from physical servers to AWS and S3. We're working on a backup script from S3 to secondary storage if that's of interest: https://github.com/IQSS/dataverse/pull/4271

I hope this helps. Please feel to pop in http://chat.dataverse.org (#dataverse on freenode) if you have any questions. You also might want to subscribe to the dataverse-dev list at https://groups.google.com/forum/#!forum/dataverse-dev

Thanks,

Phil

On Thu, Nov 16, 2017 at 9:33 AM, Ruben Andreassen <rube...@gmail.com> wrote:
Yes. dataverse.no is a running on a local VM here at the University and are using Feide and Shibboleth.

We are now having a project to rig Dataverse in a cloud provider and at the same time use S3 as storage and OAuth as autentication.

If we need to implement our national OAuth provider our self (sounds like it) we could do a merge request. We are not aiming to maintain a seperat fork of Dataverse.


-Ruben


On Thursday, November 16, 2017 at 1:39:18 PM UTC+1, Philip Durbin wrote:
Hi! Welcome! Are you part of https://dataverse.no ? They're using Feide and Shibboleth/SAML.

If you were to follow the model of the existing OAuth providers (ORCID, GitHub, and Google), I don't think it would be too difficult to add Twitter or Facebook or any other OAuth provider.

I don't mean to encourage forks, but would you plan to fork the code (and maintain that fork over time) or are you thinking you'd like to make a pull request?

Phil
On Thu, Nov 16, 2017 at 5:07 AM, Ruben Andreassen <rube...@gmail.com> wrote:
Hi, this is my first question here.

Is it possible to add other Oauth providers to Dataverse?

I work as a developer at a University in Norway, and we have a national login portal that we want to use:

Would it be possible without to much hassle?

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsubscribe...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsubscribe...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/fdcf03fa-1aa7-4a0c-a7d0-85d5805924a8%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Durand, Gustavo

unread,
Nov 16, 2017, 4:43:50 PM11/16/17
to dataverse...@googlegroups.com
Hi,

Could I ask for a little more information on your national OAuth provider? The reason I ask with additions like this, we've been trying to be "smarter" about whether it belongs in the core code or not. To that end, we've been working on SPIs (Service Provider Interfaces) for areas of functionality where we want to allow for modular additions. In other words, by making the core code modular, we can make sure that the core code stays lean (only incorporating general pieces of functionality useful to many installations), while allowing individual installations to run modules that would be specific for there use.

Authentication is obviously ripe for that kind of modularity. In this case, if your Oauth provider is something that only your installation would use, then the what should happen is that the core code is modified to make Authentication (or even maybe more specifically Oauth authentication) modular via SPI, and then this Oauth provider code would live outside of the core.

You would then still be able to use the core .war file of the application with your Oauth module.

Gustavo

On Thu, Nov 16, 2017 at 10:53 AM, Durbin, Philip <philip...@harvard.edu> wrote:
Cool, I'm glad you know each other. Say hi to Phillip and Obi for me. :)

We at IQSS certainly do not have the cycles to develop the OAuth provider for Norway for you. We can certainly help you get up to speed with the code. I guess I was thinking that someone would come along offering to implement support for logging in to Dataverse via Twitter or Facebook but it makes sense that there's demand for being able to login to a national OAuth provider.

Can you please create an issue at https://github.com/IQSS/dataverse/issues about what you're proposing? That way we can start tracking the work and I can link to it from the "Dev Efforts by the Dataverse Community" spreadsheet I introduced at https://groups.google.com/d/msg/dataverse-community/X2diSWYll0w/ikp1TGcfBgAJ

We do all of our estimation by talking about specific GitHub issues, so in a future backlog grooming meeting, perhaps the IQSS team can talk about how much work on our part it would be. To start we would mostly be guiding you around the code and documentation until you're comfortable. Then there's code review and QA, as represented on our kanban board at https://waffle.io/IQSS/dataverse . I'm not sure how we'd even test your code without you giving us credentials to your national OAuth provider. I get slightly nervous about having code in our source tree that we aren't sure works or aren't sure will work after we refactor related code in the future.

Speaking of code, you're welcome to review the pull request at https://github.com/IQSS/dataverse/pull/4280 where we're introducing support for the latest ORCID OAuth API (from v1.2 to v2.0). It will help you see some of the moving parts for OAuth.

Since you mentioned S3, you might be interested to know that Harvard Dataverse is moving from physical servers to AWS and S3. We're working on a backup script from S3 to secondary storage if that's of interest: https://github.com/IQSS/dataverse/pull/4271

I hope this helps. Please feel to pop in http://chat.dataverse.org (#dataverse on freenode) if you have any questions. You also might want to subscribe to the dataverse-dev list at https://groups.google.com/forum/#!forum/dataverse-dev

Thanks,

Phil


On Thu, Nov 16, 2017 at 9:33 AM, Ruben Andreassen <rube...@gmail.com> wrote:
Yes. dataverse.no is a running on a local VM here at the University and are using Feide and Shibboleth.

We are now having a project to rig Dataverse in a cloud provider and at the same time use S3 as storage and OAuth as autentication.

If we need to implement our national OAuth provider our self (sounds like it) we could do a merge request. We are not aiming to maintain a seperat fork of Dataverse.


-Ruben


On Thursday, November 16, 2017 at 1:39:18 PM UTC+1, Philip Durbin wrote:
Hi! Welcome! Are you part of https://dataverse.no ? They're using Feide and Shibboleth/SAML.

If you were to follow the model of the existing OAuth providers (ORCID, GitHub, and Google), I don't think it would be too difficult to add Twitter or Facebook or any other OAuth provider.

I don't mean to encourage forks, but would you plan to fork the code (and maintain that fork over time) or are you thinking you'd like to make a pull request?

Phil
On Thu, Nov 16, 2017 at 5:07 AM, Ruben Andreassen <rube...@gmail.com> wrote:
Hi, this is my first question here.

Is it possible to add other Oauth providers to Dataverse?

I work as a developer at a University in Norway, and we have a national login portal that we want to use:

Would it be possible without to much hassle?

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsubscribe...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsubscribe...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/fdcf03fa-1aa7-4a0c-a7d0-85d5805924a8%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.

Ruben Andreassen

unread,
Nov 17, 2017, 2:36:42 AM11/17/17
to Dataverse Users Community

Thanks for great info guys!

Everyone who uses Feide today will be using "Dataporten" in the future. Its kind of the next gen in the national SSO solution.

There are many institutions using Feide today, and will use Dataporten in the furure, but how many Dataverse-installations there will be is probably limited.

Since we have had success with dataverse.no we want to scale it up to a national level so more educational institutions can use it without setting it up them self.

But every university in Norway could benefit from OAuth integration with Dataporten if they want to host their own Dataverse-installation instead of using the one we plan to set up.

Philip Durbin

unread,
Nov 17, 2017, 7:49:47 AM11/17/17
to dataverse...@googlegroups.com
I'm glad you've had so much success with DataverseNO! I still sometimes think about the TROLLing video about how important it is for linguists to share data: https://www.youtube.com/watch?v=uEf0c0NT9_A

I'm not sure if you've thought much about how you'll transition your users from one authentication provider to another but right now the supported way would be to migrate each user from Feide (Shibboleth/SAML) to a "builtin" user and they have each user pick an OAuth provider to convert from "builtin" to OAuth. This is cumbersome, obviously. Early on in Dataverse 4 development we had a dream to allow multiple logins to the same Dataverse account so that you could still be you when you login via various auth providers (like how Stack Overflow works, if you're familiar). There some more information about this idea at https://github.com/IQSS/dataverse/issues/3487

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.
To post to this group, send email to dataverse-community@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Michael Bar-sinai

unread,
Nov 17, 2017, 9:26:15 AM11/17/17
to Dataverse Users Community
Hello Ruben,

Welcome to Dataverse's Google group! The coding itself should not be much of a hassle - we already have three implementations to 
build upon, and I assume dataporten.no would resemble one of them. The big question is how we integrate the dataporten.no adapter 
with the rest Dataverse. As Gustavo mentioned, we plan on moving to a plug-in architecture where functionalities like these will be 
added dynamically by dropping a .jar in the right place. But I'm not sure when this will happen.

Thus, the biggest question is, (as usual?) what's your timeline? When do you need this done?

Ruben Andreassen

unread,
Nov 20, 2017, 2:03:15 AM11/20/17
to Dataverse Users Community
Since you know so much of dataverseNO I must say that I was not directly involved in that project. I only started working at the University a few months ago and got pulled into the "cloudification" of that project.

Good to know about he migration process between signin methods. 

The timeline in this project is a little diffuse, but I can use as much time as possible on this project this year. Next year I don't know, but maybe a month or so (not timeline, work hours).

There are three goals:
- Move Dataverse installation to the cloud (VM or containers on Kubernetes)
- Integrate with scalable storage (S3 or other)
- Dataporten as authentication

I have spent a few days on running Dataverse in containers, looking at your discussion on Github (https://github.com/IQSS/dataverse/issues/4040) and the work of NDS Labs. My conclusion was to wait it out and focus on the other to goals of the project.

If you believe it would be best to wait a little with the implementation of Dataporten, I will focus my effort on the storage goal first.

I'm not doing all the work my self off course. We have one other developer on the team (Lars T) and one from IT Operations.

danny...@g.harvard.edu

unread,
Nov 20, 2017, 10:44:04 AM11/20/17
to Dataverse Users Community
Hey Ruben, good to hear about your plans for Dataverse. As Phil mentioned, Harvard Dataverse is moving to S3 and we already have the code to support running Dataverse on S3 as of 4.8 (https://github.com/IQSS/dataverse/releases/tag/v4.8). I think this makes sense for you as a first step. We'd be interested to hear your experiences and any ways that the S3 documentation (or code :)) can be improved.

Cheers,

Danny

Philip Durbin

unread,
Nov 20, 2017, 9:06:11 PM11/20/17
to dataverse...@googlegroups.com
Hi Ruben (and others),

In addition to hearing your experiences with S3 (like Danny said), since you mentioned issue #4040 about Docker/OpenShift/Kubernetes I'd like to say that I was the one who was hacking away on that issue and I didn't get as far as I hoped, which is why we put #3938 through QA instead, which is much more modest and about getting Dataverse developers comfortable running Docker. #4040 is still open because we know there's more work to do to make Docker/OpenShift/Kubernetes work for Dataverse users but we could use some help from the community. I'm new at all this Docker stuff so if you have any suggestions on how you'd like to see the Docker images evolve, please leave a comment on #4040.

Meanwhile, you should be able to run Dataverse just fine in VMs or whatever. https://demo.dataverse.org for example, already runs on AWS EC2.

Thanks,

Phil

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsubscribe...@googlegroups.com.

To post to this group, send email to dataverse-community@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Ruben Andreassen

unread,
Nov 30, 2017, 7:38:36 AM11/30/17
to Dataverse Users Community
Ok, so I have now started hacking away on this. 

First problem: I have a developer VM that I access in something like this: http://158.39.77.134:8080/

But the "redirect_uri" is somehow generated to redirect_uri=https%3A%2F%2F158.39.77.134

That is the wrong protocol (HTTPS instead of HTTP) and missing the port. 

Is there some place in the code I can hardcode my preferred redirect_uri for testing purposes?

Philip Durbin

unread,
Nov 30, 2017, 7:53:54 AM11/30/17
to dataverse...@googlegroups.com
I would suggest trying to set your "dataverse.siteUrl" JVM option to "http://158.39.77.134:8080"

You can read more about this setting at http://guides.dataverse.org/en/4.8.3/installation/config.html#dataverse-siteurl

I hope this helps. Please keep the questions coming. We can switch to the dataverse-dev list or IRC channel I mentioned if this starts to get pretty involved.

Phil

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-community+unsub...@googlegroups.com.

To post to this group, send email to dataverse-community@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/1993cf87-05dd-40f4-9bee-08533b91555f%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages