Any experience with iRODS for Backend and INVENIO for Frontend ?

[MBANZA Sara]

Hello everyone,

My name is Sara, I'm working on a Data Management project.

For Backend, I have used iRODS and used PAM Authentication. Everything is working very well. The user can use the iRODS client "iCommands" with linux commands to interact with iRODS Server. Moreover, it is also possible to use python API for IRODS: https://github.com/irods/python-irodsclient.

I' m a beginner regarding building a GUI (graphical user interface). Sorry for that. Now I would like to build a GUI  Frontend based on INVENIO framework (https://inveniosoftware.org/) and I' m wondering if this is possible. Because iRODS has this python API https://github.com/irods/python-irodsclient and INVENIO based on python, I think that it could maybe be possible to connect both systems (iRODS backend and INVENIO frontend). I don't know if I' m right or wrong. If possible, how could this be achieved? If someone has an experience with this, any idea/suggestion/guidance/... would be appreciated. If not possible by using INVENIO,  any suggestion/... to build a GUI by using python would be also appreciated.

Thank you.

Best Regards  Sara

[Nyongha Rose]

Hi,

Me too I' m  interested in whether there is/could be an iRODS-INVENIO integration.

[Kory Draughn]

Hi Nyongha,

There's likely always some level of integration that can be achieved between software systems. I'm not familiar with Invenio so I can't say what an integration would look like.

Are you using Invenio?

What issues are you looking to solve via an integration?

Kory Draughn

Chief Technologist

iRODS Consortium

--
--
The Integrated Rule-Oriented Data System (iRODS) - https://irods.org
 
iROD-Chat: http://groups.google.com/group/iROD-Chat
---
You received this message because you are subscribed to the Google Groups "iRODS-Chat" group.
To unsubscribe from this group and stop receiving emails from it, send an email to irod-chat+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/irod-chat/95ed2903-5f96-4654-bc7a-6ac43aa85cb7n%40googlegroups.com.

[Nyongha Rose]

Hi Kory,

Thanks for the reply.

I am not yet using Invenio/InvenioRDM. But I plan to use it to build a repository for publishing research data using InvenioRDM. 

The problem first is to know if I will then be able to integrate InvenioRDM with iRODS, so that the user can publish the data from iRODS via InvenioRDM.

KR,

Rose

[Jean-Yves Nief]

hello Rose,

            as many web apps, the most obvious interface that is
available is through a local file system or any mounted filesystem on
the server running InvenioRDM. In iRODS case, one solution would to use
irods fuse, so the integration of iRODS with InvenioRDM would be pretty
straightforward. It is a suggestion as I only did some testing lately
and I don't know the kind of load that you will experience on your
system. That could be enough... or not. Maybe, it is interesting to give
it a try before considering an other solution.
cheers,
JY

> <https://github.com/irods/python-irodsclient>.

> <https://groups.google.com/d/msgid/irod-chat/95ed2903-5f96-4654-bc7a-6ac43aa85cb7n%40googlegroups.com?utm_medium=email&utm_source=footer>.

>
> --
> --
> The Integrated Rule-Oriented Data System (iRODS) - https://irods.org
>
> iROD-Chat: http://groups.google.com/group/iROD-Chat
> ---
> You received this message because you are subscribed to the Google
> Groups "iRODS-Chat" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to irod-chat+...@googlegroups.com

> <mailto:irod-chat+...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/irod-chat/0a0f6aa0-f410-4507-8ed8-7679703d79c4n%40googlegroups.com
> <https://groups.google.com/d/msgid/irod-chat/0a0f6aa0-f410-4507-8ed8-7679703d79c4n%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Kory Draughn]

Nyongha,

There are likely many ways to integrate the two systems.

I think there are two things that need to happen before we can make any assertions.

• First, you must learn what Invenio needs/expects for publishing
• Second, you must brain out how you would like the systems to work together

Once you have that, we'll have a better picture of how to approach the integration.

Kory Draughn

Chief Technologist

iRODS Consortium

To view this discussion on the web visit https://groups.google.com/d/msgid/irod-chat/3c2bb0fd-c4f3-4da2-c8ff-484005e94665%40cc.in2p3.fr.

[Nyongha Rose]

Hi Jean_Yves, Hi Kory,

Thank you for your messages. I will work first on these messages and then get back.

KR,

Rose

[Bazi Jana]

Hi,

Has anyone tried integrating iRODS and invenioRDM using irods fuse (as suggested by Jean-Yves)? If so: 

1. Does it work without problems, so that I can also use it in the production system?
2. How can I  proceed/set/configure to make this integration working?

For the info: Currently, each system has its own storage: iRODS uses NFS Share that I mounted on the iRODS server, and invenioRDM uses NFS Share that I mounted on the invenioRDM server.

Kind Regards,

Jana

[jc...@sanger.ac.uk]

I've not got experience of Invenio, but I do have advice, which is to avoid FUSE for production systems and instead look to NFSRODS in this situation.

My issue with fuse is that they are often  poor approximations of POSIX filesystems and tend to fail in unpredictable ways which have led to data loss and corruption. 

Depending on implementation they also make it hard to attribute access to users, which may or may not be a concern  for your use case.

I have implemented NFSRODS as a POSIX interface to IGV and similar genome viewers and that works fine.

Hope that helps!

John

[Bazi Jana]

Hi John,

Many thanks. I will check NFSRODS and try it ...

Kind Regards,

Jana

[Lenoci, L. (Leonardo)]

I don't have this in production, but after a quick test, I would select Invenio's MinIO S3 storage backend upon installation. This latter is easy to manage using Invenio's cmd line tools and can also be configured as a storage resource in iRODS. In fact, there are several iRODS production instances using S3 storage resources. As to the performance of Invenio/S3/iRODS, there are too many factors in this equation, including the infrastructure 

on which you are planning to run it.

-- 

Sincerely yours,

Leonardo Lenoci, PhD

ICT Research and Security Advisor

Leiden University | Faculty of Science 

Contact me in private:

https://keys.openpgp.org/search?q=l.le...@science.leidenuniv.nl

Personal Website:

https://social.edu.nl/@the_dr_leonardo_lenoci

--

[Lenoci, L. (Leonardo)]

If anyone is still interested in connecting InvenioRDM to iRODS, please
have a look at this proof-of-concept

https://github.com/ll4strw/invenio-irods

I wouldn't guarantee it is production-ready, but it does what it is
supposed to: provide an  
iRODS file storage backend to InvenioRDM. It uses the iRODS python
client (RPC)
file-like objects for reading and writing files.

I know a ton of researchers and data managers who would benefit from
using InvenioRDM (Zenodo)
alongside iRODS.

The next step is to tackle metadata exchanges between the two system,
but this PoC should 
get you there in little time.
I hope this helps.

--
Sincerely yours,

Leonardo Lenoci, PhD
ICT Research and Security Advisor

Leiden University | Faculty of Science 

Contact me in private:
https://keys.openpgp.org/search?q=l.le...@science.leidenuniv.nl

Personal Website:
https://social.edu.nl/@the_dr_leonardo_lenoci

[Nyongha Rose]

Many thanks @Lenoci.

[Terrell Russell]

I have added Invenio-iRODS to https://irods.org/clients

Terrell

> --
> --
> The Integrated Rule-Oriented Data System (iRODS) - https://irods.org
>
> iROD-Chat: http://groups.google.com/group/iROD-Chat
> ---
> You received this message because you are subscribed to the Google Groups "iRODS-Chat" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to irod-chat+...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/irod-chat/72d98e5b-55ef-4937-ab82-db35fd1dae96n%40googlegroups.com.