Running nextflow pipeline through private github server

[garima kushwaha]

Hi nextflow users!

I have my pipeline in a repository hosted on a private github server. But I am not able to run/pull my project with Nextflow using the following command:  

 nextflow run kushwahg/run_from_github_repo -user kushwahg

where 'kushwahg' is the owner and 'run_from_github_repo' is the repository name.

Error message that I am getting:

N E X T F L O W  ~  version 0.22.2

 Pulling kushwahg/run_from_github_repo ...

Cannot find `kushwahg/run_from_github_repo` -- Make sure exists a GitHub repository at this address ....

I have also tried setting my credentials in a SCM configuration file '$HOME/.nextflow/scm'  and got the same error message.

The .nextflow.log shows that it is looking for 'main.nf' and 'nextflow.config' file with 'repos' and 'contents' as additional directories in its path.  Nextflow.log looks like the following: 

Nov-15 14:51:03.069 [main] DEBUG nextflow.cli.Launcher - $> /usr/local/bin/nextflow run http://**.**.**.**/kushwahg/run_from_github_repo -user kushwahg
Nov-15 14:51:03.883 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 0.22.2
Nov-15 14:51:04.629 [main] DEBUG nextflow.scm.AssetManager - Repository URL: http://**.**.**.**/kushwahg/run_from_github_repo; Project: kushwahg/run_from_github_repo; Hub provider: roche_git
Nov-15 14:51:07.573 [main] INFO  nextflow.cli.CmdRun - Pulling kushwahg/run_from_github_repo ...
Nov-15 14:51:07.573 [main] DEBUG nextflow.scm.RepositoryProvider - Request [credentials kushwahg:*********] -> http://**.**.**.**/repos/kushwahg/run_from_github_repo/contents/nextflow.config
Nov-15 14:51:08.205 [main] DEBUG nextflow.scm.RepositoryProvider - Request [credentials kushwahg:*********] -> http://**.**.**.**/repos/kushwahg/run_from_github_repo/contents/main.nf
Nov-15 14:51:08.815 [main] DEBUG nextflow.scm.RepositoryProvider - Request [credentials kushwahg:*********] -> http://**.**.**.**/repos/kushwahg/run_from_github_repo
Nov-15 14:51:09.443 [main] DEBUG nextflow.cli.Launcher - Operation aborted
nextflow.exception.AbortOperationException: Cannot find `kushwahg/run_from_github_repo` -- Make sure exists a GitHub repository at this address `http://**.**.**.**/kushwahg/run_from_github_repo`
        at nextflow.scm.RepositoryProvider.validateFor(RepositoryProvider.groovy:210)
        at nextflow.scm.AssetManager.checkValidRemoteRepo(AssetManager.groovy:343)
        at nextflow.scm.AssetManager.download(AssetManager.groovy:522)
        at nextflow.scm.AssetManager.download(AssetManager.groovy)
        at nextflow.cli.CmdRun.getScriptFile(CmdRun.groovy:261)
        at nextflow.cli.CmdRun.run(CmdRun.groovy:178)
        at nextflow.cli.Launcher.run(Launcher.groovy:406)
        at nextflow.cli.Launcher.main(Launcher.groovy:554)

Has anyone else encountered this issue as well? I would really appreciate any help on this.

Thanks,

Garima

[Paolo Di Tommaso]

Hi, 

Have you configured your GitHub server in the NF scm file as described here? 

Also not that you will need to specify the `-hub` command line option or the repository full path as described in the above link. 

Hope it helps.

Cheers,

Paolo

--
You received this message because you are subscribed to the Google Groups "Nextflow" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/nextflow.
For more options, visit https://groups.google.com/d/optout.

[garima kushwaha]

Hi Paolo,

Yes I tried setting up NF scm file and ran the command with '-hub' and and full path but it still gave me the same error. My scam file looked like this

providers {
   my_git {
       server = 'http://**.**.**.**'
       user = 'kushwahg'
       platform = 'github'
       password = 'de66d2dbe886ea01236ec77eaa4fcceaf051c962'
       endpoint = 'http://**.**.**.**' (same as server)
   }
}

Here, I tried using both actual password and my token as password. 

The command I used to run was:

nextflow run http://**.**.**.**/kushwahg/run_from_github_repo -hub my_git

It is still giving the same error and log file.

Garima

To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+u...@googlegroups.com.

[Paolo Di Tommaso]

Hi, 

I think the endpoint you are using is not valid.  Can you query it by using `curl` ? 

For example with the public one I can do the following: 

curl https://api.github.com/repos/nextflow-io/nextflow/contents/

and it prints the GH response. 

Does it work the same (replacing your user name and project obviously) for your GH installation? 

Cheers,

Paolo

To unsubscribe from this group and stop receiving emails from it, send an email to nextflow+unsubscribe@googlegroups.com.

[garima kushwaha]

Hi Paolo,

Yes, endpoint was the culprit! I figured that for enterprise github you have to add "/api/v3" after the url.

I used it this and it worked:

  endpoint = 'http://********.com/api/v3'

I got my hints from this link:

   https://developer.github.com/enterprise/2.6/v3/

Thanks for help!

Garima

--
You received this message because you are subscribed to a topic in the Google Groups "Nextflow" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nextflow/_1uB5yETvBQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nextflow+unsubscribe@googlegroups.com.

[Paolo Di Tommaso]

Great!

p