Trying to contribute a package to Spack

99 views
Skip to first unread message

Rob Groner

unread,
Feb 2, 2021, 12:02:10 PM2/2/21
to Spack
I've been following the instructions under Contribution, but either I missed something or the instructions have changed.  I'm trying to push my branch upstream, but I get an error: 

ERROR: Permission to spack/spack.git denied to robgics.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

I'm guessing there's a step somewhere where I request permission to push the branch?

Rob

Erik Schnetter

unread,
Feb 2, 2021, 1:59:40 PM2/2/21
to Rob Groner, Spack
Rob

Typically, you would begin by cloning the Spack repo. You will then
have write permissions to your own repo.

If you did this, then the problem might be that you checked out Spack
via "https" instead of via "ssh+git". If you, you can also fix this
manually by editing your ".git/config". I assume you don't have a
"pushurl" line such as the one below; if you add it, you can push.
(You will need to adapt this to your own repo.)

[remote "origin"]
url = https://githug.com/spack/spack.git
pushurl = ssh://g...@github.com/robgics/spack.git

-erik
> --
> You received this message because you are subscribed to the Google Groups "Spack" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to spack+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/spack/a6573d3f-5299-4483-89bc-44eedd3e2cafn%40googlegroups.com.



--
Erik Schnetter <schn...@gmail.com>
http://www.perimeterinstitute.ca/personal/eschnetter/

Gamblin, Todd

unread,
Feb 2, 2021, 2:08:01 PM2/2/21
to Erik Schnetter, Rob Groner, Spack
I think Erik means “forking” the spack repo here — i.e. see:
https://docs.github.com/en/github/getting-started-with-github/fork-a-repo

If you create a fork, you can push your branch there as Erik says, and you create a PR to the main spack repository from the branch in your fork.

If you’ve already cloned Spack, you can add the fork as a remote like this:

git remote rename origin upstream # rename your existing origin to “upstream"
git remote add origin g...@github.com:yourusername/spack

-Todd




> On Feb 2, 2021, at 10:59 AM, Erik Schnetter <schn...@gmail.com> wrote:
>
> Rob
>
> Typically, you would begin by cloning the Spack repo. You will then
> have write permissions to your own repo.
>
> If you did this, then the problem might be that you checked out Spack
> via "https" instead of via "ssh+git". If you, you can also fix this
> manually by editing your ".git/config". I assume you don't have a
> "pushurl" line such as the one below; if you add it, you can push.
> (You will need to adapt this to your own repo.)
>
> [remote "origin"]
> url = https://secure-web.cisco.com/1y9_P70HEGcZqWvxu-PpjfonwyjuO9A_PQ_RR-ahMOnDRvD2KpZyWeoo3vDfygb_rUbycS8XGCzV_fRSGXmZWtvIRrR1_KqvvrmrV-LCXH9LyWigz-2TnRx6FtcunUE7giEBiZkKDKxaEM3knwrYOkpnJxLyO6SnSABiDFnuy6y0gZ-5xAX02uxKvxavbx54iNuX3v9oWRfp8Dc6HLtF3r37qcUlUsITCKjm0j86hDraJKwV77f7AWYgIx_4GG3DSuosgRxyfqvt_7-o3VWtpkaM5Zjs1uREp8YVh42Juh-XDs8O-7a9AfoZ5tjL8OQD4TSRHamQKq1bQpdx15kMnUtfj2Koku5tCuotm7GkOiSq-SbApUrwJobE74ZWbvr9xzBKymQLYwzXkYwklxjP1hQ/https%3A%2F%2Fgithug.com%2Fspack%2Fspack.git
> pushurl = ssh://g...@github.com/robgics/spack.git
>
> -erik
>
> On Tue, Feb 2, 2021 at 12:02 PM Rob Groner <rug...@psu.edu> wrote:
>>
>> I've been following the instructions under Contribution, but either I missed something or the instructions have changed. I'm trying to push my branch upstream, but I get an error:
>>
>> ERROR: Permission to spack/spack.git denied to robgics.
>> fatal: Could not read from remote repository.
>>
>> Please make sure you have the correct access rights
>> and the repository exists.
>>
>> I'm guessing there's a step somewhere where I request permission to push the branch?
>>
>> Rob
>>
>> --
>> You received this message because you are subscribed to the Google Groups "Spack" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to spack+un...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/msgid/spack/a6573d3f-5299-4483-89bc-44eedd3e2cafn%40googlegroups.com.
>
>
>
> --
> Erik Schnetter <schn...@gmail.com>
> http://www.perimeterinstitute.ca/personal/eschnetter/
>
> --
> You received this message because you are subscribed to the Google Groups "Spack" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to spack+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/spack/CADKQjje1N7Q3weHkqb%3D6w5gz1RANHiZUpw%3DkjP0-2mFNSVFvkw%40mail.gmail.com.
>

Rob Groner

unread,
Feb 2, 2021, 2:20:16 PM2/2/21
to Spack
I followed the github forking directions, and then cloned that corresponding repo and made my changes.  So yes, now I have full access, since it's all "mine".
Github appears to have a "pull request" button with my forked repo, so I'll try that after I run the tests.

Thanks for the guidance.  I looked over the Contribution docs and I couldn't find where it said to fork instead of clone.

Rob

Rob Groner

unread,
Feb 2, 2021, 2:35:04 PM2/2/21
to Spack
Ok, I do see where the Contribution docs mention pushing to your remote fork.  So, the "fork" is referenced a couple places, and "cloning" never is.  But since I've never forked a repo before, I didn't realize it was its own separate thing.  The docs mention it like something you should have already done.

Rob

Erik Schnetter

unread,
Feb 2, 2021, 2:51:30 PM2/2/21
to Gamblin, Todd, Rob Groner, Spack
D'oh; yes, I meant "forking". Apologies.

-erik
> To view this discussion on the web visit https://groups.google.com/d/msgid/spack/90E1279B-0B3F-4D73-B390-AEE09057CEA0%40llnl.gov.

Rob Groner

unread,
Feb 2, 2021, 5:39:08 PM2/2/21
to Spack
I'm running into a problem trying to do the docs test.  One of the dependencies for all of the spack items to be installed is failing. 

success copy gatsby files - 0.036s
 ERROR #11321  PLUGIN
"gatsby-plugin-prefetch-google-fonts" threw an error while running the onPreBootstrap lifecycle:
ENOENT: no such file or directory, stat '.cache/google-fonts/__GATSBY_IPFS_PATH_PREFIX__/fonts'

  Error: ENOENT: no such file or directory, stat '.cache/google-fonts/__GATSBY_I  PFS_PATH_PREFIX__/fonts'

not finished onPreBootstrap - 0.362s
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! npm-cl...@0.1.0 build:static: `GATSBY_IS_STATIC=true gatsby build --prefix-paths "echo"`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the npm-cl...@0.1.0 build:static script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

I'll try installing it on a different machine tomorrow.

Rob Groner

unread,
Feb 3, 2021, 5:26:33 PM2/3/21
to Spack
Installed on another machine, and it has the same problem...npm won't install.

Rob Groner

unread,
May 12, 2021, 2:23:58 PM5/12/21
to Spack
I could start a new conversation, but this one still seems to apply.  I'm still having issues trying to contribute anything to Spack.

I decided to start over.  So I checked out the latest spack version from github and went back to the Contribution Guide.  I figured that maybe I should try all the spack  tests on the baseline BEFORE I make any changes at all, so I can remove that as a possible cause of error.

I executed "spack unit-test" and that succeeded.  Well, it did say "Fail" in a couple places, but they seem to have been expected failures...so, successes.  Task failed successfully.  I then tried "spack style" since that was the next test the Contribution guide said to try.  That didn't get far at all:

==> style: running code checks on spack.
==> style: tools selected: flake8, mypy
==> Modified files:
==> style: running flake8 checks on spack.
==> Flake8 style checks were clean
==> Error: style: mypy is not available in path, skipping
spack style found errors.

Hmmm....well, I know the docs said it used mypy, but I don't recall it saying I had to install it myself.

I decide that maybe the development branch isn't the best place to be trying this.  Let me checkout one of the release branches, because they're sure to pass all tests (and maybe development wouldn't because of unresolved issues....though ya, that's not supposed to happen).

I checkout releases/v0.16.  I execute "spack unit-test" and it fails due to failures in test_all_dependencies_exist, among other things.

TL;DR

I'm having a lot of trouble trying to contribute packages back to Spack.  Following the Contribution guide doesn't seem to be working for me.  Am I doing it wrong?

Thanks.

Rob

Gamblin, Todd

unread,
May 12, 2021, 3:02:01 PM5/12/21
to Rob Groner, Spack
Rob:

We should work on the message there — mypy is relatively new — I don’t expect you’ll need to do much to satisfy it.

If `spack unit-test` passes and flake8 checks are clean, you should be able to contribute a PR on develop, and people will work with you there.

-Todd


Rob Groner

unread,
May 12, 2021, 3:32:38 PM5/12/21
to Spack
Ok, I can work with that.

So I JUST checked out the latest spack, changed to the develop branch, and executed "spack unit-test".

_________________________________________ TestStage.test_get_stage_root_in_spack __________________________________________

self = <spack.test.stage.TestStage object at 0x7f781c15a190>, clear_stage_root = None

    def test_get_stage_root_in_spack(self, clear_stage_root):
        """Ensure an instance path is an accessible build stage path."""
        base = canonicalize_path(os.path.join('$spack', '.spack-test-stage'))
        mkdirp(base)
        test_path = tempfile.mkdtemp(dir=base)
    
        try:
            with spack.config.override('config:build_stage',  test_path):
                path = spack.stage.get_stage_root()
    
>               assert 'spack' in path.split(os.path.sep)
E               AssertionError: assert 'spack' in ['', 'home', 'rug262', 'development', 'spack-pullreq', '.spack-test-stage', ...]
E                +  where ['', 'home', 'rug262', 'development', 'spack-pullreq', '.spack-test-stage', ...] = <built-in method split of str object at 0x7f7784569030>('/')
E                +    where <built-in method split of str object at 0x7f7784569030> = '/home/rug262/development/spack-pullreq/.spack-test-stage/tmpj41gewiw'.split
E                +    and   '/' = <module 'posixpath' from '/usr/lib64/python3.7/posixpath.py'>.sep
E                +      where <module 'posixpath' from '/usr/lib64/python3.7/posixpath.py'> = os.path

lib/spack/spack/test/stage.py:839: AssertionError

Hrmmm.  Using python 3.7.9. on Fedora 31.


Gamblin, Todd

unread,
May 13, 2021, 11:34:52 AM5/13/21
to Rob Groner, Spack
Do you have the stage configured specially?

Groner, Rob

unread,
May 13, 2021, 12:06:29 PM5/13/21
to Gamblin, Todd, Spack

Not that I'm aware of.  I just checked out the repo, sourced the setup-env.sh, and ran spack unit-test.

I went back and delete ~/.spack since I've run spack before on this computer, but I get the same result.

spack config blame config shows everything is coming form the spack default config.yaml.

Rob


From: Gamblin, Todd <gamb...@llnl.gov>
Sent: Thursday, May 13, 2021 11:34 AM
To: Groner, Rob <rug...@psu.edu>
Cc: Spack <sp...@googlegroups.com>
Subject: Re: [spack] Trying to contribute a package to Spack
 

Rob Groner

unread,
May 27, 2021, 2:28:45 PM5/27/21
to Spack
Has anyone successfully run "spack unit-test" on Centos7?  I've tried on a vangrant VM I created from scratch and from a RHEL7 VM I had handy, and they both hang at the same spot:

[vagrant@localhost spack]$ spack unit-test
===================================================================== test session starts =====================================================================
platform linux -- Python 3.6.12, pytest-3.2.5, py-1.4.34, pluggy-0.4.0
rootdir: /vagrant/spack, inifile: pytest.ini
collected 2710 items                                                                                                                                           

lib/spack/spack/test/abi.py .............................
lib/spack/spack/test/architecture.py ................................................................
lib/spack/spack/test/bindist.py .....


I haven't dug into that file yet to see what it may be doing.

This is with a clean vagrant VM based off of Centos7 and a fresh checkout of the Spack git repo.

Rob


Gamblin, Todd

unread,
May 27, 2021, 2:35:43 PM5/27/21
to Rob Groner, Spack
Rob:

Getting this passing locally isn’t actually a hard requirement for contributing — we run these tests in CI and that’s really what people will look at for approval.  If you like pushing to your PR more than running a local vagrant instance that might be easier.

-Todd



Rob Groner

unread,
May 27, 2021, 3:07:07 PM5/27/21
to Spack
Sooo...skip the unit-test and style test?  Just fork the repo, add my package, commit to my repo, and then generate a pull-request?

Rob

Gamblin, Todd

unread,
May 27, 2021, 3:10:38 PM5/27/21
to Rob Groner, Spack
Yep!  If it helps to run the tests locally you can but it shouldn’t be an obstacle — that’s why we run them in CI.

Rob Groner

unread,
May 27, 2021, 3:15:59 PM5/27/21
to Spack
Well, it's not so much that it helps to run the tests locally...I'm just puzzled as to why I CAN'T.

I'll try your suggestion.  Obviously not a lot of tests to run as it's packages, not code changes.

Rob

Groner, Rob

unread,
May 28, 2021, 5:12:59 PM5/28/21
to Spack
Ok, so pull request submitted (yay!) and the unit tests seem to have run and passed.  Everything else seems to be waiting for a review.  Is that something I initiate?  In other words, is the next step waiting on me?

Rob




From: sp...@googlegroups.com <sp...@googlegroups.com> on behalf of Rob Groner <rug...@psu.edu>
Sent: Thursday, May 27, 2021 3:15 PM
To: Spack <sp...@googlegroups.com>

Thomas M. Payerle

unread,
May 28, 2021, 5:26:01 PM5/28/21
to Groner, Rob, Spack
One of the spack maintainers, or a package maintainer if updating an existing package, will review your submission.  They might have suggestions/recommended changes/etc before approving, or might just approve without suggestions.  Yuo should get email from github when updates made.

To view this discussion on the web visit https://groups.google.com/d/msgid/spack/BL0PR02MB44998F21A8405CE076A8F52480229%40BL0PR02MB4499.namprd02.prod.outlook.com.


--
Tom Payerle
DIT-ACIGS/Mid-Atlantic Crossroads        pay...@umd.edu
5825 University Research Park               (301) 405-6135
University of Maryland
College Park, MD 20740-3831

Gamblin, Todd

unread,
May 28, 2021, 5:31:27 PM5/28/21
to Groner, Rob, Spack
Is it this one?  https://github.com/spack/spack/pull/23974

I can’t seem to find anything terribly identifiable in the queue from you.


Groner, Rob

unread,
May 28, 2021, 6:06:12 PM5/28/21
to Gamblin, Todd, Spack
Yep, that's it.


Rob


From: Gamblin, Todd <gamb...@llnl.gov>
Sent: Friday, May 28, 2021 5:31 PM

Groner, Rob

unread,
Jun 3, 2021, 11:41:18 AM6/3/21
to Gamblin, Todd, Spack
I've got in mind a few more things to try contributing.  The contribution guide seems to indicate the best way to do that is to fork the spack repo once, and then just use a separate branch for each thing being added.  Have I got the right idea? (I've never done this before)

Thanks,

Rob


From: sp...@googlegroups.com <sp...@googlegroups.com> on behalf of Groner, Rob <rug...@psu.edu>
Sent: Friday, May 28, 2021 6:06 PM
To: Gamblin, Todd <gamb...@llnl.gov>

Gamblin, Todd

unread,
Jun 3, 2021, 12:02:03 PM6/3/21
to Groner, Rob, Spack
Yep definitely.

You want each PR to be on a new branch, based on develop, and you want the PR to be submitted back to upstream/develop. Once the PR is merged, you pull the changes from upstream (your commits and others') back onto your local develop branch, and you can just delete the branch you made for the PR.

This is a pretty common model for git repos, but it takes some getting used to if you haven’t done it before.

-Todd

Groner, Rob

unread,
Jun 3, 2021, 12:03:30 PM6/3/21
to Gamblin, Todd, Spack
Great, thanks!

Rob



From: 'Gamblin, Todd' via Spack <sp...@googlegroups.com>
Sent: Thursday, June 3, 2021 12:01 PM

Groner, Rob

unread,
Jun 25, 2021, 3:30:39 PM6/25/21
to Gamblin, Todd, Spack
So, just checking that I'm doing this right....

Submitting packages requires a lot of waiting for something to happen, right?  I keep checking the PRs (I've a total of 3, though only 2 are new packages).  I understand that at some point, an actual human is going to have to look at them.  I guess that's the bottleneck, right?  I can't tell if the automated tests are the bottleneck or not, as they always seem to be sitting at "19 expected, and 1 successful check".

I also keep getting these emails telling me that the macOS build nightly workflow failed on gcc with clang.  I have no idea what this in relation to, as I can't find any details about it and honestly can't even guess why my PRs would be causing a macOS failure.

I'm not impatient (ok, probably a little), I just haven't been through this process before so I have no concept of what to expect, nor when it may be going different than it should.

Thanks,

Rob




Sent: Thursday, June 3, 2021 12:03 PM
You received this message because you are subscribed to a topic in the Google Groups "Spack" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/spack/wk4gBOrCbok/unsubscribe.
To unsubscribe from this group and all its topics, send an email to spack+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/spack/BL0PR02MB44994BEB038319D20F9BE27E803C9%40BL0PR02MB4499.namprd02.prod.outlook.com.

Groner, Rob

unread,
Jul 22, 2021, 9:53:14 AM7/22/21
to Gamblin, Todd, Spack
Ok, something approaching success!  The package I submitted has been merged.  Yay, I'm big leagues now!

The other PR was simply to add some additional output to a Spack error message that was confusing me for quite some time.  That PR seems to be hung up on a failing "codecov" test.  Unfortunately, when I try to drill down to see what it could possibly be by "View this Pull Request on Codecov"...that gives me an error page.

Other than refreshing my PR upstream, I'm not sure how to get past this.

Thanks,

Rob


------------------------------------
Rob Groner
HPC Implementation Engineer

Institute for Computational and Data Sciences
The Pennsylvania State University
225F Computer Building
University Park, PA 16802


Sent: Friday, June 25, 2021 3:30 PM

Groner, Rob

unread,
Aug 2, 2021, 11:59:28 AM8/2/21
to Gamblin, Todd, Spack
So if I have a PR that has passed all tests, but (understandably) needs someone to review and approve it...is there anything I need to do to make that happen?  Someone I need to contact?  Or will the PR just get noticed on its own when someone has the time?

Thanks,

Rob



Sent: Thursday, July 22, 2021 9:53 AM

Groner, Rob

unread,
Aug 16, 2021, 5:12:27 PM8/16/21
to Gamblin, Todd, Spack
Not trying to sound like a broken record....but I still don't know if I'm supposed to do anything to make the review happen or if it's just wait and cross my fingers.  I mean, I'm REALLY HAPPY that the package I submitted was PR'd into develop a month ago....but the new PR I created to fix​ that package and make it actually work is sitting there waiting.  I'd hate to have the next official release contain my broken package....a little embarrassing.  

Rob





Sent: Monday, August 2, 2021 11:59 AM

Groner, Rob

unread,
Aug 16, 2021, 5:29:43 PM8/16/21
to Gamblin, Todd, Spack
Thanks Todd.



Sent: Monday, August 16, 2021 5:12 PM
Reply all
Reply to author
Forward
0 new messages