Unable to build SIMP 6 ISO

[smithcl...@gmail.com]

Hello,

 

I have been attempting to build a simp 6 ISO using the simp 6 instructions however I have been having a few issues with getting the build to complete. I attempted to use both the docker and the iso_vbox methods described in the documentation. I have the following CentOS iso located in spec/fixtures/ISO

c455ee948e872ad2194bdddd39045b83634e8613249182b88f549bb2319d97eb  CentOS-7-x86_64-DVD-1611.iso

I used the following command to attempt to build the iso: 

simp-core]$ SIMP_BEAKER_build_version=6.0.0-0 SIMP_BEAKER_build_map=6.X rake beaker:suites[default]

After running the command I received the following (trimmed) output:
puppet 20:14:43$               wget https://github.com/simp/puppet-haveged/archive/master.tar.gz -O /tmp/module-tars/puppet-haveged.tar.gz
              puppet module install /tmp/module-tars/puppet-haveged.tar.gz --ignore-dependencies --target-dir=/tmp/.2V11HJ
  --2017-08-11 00:14:43--  https://github.com/simp/puppet-haveged/archive/master.tar.gz
  Resolving github.com (github.com)...   192.30.253.113, 192.30.253.112
  Connecting to github.com (github.com)|192.30.253.113|:443...   connected.
  HTTP request sent, awaiting response...   302 Found
  Location: https://codeload.github.com/simp/puppet-haveged/tar.gz/master [following]
  --2017-08-11 00:14:44--  https://codeload.github.com/simp/puppet-haveged/tar.gz/master
  Resolving codeload.github.com (codeload.github.com)...   192.30.253.120, 192.30.253.121
  Connecting to codeload.github.com (codeload.github.com)|192.30.253.120|:443...   connected.
  HTTP request sent, awaiting response...   200 OK
  Length: unspecified [application/x-gzip]
  Saving to: ‘/tmp/module-tars/puppet-haveged.tar.gz’
       0K ..  .....                                                6.83M=0.001s
  2017-08-11 00:14:44 (6.83 MB/s) - ‘/tmp/module-tars/puppet-haveged.tar.gz’ saved [8143]
  Notice: Preparing to install into /tmp/.2V11HJ ...
  Notice: Installing -- do not interrupt ...
  /tmp/.2V11HJ
  └── stm-haveged (v0.2.0)
puppet executed in 2.96 seconds
    should install base modules for pupmod
puppet 20:14:46$ mktemp -t apply_manifest.pp.XXXXXX
  /tmp/apply_manifest.pp.jW4bSv
puppet executed in 0.05 seconds
localhost $ scp /tmp/beaker20170810-2807-bj89aq puppet:/tmp/apply_manifest.pp.jW4bSv {:ignore => }
puppet 20:14:46$ puppet apply --verbose --modulepath=/tmp/.2V11HJ /tmp/apply_manifest.pp.jW4bSv
  Info: Loading facts
  Info: Loading facts
  Info: Loading facts
  Info: Loading facts
  Warning: ModuleLoader: module 'pupmod' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
     (file & line not available)
  Warning: This method is deprecated, please use the stdlib validate_legacy function, with Pattern[]. There is further documentation for validate_legacy function in the README. at ["/tmp/.2V11HJ/pupmod/manifests/init.pp", 169]:["/tmp/.2V11HJ/pupmod/manifests/master.pp", 143]
     (at /tmp/.2V11HJ/stdlib/lib/puppet/functions/deprecation.rb:25:in `deprecation')
  Warning: ModuleLoader: module 'pam' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
     (file & line not available)
  Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Unknown resource type: 'puppet_authorization::rule' at /tmp/.2V11HJ/pupmod/manifests/master/simp_auth.pp:66:3  at /tmp/.2V11HJ/pupmod/manifests/init.pp:241 on node puppet.localdomain
puppet executed in 8.46 seconds
Exited: 1
    should set up a puppetserver (FAILED - 1)
ssh connection to puppet has been terminated
ssh connection to agent has been terminated
removing temporary ssh-config files per-vagrant box
Destroying vagrant boxes
==> agent: Forcing shutdown of VM...
==> agent: Destroying VM and associated drives...
==> puppet: Forcing shutdown of VM...
==> puppet: Destroying VM and associated drives...
Failures:
  1) install environment via r10k and puppetserver master should set up a puppetserver
     Failure/Error: apply_manifest_on(master, master_manifest, :modulepath => moduledir, :accept_all_exit_codes => true)
     Beaker::Host::CommandFailure:
       Host 'puppet' exited with 1 running:
        puppet apply --verbose --modulepath=/tmp/.2V11HJ /tmp/apply_manifest.pp.jW4bSv
       Last 10 lines of output were:
        Info: Loading facts
        Info: Loading facts
        Info: Loading facts
        Warning: ModuleLoader: module 'pupmod' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
           (file & line not available)
        Warning: This method is deprecated, please use the stdlib validate_legacy function, with Pattern[]. There is further documentation for validate_legacy function in the README. at ["/tmp/.2V11HJ/pupmod/manifests/init.pp", 169]:["/tmp/.2V11HJ/pupmod/manifests/master.pp", 143]
           (at /tmp/.2V11HJ/stdlib/lib/puppet/functions/deprecation.rb:25:in `deprecation')
        Warning: ModuleLoader: module 'pam' has unresolved dependencies - it will only see those that are resolved. Use 'puppet module list --tree' to see information about modules
           (file & line not available)
        Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Resource Statement, Unknown resource type: 'puppet_authorization::rule' at /tmp/.2V11HJ/pupmod/manifests/master/simp_auth.pp:66:3  at /tmp/.2V11HJ/pupmod/manifests/init.pp:241 on node puppet.localdomain
     # /home/user/.rvm/gems/ruby-2.1.9/bundler/gems/beaker-4e10bde3e1bd/lib/beaker/host.rb:351:in `exec'
     # /home/user/.rvm/gems/ruby-2.1.9/bundler/gems/beaker-4e10bde3e1bd/lib/beaker/dsl/helpers/host_helpers.rb:83:in `block in on'
     # /home/user/.rvm/gems/ruby-2.1.9/bundler/gems/beaker-4e10bde3e1bd/lib/beaker/shared/host_manager.rb:127:in `run_block_on'
     # /home/user/.rvm/gems/ruby-2.1.9/bundler/gems/beaker-4e10bde3e1bd/lib/beaker/dsl/patterns.rb:37:in `block_on'
     # /home/user/.rvm/gems/ruby-2.1.9/bundler/gems/beaker-4e10bde3e1bd/lib/beaker/dsl/helpers/host_helpers.rb:63:in `on'
     # /home/user/.rvm/gems/ruby-2.1.9/bundler/gems/beaker-4e10bde3e1bd/lib/beaker/dsl/helpers/puppet_helpers.rb:480:in `block in apply_manifest_on'
     # /home/user/.rvm/gems/ruby-2.1.9/bundler/gems/beaker-4e10bde3e1bd/lib/beaker/shared/host_manager.rb:127:in `run_block_on'
     # /home/user/.rvm/gems/ruby-2.1.9/bundler/gems/beaker-4e10bde3e1bd/lib/beaker/dsl/patterns.rb:37:in `block_on'
     # /home/user/.rvm/gems/ruby-2.1.9/bundler/gems/beaker-4e10bde3e1bd/lib/beaker/dsl/helpers/puppet_helpers.rb:409:in `apply_manifest_on'
     # ./spec/acceptance/suites/default/01_simp_server_spec.rb:75:in `block (4 levels) in <top (required)>'
Finished in 2 minutes 38.1 seconds (files took 5 minutes 31 seconds to load)
8 examples, 1 failure
Failed examples:
rspec ./spec/acceptance/suites/default/01_simp_server_spec.rb:74 # install environment via r10k and puppetserver master should set up a puppetserver

Since that failed I decided to move over the to vbox builder since the documentation mentioned that method was easier to debug. I first ran into the following issue with rvm however I was able to fix that based on the comments in the issue: https://github.com/rvm/rvm/issues/4068. Next I ran into the following error

 

simp-core]$ SIMP_BEAKER_build_version=6.0.0-0 SIMP_BEAKER_build_map=6.X rake beaker:suites[iso_vbox]

 

  1) iso build build SIMP ISO should build and retrieve the releases
     Failure/Error: on(host, %(#{run_cmd} "rvm use default; cd #{branch}; rake build:auto[#{release_version},/ISO]"))
     Beaker::Host::CommandFailure:
       Host 'el7' exited with 1 running:
        runuser build_user -l -c  "rvm use default; cd 6.0.0-0; rake build:auto[6.X,/ISO]"
       Last 10 lines of output were:
        /home/build_user/.rvm/gems/ruby-2.1.10/gems/simp-rake-helpers-3.4.0/lib/simp/rake/build/auto.rb:106:in `block (2 levels) in define'
        /home/build_user/.rvm/gems/ruby-2.1.10/gems/rake-12.0.0/exe/rake:27:in `<top (required)>'
        /home/build_user/.rvm/gems/ruby-2.1.10/bin/ruby_executable_hooks:15:in `eval'
        /home/build_user/.rvm/gems/ruby-2.1.10/bin/ruby_executable_hooks:15:in `<main>'
        Tasks: TOP => build:auto
        (See full trace by running task with --trace)
        ================================================================================
        ## validating ISOs for target:
              '/ISO' in '/home/build_user/6.0.0-0/6.X'
        ================================================================================

     # /home/clayton/.rvm/gems/ruby-2.1.9/bundler/gems/beaker-4e10bde3e1bd/lib/beaker/host.rb:351:in `exec'

     # /home/clayton/.rvm/gems/ruby-2.1.9/bundler/gems/beaker-4e10bde3e1bd/lib/beaker/dsl/helpers/host_helpers.rb:83:in `block in on'

     # /home/clayton/.rvm/gems/ruby-2.1.9/bundler/gems/beaker-4e10bde3e1bd/lib/beaker/shared/host_manager.rb:127:in `run_block_on'

     # /home/clayton/.rvm/gems/ruby-2.1.9/bundler/gems/beaker-4e10bde3e1bd/lib/beaker/dsl/patterns.rb:37:in `block_on'

     # /home/clayton/.rvm/gems/ruby-2.1.9/bundler/gems/beaker-4e10bde3e1bd/lib/beaker/dsl/helpers/host_helpers.rb:63:in `on' 

     # ./spec/acceptance/suites/iso_vbox/00_iso_build_spec.rb:176:in `block (5 levels) in <top (required)>'
     # ./spec/acceptance/suites/iso_vbox/00_iso_build_spec.rb:173:in `each_pair'
     # ./spec/acceptance/suites/iso_vbox/00_iso_build_spec.rb:173:in `block (4 levels) in <top (required)>'
Finished in 17 minutes 26 seconds (files took 2 minutes 35.6 seconds to load)
8 examples, 1 failure
Failed examples:
rspec ./spec/acceptance/suites/iso_vbox/00_iso_build_spec.rb:169 # iso build build SIMP ISO should build and retrieve the releases

I then tried moving over to building in a simple Vagrant environment locally on my system without rake and beaker. I have attached the Vagrantfile I used to build. The Vagrantfile was built following the simp6 environment preparation guide. I have the same CentOS ISO I mentioned above in the Input_ISO folder. Below is my input and outputs:

simp-core $ sg mock -c 'bundle exec rake build:auto[/vagrant/Input_ISO,6.X]'

==> default: 
==> default: pkg:simp_cli: |
==> default: ================================================================|
==> default: 
==> default: pkg:aux: |
==> default: =============
==> default: ==============
==> default: ==============
==> default: ================================================================================
==> default: Failed ISOs:
==> default:   * CentOS 7 x86_64
==> default:     * Error: No SRPMs generated for /home/vagrant/simp-core/src/rsync
==> default:     * Context: /home/vagrant/.rvm/gems/ruby-2.1.9/gems/simp-rake-helpers-3.3.0/lib/simp/rake/build/pkg.rb:818:in `block (2 levels) in build'

I then tried to just build rsync since that was the rsync directly from its folder  and I received the following: 

simp-core/src/rsync $ SIMP_BUILD_freshclam=no SIMP_BUILD_version=6.0.0-0.el7.centos rake pkg:rpm[/home/vagrant/simp-core/build/distributions/RedHat/7/x86_64/mock.cfg,true,false]

================================================================================
#### Skipping freshclam update (because SIMP_BUILD_freshclam = no)
================================================================================
ERROR: Exception(/home/vagrant/simp-core/src/rsync/dist/simp-rsync-6.0.1-0.el7.centos.src.rpm) Config(simp-rhel-7-x86_64) 0 minutes 1 seconds
ERROR: Command failed: 
 # /usr/bin/yum-builddep --installroot /var/lib/mock/simp-rhel-7-x86_64-simp-rsync__vagrant/root/ --releasever 7 /var/lib/mock/simp-rhel-7-x86_64-simp-rsync__vagrant/root//builddir/build/SRPMS/simp-rsync-6.0.1-0.el7.centos.src.rpm --setopt=tsflags=nocontexts
Getting requirements for simp-rsync-6.0.1-0.el7.centos.src
Error: No Package found for policycoreutils = 2.2.5
Error: No Package found for policycoreutils-devel = 2.2.5
Error: No Package found for policycoreutils-python = 2.2.5
Error: No Package found for selinux-policy = 3.12.1
Error: No Package found for selinux-policy-devel = 3.12.1
rake aborted!
Command failed with status (30): [/usr/bin/mock --quiet --uniqueext=simp-rsy...]
/home/vagrant/.rvm/gems/ruby-2.1.9/gems/simp-rake-helpers-3.3.0/lib/simp/rake/pkg.rb:404:in `block (3 levels) in define_pkg_rpm'
/home/vagrant/.rvm/gems/ruby-2.1.9/gems/simp-rake-helpers-3.3.0/lib/simp/rake/pkg.rb:393:in `each'
/home/vagrant/.rvm/gems/ruby-2.1.9/gems/simp-rake-helpers-3.3.0/lib/simp/rake/pkg.rb:393:in `block (2 levels) in define_pkg_rpm'
/home/vagrant/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/exe/rake:27:in `<top (required)>'
/home/vagrant/.rvm/gems/ruby-2.1.9/bin/ruby_executable_hooks:15:in `eval'
/home/vagrant/.rvm/gems/ruby-2.1.9/bin/ruby_executable_hooks:15:in `<main>'
Tasks: TOP => pkg:rpm
(See full trace by running task with --trace)

What is the best way to build a working RHEL and CentOS VM for SIMP 6.0.0-0? 

Thank you,

Clayton

[Trevor Vaughan]

Hi Clayton,

I'm sorry that you're having so many issues with this.

My guess is that they moved the ClamAV packages into the EPEL archives and our original URL locations will no longer work.

I don't have time to troubleshoot this today, but will make it a priority as soon as possible since repeatable builds are very important to us.

In the mean time, would you be able to use a prebuilt ISO?

Thanks,

Trevor

--
You received this message because you are subscribed to the Google Groups "SIMP Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to simp-users+unsubscribe@googlegroups.com.
To post to this group, send email to simp-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/simp-users/b9b7d286-617b-4905-bc44-39d699d7fd5e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Judith Johnson]

Clayton,

I am assuming you were the Clayton in the chatroom, and I am not sure if you saw that Trevor posted his gist -
"(Trevor Vaughan·Aug-8 10:17) I'll try to post a horrible build gist that I'm using in AWS later tonight"
 https://gist.github.com/trevor-vaughan/128f90351a51defd5cbd7b435225438e

Or, as Trevor hinted, If you need a pre-built ISO, it would be here https://simp-project.com/ISO/SIMP/

but i think you were building for RedHat?

Let us know if we can help, or we can wait for Trevor's response.

To view this discussion on the web visit https://groups.google.com/d/msgid/simp-users/CANs%2BFoWGoQx8nraVWUPR0m3KEsGDiLUVj%2B32LHdZYggw246c3g%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

	judy johnson Software Engineer | Onyx Point, Inc. 7050 Hi Tech Drive, Suite 102 Hanover, MD. 21076 e: judith....@onyxpoint.com w: 443-737-9891 c: 410-262-6959

[smithcl...@gmail.com]

Hello Trevor and Judith,

Thank you both for the prompt replies! Judith you are correct that I am building for RedHat, I was only testing a build for CentOS to try to debug if the issue I was encountering was RHEL specific or was more overarching.

Trevor, I am attempting to get the script you sent working however it seems to be looking for a tarball. I am having trouble tracking down which tar bundle to use since I do not see a tar bundle for redhat 6.0.0-0. Could you clarify on which bundle I should be using?

Thank you,

Clayton

[Trevor Vaughan]

You can use the one labeled CentOS, there are no differences.

Thanks,

Trevor

--
You received this message because you are subscribed to the Google Groups "SIMP Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to simp-users+unsubscribe@googlegroups.com.
To post to this group, send email to simp-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/simp-users/6acc98db-d333-4c1b-8e8c-a550a405ac72%40googlegroups.com.

[smithcl...@gmail.com]

Thank you for the reply!

I am not sure why, however even using the script I seem to still be receiving failures due to the rsync. The command I ran along with the error it generated is below:

[clayton@localhost rsync]$ SIMP_BUILD_version=6.0.0-0.el7 rake pkg:rpm[/home/clayton/EL7/simp-core/build/distributions/RedHat/7/x86_64/mock.cfg,true,false] --trace

** Invoke pkg:rpm (first_time)
** Invoke pkg:srpm (first_time)
** Invoke pkg:tar (first_time)
** Invoke pkg:initialize_spec_info (first_time)
** Invoke /home/clayton/EL7/simp-core/src/rsync/dist (first_time, not_needed)
** Execute pkg:initialize_spec_info
Traceback (most recent call last):
  File "/usr/libexec/mock/mock", line 934, in <module>
    main()
  File "/usr/lib/python2.7/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/libexec/mock/mock", line 737, in main
    run_command(options, args, config_opts, commands, buildroot, state)
  File "/usr/lib/python2.7/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/libexec/mock/mock", line 852, in run_command
    shutil.copytree(src, dest2)
  File "/usr/lib64/python2.7/shutil.py", line 177, in copytree
    os.makedirs(dst)
  File "/usr/lib64/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 20] Not a directory: '/var/lib/mock/simp-rhel-7-x86_64-simp-rsync__clayton/root/tmp/cqcbxallwe_tmp/build'
rake aborted!
Command failed with status (1): [/usr/bin/mock --quiet --uniqueext=simp-rsy...]
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/file_utils.rb:66:in `block in create_shell_runner'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/file_utils.rb:56:in `call'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/file_utils.rb:56:in `sh'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/simp-rake-helpers-3.4.0/lib/simp/rake/pkg.rb:164:in `block in initialize_spec_info'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/simp-rake-helpers-3.4.0/lib/simp/rake/pkg.rb:162:in `each'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/simp-rake-helpers-3.4.0/lib/simp/rake/pkg.rb:162:in `initialize_spec_info'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/simp-rake-helpers-3.4.0/lib/simp/rake/pkg.rb:219:in `block (2 levels) in define_pkg_tar'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/task.rb:250:in `call'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/task.rb:250:in `block in execute'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/task.rb:250:in `each'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/task.rb:250:in `execute'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/task.rb:194:in `block in invoke_with_call_chain'
/home/clayton/.rvm/rubies/ruby-2.1.9/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/task.rb:187:in `invoke_with_call_chain'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/task.rb:216:in `block in invoke_prerequisites'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/task.rb:214:in `each'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/task.rb:214:in `invoke_prerequisites'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/task.rb:193:in `block in invoke_with_call_chain'
/home/clayton/.rvm/rubies/ruby-2.1.9/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/task.rb:187:in `invoke_with_call_chain'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/task.rb:216:in `block in invoke_prerequisites'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/task.rb:214:in `each'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/task.rb:214:in `invoke_prerequisites'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/task.rb:193:in `block in invoke_with_call_chain'
/home/clayton/.rvm/rubies/ruby-2.1.9/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/task.rb:187:in `invoke_with_call_chain'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/task.rb:216:in `block in invoke_prerequisites'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/task.rb:214:in `each'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/task.rb:214:in `invoke_prerequisites'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/task.rb:193:in `block in invoke_with_call_chain'
/home/clayton/.rvm/rubies/ruby-2.1.9/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/task.rb:187:in `invoke_with_call_chain'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/task.rb:180:in `invoke'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/application.rb:152:in `invoke_task'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/application.rb:108:in `block (2 levels) in top_level'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/application.rb:108:in `each'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/application.rb:108:in `block in top_level'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/application.rb:117:in `run_with_threads'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/application.rb:102:in `top_level'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/application.rb:80:in `block in run'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/application.rb:178:in `standard_exception_handling'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/lib/rake/application.rb:77:in `run'
/home/clayton/.rvm/gems/ruby-2.1.9/gems/rake-12.0.0/exe/rake:27:in `<top (required)>'
/home/clayton/.rvm/gems/ruby-2.1.9/bin/rake:23:in `load'
/home/clayton/.rvm/gems/ruby-2.1.9/bin/rake:23:in `<main>'
/home/clayton/.rvm/gems/ruby-2.1.9/bin/ruby_executable_hooks:15:in `eval'
/home/clayton/.rvm/gems/ruby-2.1.9/bin/ruby_executable_hooks:15:in `<main>'
Tasks: TOP => pkg:rpm => pkg:srpm => pkg:tar => pkg:initialize_spec_info

[clayton@localhost rsync]$ sudo file /var/lib/mock/simp-rhel-7-x86_64-simp-rsync__clayton/root/tmp/lsbokrzvdg_tmp

/var/lib/mock/simp-rhel-7-x86_64-simp-rsync__clayton/root/tmp/lsbokrzvdg_tmp: ASCII text

I am not sure how but the directory it is trying to create is being seen as a file. I have not messed around with the mock directory manually other than to clear it after it failed to see if it would have the same effect.

Thanks,

Clayton

On Friday, August 11, 2017 at 9:22:47 AM UTC-4, smithcl...@gmail.com wrote:

[Trevor Vaughan]

Hi Clayton,

So, I just built that module twice without issue and I'm thinking that something might be wrong with your mock workspaces.

Can you try:

1) In the 'simp-core' directory, run 'bundle update'

2) Run 'mock -r epel-7-x86_64 --clean all'

3) As root, remove everything under /var/lib/mock/*

4) In the 'simp-core' directory, run SIMP_BUILD_version=default rake pkg:single[src/puppet/modules/rsync,epel-7-x86_64]

If all of that works, then the SIMP_BUILD_version=6.0.0-0.el7 rake pkg:rpm[/home/clayton/EL7/simp-core/build/distributions/RedHat/7/x86_64/mock.cfg,true,false] command should also work at that time.

Thanks,

Trevor

--
You received this message because you are subscribed to the Google Groups "SIMP Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to simp-users+unsubscribe@googlegroups.com.
To post to this group, send email to simp-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/simp-users/591d15df-219e-4709-b552-edf787583fd0%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Trevor Vaughan
Vice President, Onyx Point, Inc

(410) 541-6699 x788

-- This account not approved for unencrypted proprietary information --

[smithcl...@gmail.com]

Hello Trevor,

You were correct, the issue seemed to be with the version of mock. It seems mock 1.4.2-1 has moved all the contents of /etc/pki/mock/ so when using that version I get the error 

GPG key retrieval failed: [Errno 14] curl#37 - "Couldn't open file /etc/pki/mock/RPM-GPG-KEY-CentOS-7"

I have fixed this by running

sudo yum remove mock && sudo yum install mock-1.2.17-1.el7.centos

Note that this only works on CentOS, I cannot find an older version of mock for RHEL. Once I did this I was able to get much further along. I am now near the end of the build however I am getting 

Updating: syslinux-tftpboot-4.05-13.el7.x86_64.rpm with syslinux-tftpboot-4.05-12.el7.x86_64.rpm

Warning: There were errors updating some files:

  * postgresql94-9.4.11-2PGDG.rhel7.x86_64.rpm => Could not download

  * postgresql94-libs-9.4.11-2PGDG.rhel7.x86_64.rpm => Could not download

  * postgresql94-server-9.4.11-2PGDG.rhel7.x86_64.rpm => Could not download

  * puppet-agent-1.8.3-1.el7.x86_64.rpm => Could not download

  * puppet-client-tools-1.1.1-1.el7.x86_64.rpm => Could not download

  * puppetdb-termini-4.3.0-1.el7.noarch.rpm => Could not download

  * puppetlabs-release-pc1-1.1.0-5.el7.noarch.rpm => Could not download

================================================================================

Failed ISOs:

  * RedHat 7 x86_64

    * Error: Could not update all packages

    * Context: /home/vagrant/.rvm/gems/ruby-2.1.9/gems/simp-rake-helpers-3.4.0/lib/simp/rake/build/build.rb:517:in `block in update_packages'

 It seems everything else is working correctly, so I am not sure the cause of this specific error.

Thank you,

Clayton

On Friday, August 11, 2017 at 9:22:47 AM UTC-4, smithcl...@gmail.com wrote:

[Trevor Vaughan]

You *should* be able to install the older version of mock from CentOS on RHEL without any issues.

It would be *really* nice if the upstream repos would stop moving everything.

For the items that are not being found, you should be able to add a file with the following at simp-core/build/distributions/RedHat/7/x86_64/yum_data/repos/simp_6.repo.

[simp6]

name=simp6

baseurl=https://simp-project.com/yum/SIMP/6.X/EL7/core/x86_64/

enabled=1

[simp6-deps]

name=simp6-deps

baseurl=https://simp-project.com/yum/SIMP/6.X/EL7/dependencies/x86_64/

enabled=1

We're going to start pointing into our repos directly as time goes on because we've gotten tired of shifting Internet packages causing our builds to break.

Thanks,

Trevor

--
You received this message because you are subscribed to the Google Groups "SIMP Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to simp-users+unsubscribe@googlegroups.com.
To post to this group, send email to simp-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/simp-users/59c92c1b-4228-4816-b9ca-fb71ba4d1608%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[smithcl...@gmail.com]

Adding the repos did fix the issue, I now seem to be getting failures on the ISO build step. I have tried the build on both CentOS and RHEL systems and seem to be getting the same issue. I saw the step of copying the additional RHEL yum repos over on the RHEL system and I have tried that, however the error still persists. The error is as follows

================================================================================
#### iso:build[/home/vagrant/simp-core/build/distributions/RedHat/7/x86_64/DVD_Overlay/SIMP-6.0.0-0.el7.centos-RedHat-7-x86_64.tar.gz, /home/vagrant/simp-core/build/distributions/RedHat/7/x86_64/SIMP_ISO_STAGING, true]
================================================================================

--------------------------------------------------------------------------------
### Pruning packages not in file '/home/vagrant/simp-core/build/distributions/RedHat/7/x86_64/SIMP_ISO_STAGING/RedHat7.3-x86_64/7-simp_pkglist.txt'
   (override this with `SIMP_PKGLIST_FILE=<file>`)
--------------------------------------------------------------------------------
Starting to prune...
Info: Pruned 0 packages from /home/vagrant/simp-core/build/distributions/RedHat/7/x86_64/SIMP_ISO_STAGING/RedHat7.3-x86_64
mkdir x86_64
mkdir 5
Spawning worker 0 with 47 pkgs
Spawning worker 1 with 47 pkgs
Spawning worker 2 with 46 pkgs
Spawning worker 3 with 46 pkgs
Workers Finished
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
WARN -> Puppetfiles should not contain duplicate module names and will result in an error in r10k v3.x.
WARN -> Remove the duplicates of the following modules: rsync
mkdir -p repos/base
mkdir -p repos/lookaside
mkdir -p repodata
createrepo .
Spawning worker 0 with 505 pkgs
Spawning worker 1 with 505 pkgs
Spawning worker 2 with 505 pkgs
Spawning worker 3 with 505 pkgs
Workers Finished
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
createrepo .
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete

================================================================================
Failed ISOs:
  * RedHat 7 x86_64

    * Error: Error: REPOCLOSURE FAILED:
Repository base is listed more than once in the configuration
Reading in repository metadata - please wait....
Checking Dependencies
Repos looked at: 2
   base
   lookaside
Num Packages in Repos: 2019
package: libarchive-devel-3.1.2-7.el7.x86_64 from base
  unresolved deps: 
     libarchive(x86-64) = 0:3.1.2-7.el7
package: puppetserver-1.1.3-1.el7.noarch from base
  unresolved deps: 
     puppet < 0:4.0.0
package: razor-server-1.1.0-1.el7.noarch from base
  unresolved deps: 
     razor-torquebox
package: redhat-support-tool-0.9.8-6.el7.noarch from base
  unresolved deps: 
     python-magic
package: simp-adapter-0.0.3-0.el7.noarch from base
  unresolved deps: 
     puppetserver >= 0:2.6.0
     puppetdb >= 0:4.2.2
package: subscription-manager-1.17.15-1.el7.x86_64 from base
  unresolved deps: 
     python-rhsm >= 0:1.17.0
    * Context: /home/vagrant/.rvm/gems/ruby-2.1.9/gems/simp-rake-helpers-3.4.0/lib/simp/rake/build/pkg.rb:724:in `block (4 levels) in define_tasks'

I have tried cleaning the yum cache and made sure I had the latest gems with bundle update and the issue has still persisted. This is happening while using the tarball to build.

Thank you,

Clayton

On Friday, August 11, 2017 at 9:22:47 AM UTC-4, smithcl...@gmail.com wrote:

[Trevor Vaughan]

Sorry about all of the trouble Clayton.

Can you try applying this patch and see if it resolves everything for you?

https://github.com/simp/simp-core/commit/8298f4b955c2e3eb14907baf1182de268f196a71

Thanks,

Trevor

--
You received this message because you are subscribed to the Google Groups "SIMP Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to simp-users+unsubscribe@googlegroups.com.
To post to this group, send email to simp-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/simp-users/a2a8fb0b-60ae-449c-894a-6f00a709fd4b%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[smithcl...@gmail.com]

Trevor,

Thank you for the suggestion, I have managed to cherry-pick the commit and with a few changes I was able to complete my build. Just for reference I will attach the Vagrantfile I used to build. The following steps are the additional changes I had to perform for future reference:

1. The RHEL7 commit that I cherry-picked (8298f4b955c2e3eb14907baf1182de268f196a71) uses $releasever in the simp repo path as part of the URL. On CentOS $releasever resolves to "7" however on RHEL $releasever resolves to "7Server" which causes the URLs for the SIMP packagecloud to 404.
   
2. The keys that the RHEL server uses to authenticate against repos were initially set to not be readable by regular users, however the rake tasks and therefore yumdownloader runs as the current user. I set the permissions for all keys in the entitlement directory to 755 to work around this `chmod 755 /etc/pki/entitlement/*`
   
3. Somehow there were still 3 packages missing related to rhel subscription manager. It was causing the following error:
   

==> default: Failed ISOs:
==> default:   * RedHat 7 x86_64
==> default:     * Error: Error: REPOCLOSURE FAILED:
==> default: Reading in repository metadata - please wait....
==> default: Checking Dependencies
==> default: Repos looked at: 2
==> default:    base
==> default:    lookaside
==> default: Num Packages in Repos: 2019
==> default: package: redhat-support-tool-0.9.8-6.el7.noarch from base
==> default:   unresolved deps: 
==> default:      python-magic
==> default: package: subscription-manager-1.17.15-1.el7.x86_64 from base
==> default:   unresolved deps: 
==> default:      python-rhsm >= 0:1.17.0
==> default:     * Context: /home/vagrant/.rvm/gems/ruby-2.1.9/gems/simp-rake-helpers-3.4.0/lib/simp/rake/build/pkg.rb:724:in `block (4 levels) in define_tasks'

I was able to work around the error by running `yumdownloader -c yum_tmp/yum.conf python-rhsm python-rhsm-certificates python-magic` in the `~/simp-core/build/distributions/RedHat/7/x86_64/yum_data/packages/` directory. After that I was able to get a successful build.

If anyone else attempts to build using my Vagrantfile which is attached, please note that you will need to have a RHEL account (I believe a developer account, which is free, will do fine) and you should run the command as follows:

RHEL_USERNAME='<your email here>' RHEL_PASSWORD='<your RHEL account password here>' vagrant up

 

I do have one final question, are there any issues with the hack I have in for downloading the additional RedHat dependencies or should that work properly?

Thank you,

Clayton

On Friday, August 11, 2017 at 9:22:47 AM UTC-4, smithcl...@gmail.com wrote:

[Trevor Vaughan]

Hi Clayton,

Thanks for sharing the Vagrantfile that you're using, hopefully others will find it useful as well!

There's no issue with the hack that you have in place, that should be fine.

We're working (once again) at making this a LOT easier in the future and decoupling the entire process so that the ISO build is much more straightforward.

I'm glad that you got it working and apologize that it was so much trouble.

Thanks,

Trevor

--
You received this message because you are subscribed to the Google Groups "SIMP Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to simp-users+unsubscribe@googlegroups.com.
To post to this group, send email to simp-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/simp-users/944f1237-26e3-4e8b-9eb6-56fd7c045893%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.