List of latest images (Debian and Ubuntu)

[burnettb317]

Is there a URL for all the latest images?  ( http://www.emulab.net/downloads/images-STD/  doesn't work anymore)

previously listed images were:

CENTOS71-64-STD : https://www.emulab.net/image_metadata.php?uuid=c74d9fc1-e54b-11e4-97ea-38eaa71273fa 

FBSD102-64-STD  : https://www.emulab.net/image_metadata.php?uuid=57e6bf53-4376-11e5-bf03-38eaa71273fa 

XEN46-64-STD    : https://www.emulab.net/image_metadata.php?uuid=61799d52-4dff-11e6-ac8a-90e2ba22fee4

UBUNTU14-64-STD : https://www.emulab.net/image_metadata.php?uuid=0a29c738-32b3-11e4-b30a-001143e453fe 

UBUNTU15-04-64-STD : https://www.apt.emulab.net/image_metadata.php?uuid=c1acf46a-f8eb-11e4-8b63-2f7555356a5c 

UBUNTU15-10-64-STD : https://www.apt.emulab.net/image_metadata.php?uuid=0b11252a-caa8-11e5-9986-bdbdba4fe610 

UBUNTU16-64-STD: https://www.emulab.net/image_metadata.php?uuid=7683ca8e-5e37-11e6-ac8a-90e2ba22fee4 

These can then be imported with:  image_import

We are looking for Debian 8 and 9 images specifically.

Thanks,

--Ben

[jason....@gmail.com]

I'm looking for older OS image as well, and noticed that http://www.emulab.net/downloads/images-STD/ still gets "non-existent page" error...

(I couldn't log in to emulab.net due to account inactivity)

Any chance you guys have a non-updated CentOS 7.3 image?

Thanks,

Jason

[Leigh Stoller]

at 6:01 PM, jason....@gmail.com <jason....@gmail.com> wrote:

> I'm looking for older OS image as well, and noticed that http://www.emulab.net/downloads/images-STD/ still gets "non-existent page" error...
> (I couldn't log in to emulab.net due to account inactivity)
> Any chance you guys have a non-updated CentOS 7.3 image?

Hi. I updated the image importing wiki page (at the bottom) with the list of
images we currently support.

https://gitlab.flux.utah.edu/emulab/emulab-devel/-/wikis/install/Importing-OS-Images-From-Utah

We do not have a CentOS 7.3 image. The last one we did on the 7.X line
is 7.1 …

Leigh

[jason....@gmail.com]

Thanks Leigh!

If that URL is the same "CENTOS7-64-STD" from last year, it was already up to 7.7.  Do you have the URL for the older 7.1 image you mentioned?

Regards,

Jason Boles

[David M. Johnson]

On 1/9/21 9:54 AM, jason....@gmail.com wrote:
> Thanks Leigh!
> If that URL

> <https://www.emulab.net/image_metadata.php?uuid=6fa68fd6-9163-11e6-ac8c-90e2ba22fee4> is

> the same "CENTOS7-64-STD" from last year, it was already up to 7.7.  Do
> you have the URL for the older 7.1 image you mentioned?

You should use CENTOS7-64-STD unless you know that your use case really
cares about a diff in some package between 7.3 and 7.9 , and that
there's no hack around it.

The issue is that although CentOS 7 followed a (minor) point release
model in name, the only way to obtain security updates was to jump to
the latest version of packages from the current minor release, because
CentOS 7 doesn't provide separate repos for each minor release, nor
minor release-specific updates (nor does it provide separate
security(+dep)-only package metadata, so `yum update-minimal --security`
does not do anything). This is the main reason we stopped using point
releases in the image name -- we don't want to supply images with known
vulnerabilities long-term, nor can we support 9 separate CentOS 7 images
each with a point release of CentOS.

We made this switch between CENTOS71-64-STD (whose head was surely
already 7.2 at the time) and CENTOS7-64-STD, right after 7.3 was
released. Unfortunately we did not update the Description and OS fields
correctly in the CENTOS7-64-STD image. So you end up going by the
Updated timestamp to see which of the old versions you might want to
try. The first version after 7.3 was release is probably version 3; the
last version prior to the 7.4 release looks to be version 7 (e.g.
https://www.emulab.net/portal/show-image.php?imageid=5156&version=7).

I wouldn't really recommend using this version, though... it will have
plenty of security vulnerabilities and bugfixes to the Emulab
clientside, and we can't support it.

Sorry, unsatisfying answer, but we only have enough staff to effectively
follow the upstream long-term stable policies of upstream Linux distros.
We have to hope that the vast majority of ABI changes only occur at
major release boundaries. Sometimes it works ;)

> Regards,
> Jason Boles

David

> On Saturday, January 9, 2021 at 10:38:58 AM UTC-5 sto...@flux.utah.edu
> wrote:
>
> at 6:01 PM, jason....@gmail.com <jason....@gmail.com> wrote:
>
> > I'm looking for older OS image as well, and noticed that
> http://www.emulab.net/downloads/images-STD/ still gets "non-existent
> page" error...

> > (I couldn't log in to emulab.net <http://emulab.net> due to

> account inactivity)
> > Any chance you guys have a non-updated CentOS 7.3 image?
>
> Hi. I updated the image importing wiki page (at the bottom) with the
> list of
> images we currently support.
>
> https://gitlab.flux.utah.edu/emulab/emulab-devel/-/wikis/install/Importing-OS-Images-From-Utah
>
>
> We do not have a CentOS 7.3 image. The last one we did on the 7.X line
> is 7.1 …
>
> Leigh
>

> --
> You received this message because you are subscribed to the Google
> Groups "emulab-admins" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to emulab-admin...@googlegroups.com
> <mailto:emulab-admin...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/emulab-admins/5ab3c9b5-4bbe-4f57-bc8b-b33f13287c4cn%40googlegroups.com
> <https://groups.google.com/d/msgid/emulab-admins/5ab3c9b5-4bbe-4f57-bc8b-b33f13287c4cn%40googlegroups.com?utm_medium=email&utm_source=footer>.

[jason....@gmail.com]

Hi David,

  I understand the security implications of older OS images, and I think you guys are doing the right thing w.r.t. only having 1 centos7 image that has applied updates.

  The particular reason I'm asking is because we want to test Intel OFED+ driver distribution for some old QLogic (bought by intel) QDR Infiniband HCAs, which they EOL'd and last download version was from 2018, and the SRPMs are based on OFED 4.8-1, which only supports up to RHEL 7.3.  The same compiler error message I'm getting with CentOS 7.7 was found in openfabrics mailing list thread, in which the maintainer Vlad said they don't support the kernel from 7.4 yet (back when 4.8-1 was latest).   The cards do work with newer OS and OFED, however the performance was much lower than expected.  Chuck found and fixed a threading bug in the PSM library (also abandoned by intel, in favor of PSM2, which is used by their omni-path products) - so it's not a big deal as long as we use PSM.  There was an old HPE support thread about QlogicOFED+ having fixes in the ib_qib driver for improved ibverbs performance, so just wanted to verify that claim.

image_import didn't like "&version=7" in the URL, but I noticed in that metadata there was a different UUID that referenced the same .ndz:7, and using that new URL, image_import worked to download that version.  I loaded the Image onto a node, and indeed it is release 7.3.1611, and also the intel OFED+ installer was successful.  (performance testing can wait til next week)

Thanks again!

Regards,

Jason Boles

[Brenton Walker]

Dear Emulab Admins,

    I saw that there was a new set of STD images released about a year ago, including UBUNTU22 (https://groups.google.com/g/emulab-users/c/UZG_fgVMIrs).  It is possible to use these images on smaller installations, like ours?  The install instructions, and the links in the emulab-devel git repo still use the UBUNTU20 image. 

I understand they use a new partition layout, so we may have some upgrading to do.  Our current install is from early 2023.

Thanks,

Brenton

[Mike Hibler]

Yes, you can use those images. You will need to update your server ("boss")
software to properly handle the new image format. You should do that before
you try to `image_import` any of the new images.

Ugh, you will also need to update your MFSes used to install and capture
images, so I will have to put together a new MFS tarball. I don't think you
new a new image distribution ("firsbee") MFS, but you will need a new
image capture ("admin") MFS if you want to create custom images based on
the new format standard images.

> --
> You received this message because you are subscribed to the Google Groups
> "emulab-admins" group.
> To unsubscribe from this group and stop receiving emails from it, send an email

> to emulab-admin...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/

> emulab-admins/1114044c-9c5e-4c00-80af-96d2485098ean%40googlegroups.com.

[Mike Hibler]

And just to be clear, you only need to update the Emulab software, not
the underlying FreeBSD on the boss and ops nodes.

On Fri, May 03, 2024 at 04:00:14AM -0700, Brenton Walker wrote:
> Thank  you.  I will work on upgrading boss to the current version.

> emulab-admins/63055596-4a6f-45b7-a1d2-aa5097e37b4bn%40googlegroups.com.

[Chuck Cranor]

On Fri, May 03, 2024 at 11:29:57AM -0600, Mike Hibler wrote:
> And just to be clear, you only need to update the Emulab software, not
> the underlying FreeBSD on the boss and ops nodes.

also, if you need to do something quicker, it is also possible
to build a UBUNTU22 ndz image with the older layout by starting
with the UBUNTU20-64-STD image, using the OS tools to upgrade it
to UBUNTU22, and then creating a new image from that. the dkms
mechanism should pull the emulab ipod drive along with it.


e.g. this is an emulab MBR version 3 image of ubuntu22 i made a year ago:


boss:~ % ssh h0.ok.testbed
Welcome to Ubuntu 22.04.2 LTS (GNU/Linux 5.15.0-73-generic x86_64)

* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
h0:~ % df /
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 16337876K 4231328K 11267688K 28% /
h0:~ % lsmod | fgrep ipod
ipod 16384 0
h0:~ %


which i made by:
- boot UBUNTU20-64-STD

- start update to 22:
# update current system and reboot
# node that ipod driver is in dkms so it updates too
apt update -y
apt upgrade -y
-> choose "keep local version currently installed" for ssh dialog
reboot

- relogin, get root shell:

# to be safe
apt install update-manager-core
# now do it
do-release-upgrade -d
-> "y" for ssh on 1022
-> enter to start
-> "y" on start upgrade
-> "y" on restart services
-> "n" -- keep my current /etc/sysctl.conf
-> "keep local version" of sshd_config
-> "keep local version" of unattended-upgrades file
-> "y" to remove obsolete software
-> "y" to restart

- relogin, get root shell:

# to be safe
apt autoremove --purge
apt-get clean
apt-get -y autoremove
apt-get clean

- clean up disk space

cd /boot
rm -rf config-5.4.0-150-generic
rm -rf initrd.img-5.4.0-150-generic
rm -rf vmlinuz-5.4.0-150-generic
rm -rf System.map-5.4.0-150-generic

rm initrd.img.old
ln -s initrd.img-5.15.0-73-generic initrd.img.old

rm vmlinuz.old
ln -s vmlinuz-5.15.0-73-generic vmlinuz.old

cd /usr/lib/modules
rm -rf 5.4.0-*

- create a new ndz image from the node you've been using and install it


at least that worked for me... :)


chuck

[Mike Hibler]

Thanks for the help Chuck!

> --
> You received this message because you are subscribed to the Google Groups "emulab-admins" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to emulab-admin...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/emulab-admins/ZjUlbk6DNBL3Iqin%40ece.cmu.edu.

[Brenton Walker]

Thank you!  I will give this a try.  

Brenton

[Brenton Walker]

Thank you both.  I was able to create a working UBUNTU22 image by upgrading UBUNTU20.

I also did the upgrade on boss.  The only small snag was I had to manually install cracklib from the ports tree.

Once installed, almost everything seems to work, including editing profiles and image capture.  The main problem I find is that the "instantiate" page comes up blank, with only the blue header and footer.  This happens whether I access it through the "Start Experiment" menu item, the "Instantiate" link on a profile, or the "re-run this experiment" link on the history page.  I have not been able to find any clues in the log files so far.  Any ideas?  Or is it best to revert to the backup?

Thanks!
Brenton

[Leigh Stoller]

> Once installed, almost everything seems to work, including editing profiles and image capture. The main problem I find is that the "instantiate" page comes up blank, with only the blue header and footer. This happens whether I access it through the "Start Experiment" menu item, the "Instantiate" link on a profile, or the "re-run this experiment" link on the history page. I have not been able to find any clues in the log files so far. Any ideas? Or is it best to revert to the backup?

Hi. There are two places to find errors. One is your testbed-ops email, are
you reading that?

Also, make sure boss:/usr/local/etc/php.ini has this:

;
; Redirect php errors someplace easier to see
;
display_errors = Off
log_errors = On
error_log = "/usr/testbed/log/php-errors.log"

Be sure to create the file and set it to 666. Then you can look
in that file to see if there are problems in the php code.

Leigh

[Brenton Walker]

Thank you fort the tips.

I did those things, but cannot see any error messages.  I am expecting errors to log when I try to load the instantiate.php page, right?

[brenton@boss ~]$ ls -l /usr/testbed/log/php-errors.log
-rw-rw-rw-  1 root  tbadmin  0 May 10 11:32 /usr/testbed/log/php-errors.log

In the testbed-ops emails I am also not getting errors.  The closest I see to something unusual is when I reboot boss, I get the following from the ClearingHouseDaemin:

----------

This is an automated message from the ClearingHouse Daemon

Up:

Down:
urn:publicid:IDN+filab.uni-hannover.de+authority+cm

Disabled:

-----------

I also tried turning "display_errors = On" in /usr/local/etc/php.ini, then setting the permission back to unwritable, but saw no errors.  To be thorough, I rebooted boss between config file changes.

It may be that there is no error on the server side.  When I look at the HTML that is being retrieved on the client side, it appears to contain all the information I would expect.  It has the "form-json" and "profiles-json" script blocks, filled in with reasonable looking values. 

The only actual error I find is on the client-side browser console:

Uncaught TypeError: Cannot read properties of undefined (reading 'contentWindow')
    at Object.create (showtopoiframe.js:276:29)
    at Object.initialize [as init] (profile-picker.js:86:23)
    at HTMLDocument.initialize (instantiate.js:216:20)
    at i (jquery.min.js:2:27151)
    at Object.fireWith [as resolveWith] (jquery.min.js:2:27914)
    at Function.ready (jquery.min.js:2:29707)
    at HTMLDocument.J (jquery.min.js:2:29892)

Note also, I upgraded boss this week, but not ops.  Will that cause problems?  I did this because I have boss in a small VM that I can easily snapshot and roll back to the prior version if the upgrade has problems.  ops would be a little more work.

Thanks,

Brenton

[Leigh Stoller]

> Note also, I upgraded boss this week, but not ops. Will that cause problems? I did this because I have boss in a small VM that I can easily snapshot and roll back to the prior version if the upgrade has problems. ops would be a little more work.

In general yes, you want to upgrade ops as well.

On boss, can you do the equivalent of this:

{5} boss$ manage_aggregate list
Name Nickname Abbrev
--------------------------------------------------------------
POWDERFEBOOKS POWDERFEBOOKS POWDERFEBOOKS

{6} boss$ manage_aggregate POWDERFEBOOKS show
Name: POWDERFEBOOKS

Leigh

[Brenton Walker]

Thanks.

I will have to do something similar with ops, and then try that upgrade.  It is also in a VM.  There are just more larger volumes to snapshot.  Not snapshotting /users, of course.

Here is the result of the command you suggested.

[brenton@boss ~]$ manage_aggregate list
Name                           Nickname        Abbrev
--------------------------------------------------------------
filab                          filab           filab

[brenton@boss ~]$ manage_aggregate filab show
Name:            filab
NickName:        filab
Abbrev:          filab
URN:             urn:publicid:IDN+filab.uni-hannover.de+authority+cm
URL:             https://www.filab.uni-hannover.de
Disabled:        No
Admin:           No
Federate:        No
FE               No
Mobile:          No
NoUpdate:        No
NoMonitor:       No
Deferrable:      No
Datasets:        Yes
Reservations:    Yes
SyncThing:       No
LocalImages:     Yes
PrestageImages:  No
PanicPowerOff:   No
PreCalcMaxExt:   No
Portals:         emulab
Use Feature:
Status:          up

Thanks,

Brenton

[Brenton Walker]

I found an apparent solution to this blank instantiate page problem.  There is a old and new version of the instantiate javascript.  The old version throws that error from showtopoiframe.js.  I did not dig deeper to figure out why, because switching to the new version appears to work.

$ emulabfeature create UseNewInstantiate "use the new instantiate javascript code"

$ emulabfeature enable UseNewInstantiate

Please let me know if I am activating unstable code and should be finding a way to make he old instantiate work instead.

Thanks,

Brenton

[Leigh Stoller]

Hi. Thanks for figuring that out. You are good to use the new instantiate at this point.

Leigh