Grid Appliance 2.04.00 Preview

[David Isaac Wolinsky]

Good news:

We are wrapping up the next version of the Grid Appliance - 2.04.00.

Here are some exciting features that this new version brings:
- We now use Ubuntu 9.10 sporting both 32 and 64-bit versions!
- Simics now supports both 32 and 64-bit versions!
- Simplified VM, no longer supporting COW though we have modularized and
packaged all our components to enable easy upgrades -- apt-get update;
apt-get upgrade
- Newer versions of the Virtual Network software for faster connectivity
- User names are no longer "griduser" instead it will be the name you
created on the website

Some notes about appliances and simple config:
- Booting from the floppy is no longer supported, if you're using
something other than VMWare, ensure that your first boot device is the
hard disk
- If you compiled a 64-bit application, ensure it is only sent to 64-bit
machines by adding this line in your condor submit files before the
queue statements:
Requirements = Arch == "X86_64"
- GCC is no longer installed by default, install it via sudo apt-get
install gcc
- To compile 32-bit apps, install libc6-dev-i386 via sudo apt-get
install libc6-dev-i386
- All 64-bit VMs can run 32-bit jobs
- VMs that are in the pool right now that are marked as 64-bit may not
really support running 64-bit jobs, so temporarily add the following to
all configs for 64-bit:
Requirements = Arch == "X86_64" && Group == "ACIS" -- this should be
fixed as soon as the public release is made.
- VMs are limited to 8gb of space, we will be working on solutions that
enable users to add more disk space as seamlessly as possible

Some notes about SIMICS:
- For running simics, make sure you are explicit about the environment,
always prepend all scripts with:
export SIMICS_HOST=arch-linux
or
export SIMICS_HOST=amd64-linux
even if you're running from a 32-bit machine! Simics handles the rest.
Though this could use testing from the community to verify that nothing
has been broken.
- Simics directory has moved to /opt/virtutech/simics3

Downloads:
http://www.grid-appliance.org/files/appliances/grid-appliance-386.zip
http://www.grid-appliance.org/files/appliances/grid-appliance-amd64.zip

If all goes well, we'll make this an official release on 01/18.

Regards,
David

[Dimitris Kaseridis]

Hello everybody,

I am in the process of moving our appliances to the new 2.04 version.
The web-site states:

"Release 2.04 upgrades the system to Ubuntu 9.10 with both 64-bit and
32-bit environments. In addition, all components of the VM are now
available in an apt repository for users to configure their own "Grid
Appliance" external a VM. "

running through the vm significantly slows down my simulations in
comparison to native run and it will be great if I could install the
Grid appliance on an clean ubuntu 9.10 system through an apt
repository.

Can someone point me to a documentation on how to do that? I can't
find this info on the web-site...

Thanks
--
Dimitris

> --
> You received this message because you are subscribed to the Google Groups
> "Archer User's Group" group.
> To post to this group, send email to archer-us...@googlegroups.com.
> To unsubscribe from this group, send email to
> archer-users-gr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/archer-users-group?hl=en.
>
>
>
>

[Renato Figueiredo]

Dimitris,

You can find a tutorial on how to install an individual node from Ubuntu 9.10 here:

http://www.grid-appliance.org/index.php?option=com_awiki&view=mediawiki&article=TestingGridAppliance

The instructions guide you through creating an appliance image, but the same sequence of installing packages will work on a physical machine.

I think David or Perhaad may also be able to provide hints on how to automate deployment across a cluster.

--rf

--
Dr. Renato J. Figueiredo
Associate Professor
ACIS Lab - ECE - University of Florida
UF Site Director, Center for Autonomic Computing
http://byron.acis.ufl.edu
ph: 352-392-6430

[Arjun Prakash]

Hi Dimitris, 

We also have a video tutorial in our youtube channel explaining the process of creating a custom appliance. Though it shows the steps for setting up a virtual MPI appliance, it should help you with understanding the steps in creating a physical appliance. The video can be found at http://www.youtube.com/acisp2p#p/c/D77781CEF51F72F3/2/nyg3Aye21wk  - steps mentioned from 03:30 minutes onwards would be relevant to you. 

Regards
Arjun Prakash

[Dimitris Kaseridis]

Ok thanks for the response. I will have a look at the video/site ..

--
Dimitris

[Dimitris Kaseridis]

Hi again,

I did a test installation and everything seems working great. I manage
to get a client and a worker running fine.
Thanks for the info..

One notice though.... I am currently one of the admins of the
UT-Austin pool and expect our pool it seems like nobody else has
simics installed.

THis is what I get from condor_q -better-analyse

" Condition Machines Matched Suggestion
--------- ---------------- ----------
1 target.HasArcherSimics == true 60
2 target.Memory >= 1024 339
3 target.Arch == "X86_64" 371
4 ( ( target.ARCH == "INTEL" ) || ( target.ARCH == "X86_64" ) )
376
5 ( target.OpSys == "LINUX" ) 376
6 ( target.Disk >= 22500 ) 376
7 ( target.HasFileTransfer ) 376 "

Some of our machines are down but I am going to fix that on monday...
so expect our 60 cores (120 normally) that the HasArcherSimics is true
... the rest seems like they don't support it..

Am I using the wrong flag? Our workers are still running the 2.03
version... If I move them to the new version is that going to change
sth?

If I am not doing sth wrong and the rest of the pools haven't install
simics... Is there any way to ask them to do it?
I will really want to use more than our cores from time to time for
simics/GEMS simulations.

Thanks
--
Dimitris

[David Isaac Wolinsky]

Response to all questions:

HasArcherSimics has been removed in favor of:
HasArcherX86
HasArcherAmd64

(i.e., Simics now has x86 and Amd64 binaries)

Professor Figueiredo has already pointed a link to the new appliance
build package, but there is actually some cluster deployment scripts
that make it much easier to setup a cluster. If you're going to delay
until Monday, I'll put together a package tomorrow, make a wiki, and
send a link describing the process. I'll also be on gmail to assist you
through the process as necessary. Well, before I get too ahead of
myself, would you have the ability to setup a PXE/TFTP/DHCP server on
the RAID machine, set all the other machines to be connected to a
private LAN with the RAID machine, RAID machine is then connected to the
Internet, and then reboot all the other machines into boot from pxe
mode. That's pretty much most of the setup to get an easy to manage
Archer cluster. If you have an opportunity, the key things that you can
do ahead of time are the physical components (setting up the LAN) but
don't worry about software.

I'll check the Simics installation on other resources on Monday. I
actually upgraded your local resources to the new Simics the other day,
or so I thought. I had planned on reaching out to you guys about Archer
on the physical hardware but hadn't had the opportunity due to schedule
conflicts, my apologies.

Regards,
David

[Dimitris Kaseridis]

Thanks for the response.... it seems things changed a lot from our
last communication.

so HasArcherX86, HasArcherAmd64 flags imply an x86/amd64 installation
of simics?

> If you're going to delay until Monday, I'll
> put together a package tomorrow, make a wiki, and send a link describing the
> process.  I'll also be on gmail to assist you through the process as
> necessary.

Yes I have to wait up to Monday since I am executing some simulations
on our pool and it is going to take a while for them to finish.

> Well, before I get too ahead of myself, would you have the
> ability to setup a PXE/TFTP/DHCP server on the RAID machine, set all the
> other machines to be connected to a private LAN with the RAID machine, RAID
> machine is then connected to the Internet, and then reboot all the other
> machines into boot from pxe mode.  That's pretty much most of the setup to
> get an easy to manage Archer cluster.  If you have an opportunity, the key
> things that you can do ahead of time are the physical components (setting up
> the LAN) but don't worry about software.

OK I think I can see your plan... seems easier to control all the
workers OSes in that way.
I believe I can easily do the setup... otherwise we can stick to the
"one Ubuntu9.10 per blade" installation...
but don't rush... I am pretty sure my simulations wont finish before
late Monday.

Thanks for the update,
Let me know when the wiki is ready.
Dimitris

[David Isaac Wolinsky]

I'm going a bit slower than I intended. Here's what I've gotten done so
far:
http://www.grid-appliance.org/wiki/index.php?title=Cluster

Its quite barren. I still need to update how to add Ubuntu Net Install,
the preseed file (for automated install), and how the whole thing
integrates with Archer.

Regards,
David

[David Isaac Wolinsky]

The content has been significantly updated:
http://www.grid-appliance.org/wiki/index.php/Cluster

I'd be glad to walk you through the process. There are a TON of
documents, easily accessible via google, that describe each of the
individual processes in more depth. My goal was to show a very minimal
set of steps to get it working.

Regards,
David

On 04/03/2010 11:01 PM, Dimitris Kaseridis wrote: