Grid Monitor Job Eviction

[ahoward]

Hi everyone, it's me again :-)

I'm trying to get job eviction working when there's keyboard/mouse
activity on the Windows host. Currently, it's not working properly,
and I'm not quite sure why. I'm not using the Grid Appliance, but just
have a regular VM with the IPOP binary install in it. Is there
something special I need to do to my VM to make it happy?

Thanks in advance!

[David Wolinsky]

Hey Andy,

Could you elaborate on your setup. Do you have the HostActivity.dll on
the guest? Does condor watch /dev/console?

David

[Andy Howard]

Hi David,
Yes, Condor watches /dev/console in our config, and I copied the
Brunet.Inject.HostActivity.dll to the bin/ directory (where I launch
DhtIpopNode). Do I need another DLL in there? I noticed the build
spits out two other DLLs, so I didn't know which ones to place on the
guest.

Thanks.
-Andy

--
Andrew Howard
Rosen Center for Advanced Computing
Purdue University

[David Wolinsky]

Brunet.Inject.HostActivity.dll runs on the guest in the ipop folder. It
needs to be there prior to starting Ipop at which point it is magically
instantiated by Ipop (through reflection).

GridMonitor.exe, Brunet.dll, and Brunet.IPAddresses.dll belong on the
host. If that doesn't work, grab the latest Brunet.dll and
Brunet.IPAddresses.dll from the lib folder of your latest IPOP src build.

If that's not working or something is unclear, let me know.

Cheers,
David

[David Wolinsky]

By the way, if you are using a tutorial, could you send me a link so I
know what you're referencing.

Thanks!
David

[Andy Howard]

Well I was following the short tutorial at:
http://www.grid-appliance.org/index.php?option=com_content&view=article&id=89&Itemid=76#troubleshooting

But also happened across your source code in the mercurial repo
(couldn't find anything in github), so I figured I'd try building that
for the guest. I couldn't find any tutorials that covered adding the
job eviction to a custom VM image.

[David Wolinsky]

Wowee... That's some old stuff :). I think there is some documentation
updating that needs to take place :). By the way, were you able to get
it working? Besides me verifying that it works on our end, I don't
think there are any other helpful hints that I can provide.

Regards,
David

[Renato Figueiredo]

Andy,

Have you had a chance to try the monitoring scripts again with your setup? Let us know if you still haven't been able to get it to work. Regards,

--rf

--
Dr. Renato J. Figueiredo
Associate Professor
ACIS Lab / Electrical and Computer Engineering
University of Florida
http://byron.acis.ufl.edu
ph: 352-392-6430

[Andy Howard]

Hi Renato,
I've been on vacation for the past 2 weeks and just got around to
working on this again.

I tried the monitoring scripts again to no avail. It seems the guest
OS is able to receive the commands just fine (I can send them via
another guest connected to Brunet with no problems), and the host is
recognizing the mouse clicks/keyboard activity. The problem appears to
be with the command coming from the host, and I'm not quite sure where
to go from here to debug it further. Any ideas?

FWIW I'm using the code from:
http://www.grid-appliance.org/hg/GridMonitor/file/69f982cfcca3/

Thanks!
-Andy

[David Isaac Wolinsky]

Can you ssh into the VM using any of its IP addresses? I think I
noticed in VMware Server 2 a strange behavior where the host couldn't
talk to any of the VMs on NAT or Host-only.

[ahoward]

I'm using VMware Server 1.0.8 (the latest 1.0 build). I can definitely
talk to the VM via the NAT/Host-only address. I was able to use a
python script to send the "HostActivity.UpdateConsole" command to the
IP directly. The problem seems to be when it tries to send the
multicast message, the message never actually gets there.

Any ideas why the multicast message might be getting lost? I added
some debug lines to the application to make sure it's actually trying
to send to the correct address, and it seems like it is.

Thanks,
-Andy

> >> On Wed, May 20, 2009 at 4:33 PM, David Wolinsky <davi...@ufl.edu> wrote:
>
> >>> Brunet.Inject.HostActivity.dll runs on the guest in the ipop folder.  It
> >>> needs to be there prior to starting Ipop at which point it is magically
> >>> instantiated by Ipop (through reflection).
>
> >>> GridMonitor.exe, Brunet.dll, and Brunet.IPAddresses.dll belong on the
> >>> host.  If that doesn't work, grab the latest Brunet.dll and
> >>> Brunet.IPAddresses.dll from the lib folder of your latest IPOP src build.
>
> >>> If that's not working or something is unclear, let me know.
>
> >>> Cheers,
> >>> David
>
> >>> Andy Howard wrote:
>
> >>>> Hi David,
> >>>> Yes, Condor watches /dev/console in our config, and I copied the
> >>>> Brunet.Inject.HostActivity.dll to the bin/ directory (where I launch
> >>>> DhtIpopNode). Do I need another DLL in there? I noticed the build
> >>>> spits out two other DLLs, so I didn't know which ones to place on the
> >>>> guest.
>
> >>>> Thanks.
> >>>> -Andy
>

[Renato Figueiredo]

I don't know if the multicast packet to all interfaces or to a single interface.
Do you happen to have wireshark or tcpdump in the guest and host? can you see if the multicast packet shows on the VMware vmnet host interface and then in the guest's eth interface?
--rf

[Andy Howard]

The multicast packet is sending only to the interfaces named "VMnet"
on the host. I didn't see the packet getting to the guest before, but
upon closer look it is in fact getting there. Now the question is why
the guest doesn't actually respond to this packet. Here's the output
from tcpdump on the guest (only for that packet):

14:09:34.491738 IP 192.168.17.1.4784 > 224.123.123.222.56123: UDP, length 41
0x0000: 4500 0045 6c64 0000 0111 1f41 c0a8 1101 E..Eld.....A....
0x0010: e07b 7bde 12b0 db3b 0031 1764 7200 015b .{{....;.1.dr..[
0x0020: df09 8170 0028 5f48 6f73 7441 6374 6976 ...p.(_HostActiv
0x0030: 6974 792e 5570 6461 7465 436f 6e73 6f6c ity.UpdateConsol
0x0040: 6500 2829 29 e.())

Any clues why IPhandler wouldn't pick that up?

[David Isaac Wolinsky]

Perhaps the Brunet library bundled with the GridMonitor isn't compatible
with the one on the Appliance. Is there anything in
/usr/local/ipop/var/ipop.log (shortly after a GridMonitr UpdateConsole call?

I'll run through this on my machine tomorrow.
Best,
David