"vi" & Supercomputer Performance
Mike Muuss
The problem with your argument is that you implicitly assume
that it is cheaper to transfer an entire file to another machine,
than it is to do a little editing on the file on the Cray.
In my measurements (Cray XMP48 running COS 116, Cray XMP48 running
UNICOS 3, 4, and 5, Cray-2 running UNICOS 4 and 5), for "medium" sized
source files, it actually used *less* Cray time to suck the file into
a screen editor (JOVE and VI were tested), modify a "few" lines,
and write it back out, than it was to move the file to a "front end"
and then back to the Cray. (obviously, no screen editor on COS,
I just measured file xfer times, and found it more expensive of
Cray CPU time than a UNICOS FTP or RCP).
Now, I would not suggest developing new code from scratch on the Cray,
nor would I suggest using the Cray for document processing, but
there is no reason in the world that Cray users should not be allowed
to modify their Cray files using the Cray! Keep the bloody intermediate
machines (err, "front ends") out of my way, please.
Obviously, new code and document processing should be done on your
workstation.
Best,
-Mike Muuss
U. S. Army Ballistic Research Lab
Booker bense
- While I would be be the last person to say that emacs and vi should
not be on a CRAY. I believe the efforts toward distributed editing to
be extremely worthwhile. The consideration here is performance of the
'overall' system. An editing process requires an entry that remains in
the process table for a significant amount of time using a significant
portion of that time in system calls. Also your test of editing 'just
a few lines' is IMHO not a very typical example of how screen editors
are used. A more reasonable scenerio is the following.
I ftp my code to the cray.
It doesn't compile ( I tested it on a vax $%#@%$)
I begin editing the offending file.
I scroll up and down , Moving from declarations to the actual line
of the error.
Figure out that the error is a vax non-standard type declaration.
Edit a few lines.
quit the editor.
Recompile , it runs and crashes.
etc....
So while I have only made a few changes, I have been in the
editor for many minutes. All this time the editor is patiently waiting
for keypresses, unless of course my process got swapped out. Note: in
this example I have not used any more cpu time than in the test
examples above, however I have caused the scheduler that much more
grief. While I do not pretend to know the inner workings of the UNICOS
scheduler, I know trying to edit files at around 3pm here
is an exercise in patience.
As several earlier posters recommended I got the 'ange-ftp' package
and I highly recommend it. The difference in response time in editing
on my workstation is tremendous. I had not realized how `slow' editing
on the YMP is. Also, with a windowing system, I now have the best of
both worlds. In one window I'm logged onto the YMP using compile
commands , in another I'm editing in the emacs running on the
workstation.
In summary, a screen editor should be part of any supercomputer that
supports an interactive environment, however every effort should be
made to move editing off that machine. This effort should be as
transparent to the user as possible.
-Booker C. Bense /*ben...@grumpy.sdsc.edu*/
These are my own opinions, who else would be this foolish?
%-) No coffee
&-) 2-3 cups
8-) Ready to read news.
Dan A. Dickey
Excuse my ignorance here, but doesn't anyone out there use NFS?
It seems like a fairly ideal method of editing files on your workstation,
and then compiling & testing on the Cray. Just have the workstation
NFS mount the Cray file system. Or am I missing something?
-Dan
---------
Dan A. Dickey ddi...@aspen.cray.com
Malcolm Slaney
>Excuse my ignorance here, but doesn't anyone out there use NFS?
The image of supercharging (adding NFS) to a Model T (the Cray file
system) comes to mind.....
Seriously....I'm a big fan of our Cray and NFS but with 14 character
names and no symbolic links the Cray file system is not very friendly.
I understand all of this will be fixed in the next Unicos release.
Also, the lack of interruptable NFS mounts is a big lose. I realize that
Cray is tracking a moving target. Sun threw NFS out here and has been
slowly fixing their mistakes.
And finally, there are political problems. Here at Apple if I want to use
an NFS file system in my daily work I need to convince 5 different
organizations (with different implementations of NFS) to each mount the
file systems in the same place. And I have to convince each of them that
security won't be compromised.
Hoping for a better integrated world.....
Malcolm Slaney
Apple Perception Group
Mike Muuss
The problem with using NFS is that, in the scenario under discussion,
the Cray would be required to NFS transfer the *entire* file to the
editor running on the workstation, and then receive the *entire* file
back to the Cray again using NFS.
For changing a few lines in a medium-sized source file, this is STILL
more expensive of Cray time than running the editor locally on the Cray.
For highly modularized source code, where the source files are only
a few dozen Kbytes, using NFS in this way is certainly a performance "win".
But running a GREP through your sources via NFS is likely to be a big
loose.
Face it, folks: using the Cray interactively in the edit-compile-test
cycle is a good, efficient use of time. Both Cray time and user time.
Best,
-Mike