A little stuck on starting up

[Dennis McEntire]

I have this configuration:

One box running the xtreemfs-dir service, listening on ports 32638 and
30638. It does NOT have MRC or OSD running.
One box running MRC and OSD, listening on ports 32640 and 30636. It
does NOT have the DIR service running.

Both are linux boxes, btw. The DIR box seems OK, when checking the
logs it appears to have started up properly. Hitting the IP:30638
shows the status page with information.

From the DIR box, these are the commands I have tried and the results:

xtfs_lsvol storagebox
xtreem1 -> <some UUID here>

xtfs_mount storagebox/xtreeem1 /tmp
errno = 111, strerror = Connection refused

No additional errors are generated in any log file, so I am not sure
how to proceed. This did work once when the initial test was done
where everything was done on one box. I am now trying to implement the
above setup and I get the connection refused error.

Any help would be appreciated!

Thanks in advance,

Dennis

[Minor Gordon]

Hello,

It sounds like xtfs_mount is unable to reach your MRC. Did you open
port 32636 on the MRC machine?

You can run xtfs_mount with
-f -d INFO --trace-network-operations

to see what it's trying to do on the network (and run in the foreground).

Hope this helps.

Minor

[Dennis McEntire]

Hi Minor, thanks for writing back. I have a couple questions.

So it appears that the DIRectory service is not listening on the
STORAGE box, which is true, I didn't start it. We wanted to use the
DIR service on another box. However, when I START the directory
service (xtreem-dir) on the STORAGE server, then the connection
refused error goes away, however, "unknown volume" shows up instead.

But the real question is:

Can we run the DIRectory service on one box, and have several storage
servers set up that are running both of the other daemons, the MRC and
OSD service daemon? That's what we are trying to do.

On the storage boxes, there are directives in the
/etc/xos/xtreemfs/mrcconfig.properties file that points to the DIR
box:

dir_service.host = xdir1.domainname.com (pointing to the other box
running the DIR service daemon)
dir_service.port = 32638

But it doesn't seem to talk to the DIR box. After writing all of this,
I hope it makes sense.

Thanks for your help, and if this architecture seems ok.

Dennis

--

==============================
Dennis McEntire - gmail account
dmce...@gmail.com
==============================

[Minor Gordon]

Hello Dennis,

On Wed, Nov 25, 2009 at 3:17 AM, Dennis McEntire <dmce...@gmail.com> wrote:
> So it appears that the DIRectory service is not listening on the
> STORAGE box, which is true, I didn't start it. We wanted to use the
> DIR service on another box. However, when I START the directory
> service (xtreem-dir) on the STORAGE server, then the connection
> refused error goes away, however, "unknown volume" shows up instead.

Did you xtfs_mkvol the volume first? The command is xtfs_mkvol
mrc_host/volume_name.

> Can we run the DIRectory service on one box, and have several storage
> servers set up that are running both of the other daemons, the MRC and
> OSD service daemon? That's what we are trying to do.

You can have the DIR, MRC, and OSDs on different machines, but there
can only be one MRC per volume, at least for the time being. Actual
file I/O is done with the OSDs, so you want to have these on different
machines so that they're close to clients on the network. The system
will pick OSDs for I/O based on network proximity (e.g. subnet
matching, network coordinates).

If you want multiple metadata servers then you'll need to set up
multiple volumes. This is inconvenient, I know, though it (a
centralized metadata server per volume) is a common limitation in
distributed file systems. We are working on replicated DIR and MRC
services.

> On the storage boxes, there are directives in the
> /etc/xos/xtreemfs/mrcconfig.properties file that points to the DIR
> box:
>
> dir_service.host = xdir1.domainname.com (pointing to the other box
> running the DIR service daemon)
> dir_service.port = 32638
>
> But it doesn't seem to talk to the DIR box.

That looks OK. Can you telnet to port 32638 from that machine? What
about xtfs_lsvol? You might try enabling debug.level = 6 or 7 in the
various configs and restarting the servers to get debug output in the
logs. Generally the MRC and OSDs will fail on startup if they can't
contact the DIR, however.

Minor

[Dennis McEntire]

Hi, thanks for the responses,

Let's start over with a new scenario. It doesn't seem to matter what I
do, I can't seem to get it working the way I think it should be
working.

Some facts:

1. Yes I did a xtfs_mkvol some time ago and it shows up when I do a
xtfs_lsvol - no problem there.

2. I tried moving the MRC service from the storage server over to the
directory server and changed the pointers, etc., and it didn't seem to
help.

3. The bottom line for me is that if the DIR service is NOT running on
the same box as the storage server, then I get a connection refused
error (#111) from any box.

So, in an effort to figure this out, I changed all the services to be
just on one box. I started all 3 services on one box, after changing
the configuration files of course. Once I did that, everything works,
no errors and I can mount the volume and put files into it. The files
get split as usual into the parts and put into the directories like
it's supposed to.

Here's the big question. Now that I have the single server setup and
running, what needs to happen to MOVE the directory service over to a
new server? This is where I am stuck. This is what I think needs to
happen, but correct me if I am wrong:

On the current storage server stop all the services, then update these files:
/etc/xos/xtreemfs/default_dir => update the host to point to the
directory server
/etc/xos/xtreemfs/dirconfig.properties => No change here since I am
not going to run a DIR service on this box.
/etc/xos/xtreemfs/mrcconfig.properties => Update dir_service.host to
point to the new directory server.
/etc/xos/xtreemfs/osdconfig.properties => Update dir_service.host to
point to the new directory server.

Then get the DIR service only (no MRC or OSD) running on the other
directory server, start everything up, and that's when I start getting
errors. I can still xtfs_lsvol storageserver.name and it shows my
volume, but I can't mount it (#111 connection refused).

Thanks again for your help,

Dennis

--

[Minor Gordon]

Hi Dennis,

I'm sorry you're having problems with this. Let's see if we can fix it
with this round.

1. Can you confirm that you're giving the DIR host to xtfs_mount (and
not the MRC)?
e.g. xtfs_mount <dir host>/volume_name /mnt

2. Can you run
xtfs_mount -f -d INFO --trace-network-operations <dir host>/volume_name /mnt
to see where it's trying to connect?

Regards,
Minor

[Björn Kolbeck]

Hi Dennis,

Dennis McEntire schrieb:

> Hi, thanks for the responses,
>
> Let's start over with a new scenario. It doesn't seem to matter what I
> do, I can't seem to get it working the way I think it should be
> working.

xtfs_lsvol/mkvol expect the MRC address while xtfs_mount expects the DIR
address. Did you pass the correct machine name (of the DIR) to xtfs_mount?

That should work. You also need to restart the MRC, OSDs to use the new
directory service.

>
> Then get the DIR service only (no MRC or OSD) running on the other
> directory server, start everything up, and that's when I start getting
> errors. I can still xtfs_lsvol storageserver.name and it shows my
> volume, but I can't mount it (#111 connection refused).

can you telnet to port 32638 (DIR port) on that machine?


Bj�rn

[Dennis McEntire]

I'm still trying to get this to work. Here's some info that I hope
will help figure out the problem:

1. Box #1 is called xdir1 and is ONLY running the DIR service. Telnet
into 32638 doesn't error out, but also doesn't show anything, but then
when I hit ENTER a couple times it loses it's connection and closes.
Java listeners on this box are listening on ports 32638 and 30638.

2. Box #2 is called xtreem1 and is running both MRC and OSD services.
Both configs are pointing to box #1 for directory services. Java
listeners on this box show up as 32640 and 30636.

On Box #2, the storage node, I type this command:

xtfs_mkvol xtreem1/xtreem (xtreem1 is the box name and xtreem is the
"share?" name)

Then, xtfs_lsvol xtreem1 shows the share available:

xtreem -> xxxx-xxxx-xxxxxx (UUID)

Going back to box #1, the directory server, I make a new directory
called /home/test

Then going to /home I type:

xtfs_mount xtreem1/xtreem test

I get a connection refused (errno = 111, strerror = Connection refused)

Again, this whole thing works when it's all on ONE box, I can mount
and use, etc.

Should I start over (erase everything, all dirs, etc) and remake and
make install again ?

Thanks for any help on this, I am sure it's something simple but it's
been pretty frustrating trying to get this working on more than one
box. The services still seem to be listening on the appropriate boxes
but it doesn't seem to work.

Thanks again for the help,

Dennis



2009/11/26 Björn Kolbeck <kol...@zib.de>:

> Björn

[Minor Gordon]

Hi,

It's the end of the day here here in Germany (where all the developers
are), but I'll try to answer this quickly. If you can try it and respond
again I'll check again in the morning.

Dennis McEntire wrote:
> I'm still trying to get this to work. Here's some info that I hope
> will help figure out the problem:
>
> 1. Box #1 is called xdir1 and is ONLY running the DIR service. Telnet
> into 32638 doesn't error out, but also doesn't show anything, but then
> when I hit ENTER a couple times it loses it's connection and closes.
> Java listeners on this box are listening on ports 32638 and 30638.
>
> 2. Box #2 is called xtreem1 and is running both MRC and OSD services.
> Both configs are pointing to box #1 for directory services. Java
> listeners on this box show up as 32640 and 30636.
>
> On Box #2, the storage node, I type this command:
>
> xtfs_mkvol xtreem1/xtreem (xtreem1 is the box name and xtreem is the
> "share?" name)
>
> Then, xtfs_lsvol xtreem1 shows the share available:
>
> xtreem -> xxxx-xxxx-xxxxxx (UUID)
>
> Going back to box #1, the directory server, I make a new directory
> called /home/test
>
> Then going to /home I type:
>
> xtfs_mount xtreem1/xtreem test

^

This should be xdir1/xtreem, not xtreem1. Volumes are mounted against
the DIR, not the MRC. A bit confusing, I know, since volumes are created
against the MRC. You're getting the connection refused because
xtfs_mount is trying to connect to the DIR port (32638).

> I get a connection refused (errno = 111, strerror = Connection refused)
>
> Again, this whole thing works when it's all on ONE box, I can mount
> and use, etc.

... because xtfs_mount is able to reach the DIR on that box.

Hope that helps. Let me know if it works.

Minor

[Dennis McEntire]

YES!!

Thank you it mounts and now I can push files into it. VERY
interesting. I am glad it was something simple, I will now try adding
more boxes to the mix and see how it goes.

Thanks again for your help,

Dennis

[Dennis McEntire]

On quick question, the system seems to work pretty well, but when
copying large files (250MB+) it generally starts the copying but quits
at random times with an error:

cp: writing `test\testfile250mb.mp4': Operation not permitted

When I view the mount directory the file is partially there, but is
not complete.

Dennis

[Minor Gordon]

Hi Dennis,

On Mon, Dec 7, 2009 at 11:58 PM, Dennis McEntire <dmce...@gmail.com> wrote:

On quick question, the system seems to work pretty well,

I'm glad to hear that.
 

but when
copying large files (250MB+) it generally starts the copying but quits
at random times with an error:

cp: writing `test\testfile250mb.mp4': Operation not permitted

When I view the mount directory the file is partially there, but is
not complete.

Are your client(s) and servers connected by a WAN?
We've had some problems with requests timing out on high-latency links. There's a fix for this in trunk that will be in the next release, which (we hope) will be ready on Friday.

Minor

[Dennis McEntire]

Actually they are on the same subnet, same network, connected via gigabit :-)

So I am not sure why this is happening. I will take a look to see if
there is anything causing any network latency between the boxes. I am
also going to add another storage node today to see how it works with
multiple storage boxes.

Just to be clear, on that box (the 2nd storage node) I am going to run
both MRC and OSD services, correct?

Thanks,

Dennis

[Minor Gordon]

Hi Dennis,

Dennis McEntire wrote:
> Actually they are on the same subnet, same network, connected via gigabit :-)

I see. We haven't actually been able to reproduce the request timeout
problem on Linux, only on Windows (where reads and writes are larger by
default). I'm curious as to why you're seeing this.

> So I am not sure why this is happening. I will take a look to see if
> there is anything causing any network latency between the boxes. I am
> also going to add another storage node today to see how it works with
> multiple storage boxes.

'Operation not permitted' can mean different things, not always EPERM.
I'd suggest you enable debugging in the servers (set debug.level = 6)
and in the client (-f -d INFO to run in the foreground and log at
LOG_INFO level). Then you'll see any abnormal errors. I'm assuming you
can always reproduce the problem with large copies. You usually won't
get an EPERM halfway through a copy if you're using 1.1. If you're using
trunk then you might get it. We recently added more checking of
permissions (capabilities, actually) during I/O. Are you using trunk or 1.1?

> Just to be clear, on that box (the 2nd storage node) I am going to run
> both MRC and OSD services, correct?

No. Only an OSD. The usual setup is to have a single DIR, a single MRC
at a central point and OSDs at each local site. The servers will
automatically direct client I/O to the OSD(s) that are closest to the
client (or at least that's what we're trying for). We're currently
working on replicated DIR and MRC services, but that's mostly for fault
tolerance and scaling large systems rather than decreasing I/O latency.

Minor

[Dennis McEntire]

Notes below

On Tue, Dec 8, 2009 at 9:05 AM, Minor Gordon
<minor....@googlemail.com> wrote:
> Hi Dennis,
>
> Dennis McEntire wrote:
>>
>> Actually they are on the same subnet, same network, connected via gigabit
>> :-)
>
> I see. We haven't actually been able to reproduce the request timeout
> problem on Linux, only on Windows (where reads and writes are larger by
> default). I'm curious as to why you're seeing this.
>
>> So I am not sure why this is happening. I will take a look to see if
>> there is anything causing any network latency between the boxes. I am
>> also going to add another storage node today to see how it works with
>> multiple storage boxes.
>
> 'Operation not permitted' can mean different things, not always EPERM.
> I'd suggest you enable debugging in the servers (set debug.level = 6) and in
> the client (-f -d INFO to run in the foreground and log at LOG_INFO level).
> Then you'll see any abnormal errors. I'm assuming you can always reproduce
> the problem with large copies. You usually won't get an EPERM halfway
> through a copy if you're using 1.1. If you're using trunk then you might get
> it. We recently added more checking of permissions (capabilities, actually)
> during I/O. Are you using trunk or 1.1?

I am at debugging level 7, so I'll take another look at the logs,
there was a lot of information there but nothing really popped up as a
cause of the timeout issues, I'll look more carefully. Also, I am
using xtreemfs-1.1.0 for this test.

>
>> Just to be clear, on that box (the 2nd storage node) I am going to run
>> both MRC and OSD services, correct?
>
> No. Only an OSD. The usual setup is to have a single DIR, a single MRC at a
> central point and OSDs at each local site. The servers will automatically
> direct client I/O to the OSD(s) that are closest to the client (or at least
> that's what we're trying for). We're currently working on replicated DIR and
> MRC services, but that's mostly for fault tolerance and scaling large
> systems rather than decreasing I/O latency.
>
> Minor
>

I'll move the MRC service daemon to the DIR box, and just run OSD
daemons on the storage nodes, correct?

Thanks again for your help,

Dennis

[Minor Gordon]

Hi Dennis,

Dennis McEntire wrote:
> I am at debugging level 7, so I'll take another look at the logs,
> there was a lot of information there but nothing really popped up as a
> cause of the timeout issues, I'll look more carefully. Also, I am
> using xtreemfs-1.1.0 for this test.

The server logs are a bit harder to correlate with client actions than
reading the client directly. I'd suggest piping an xtfs_mount in the
foreground (-f) to a file and looking at that. The client logs most
abnormal errors (e.g. everything besides errors generated by eager GUI
apps like the Windows explorer). You might also turn on --trace-file-io.
See -h for the various --trace-* operations.

We will be testing and trying to optimize large file copies again in the
next few days. Any feedback you can give us in the process would
certainly be appreciated.

> I'll move the MRC service daemon to the DIR box, and just run OSD
> daemons on the storage nodes, correct?

Yes.

Minor

[Dennis McEntire]

Just an update for the day,

I moved the MRC service over to the DIRectory server, so that box now
is running two daemons for xtreem. The other storage node is running
just the OSD daemon. With this setup, there are NO timeout errors and
the files copy just fine.

I'm going to test adding another storage node and will update with the results,

Dennis

[Dennis McEntire]

Couple more updates:

Things are working right now with the one storage node. When mounting
the share on a couple different computers, however, I have run into
some permission denied issues, but doing a chmod as the user who
dropped files in seems to fix those things.

One big question that comes up: How do we know when a file is done
copying into xtreem?

When using one computer to mount a share, we copy a large 1gig file to
the server.

When looking at the share from another computer, the file size just
slowly increases, showing that the file is copying into the system.
The filename is static, however.

So the question that arises, how can a "program" know when a file is
done copying (from a computer that is NOT the one doing the actual
copying)?

Sometimes, as with some FTP servers, files that are in the process of
being copied have a dot (.) in front of their name so they should not
appear in a list until the file is done uploading, then it gets
renamed to the actual file name.

This way, most people won't see the file if they do a standard LS
since it starts with a period. Once the file is done, the period
disappears and the file will show up for people doing an LS.

Hope that question makes sense.

Thanks,

Dennis

[Minor Gordon]

Hi Dennis,

The timeout errors may not have been timeout errors at all, but actual EPERMs. I'm curious as to why the errors (whatever they are) would go away when you co-located the DIR and MRC.

On Wed, Dec 9, 2009 at 3:13 AM, Dennis McEntire <dmce...@gmail.com> wrote:

Couple more updates:

Things are working right now with the one storage node. When mounting
the share on a couple different computers, however, I have run into
some permission denied issues, but doing a chmod as the user who
dropped files in seems to fix those things.

Are the users and groups the same between the machines?
The XtreemFS client uses a text representation of users and groups, obtained through a reverse-lookup of the integer uids and gids on the individual machines.
If you have a sequence of commands that produces unexpected/wrong permissions that would help us debug the problem.

 

One big question that comes up: How do we know when a file is done
copying into xtreem?

When using one computer to mount a share, we copy a large 1gig file to
the server.

When looking at the share from another computer, the file size just
slowly increases, showing that the file is copying into the system.
The filename is static, however.

So the question that arises, how can a "program" know when a file is
done copying (from a computer that is NOT the one doing the actual
copying)?

Sometimes, as with some FTP servers, files that are in the process of
being copied have a dot (.) in front of their name so they should not
appear in a list until the file is done uploading, then it gets
renamed to the actual file name.

This way, most people won't see the file if they do a standard LS
since it starts with a period. Once the file is done, the period
disappears and the file will show up for people doing an LS.

The hide-then-rename method you describe is probably the only surefire way. In the next XtreemFS release clients will cache file size updates locally by default -- continuous updates to the MRC have proved to be a bottleneck -- so a second client won't actually see the file growing (with an ls) until the copy is complete and the file is closed.

Minor

[Dennis McEntire]

Couple of random questions:

1. Replication does not seem to be working when adding a 2nd box, I am
assuming it's because the files on storage node 1 are NOT marked
read-only.

2. Trying to do a xtfs_repl --set_readonly filename gives me an error:
XtreemFS-tools jar could not be found!

Trying to find out how to install the tools leaves me puzzled, it
doesn't look like there is an installer for the tools? I found a jar
file in /usr/share/java called XtreemFS.jar - that's already there.

But the command gives that jar file missing error.

Any ideas?

Dennis

[Dennis McEntire]

In addition, it appears that maybe the tools jar is a copy of the main
one, so I made a symlink to it and called it XtreemFS-tools.jar and
it's in /usr/share/java.

Now the command runs, but I get a new error:

java.io.IOException: Cannot run program "getfattr":
java.io.IOException: error=2, No such file or directory

I have tried this with several different files and I get the same error.

Dennis

[Jan Stender]

Hi Dennis,

this is Jan. I'm working on the servers, and I'm currently in charge of
the Linux packaging.

> 1. Replication does not seem to be working when adding a 2nd box, I am
> assuming it's because the files on storage node 1 are NOT marked
> read-only.

Do you want to manually replicate a single file, or do you plan on
auto-replicating new files?

In the former case, you have to mark the file read-only before adding
replicas. In the latter case, this is done automatically. You can
trigger the auto-replication by setting the volume's replication factor
to the number of replicas n >= 2 (xtfs_repl --ocr_factor_set n).
However, this will only replicate files that will be created in the
future, not existing ones. If you need to specify which OSDs are
supposed to be used for new replicas, you can adjust the volume's OSD
selection policy (xtfs_repl --osp_set ...).

> 2. Trying to do a xtfs_repl --set_readonly filename gives me an error:
> XtreemFS-tools jar could not be found!

> Trying to find out how to install the tools leaves me puzzled, it
> doesn't look like there is an installer for the tools?

Which Linux distribution are you using? There's supposed to be an
additional package called 'xtreemfs-tools', from which the tools can be
installed. The 'xtfs_repl' script shouldn't be there unless you install
the 'xtreemfs-tools' package.

> I found a jar
> file in /usr/share/java called XtreemFS.jar - that's already there.

> In addition, it appears that maybe the tools jar is a copy of the main
> one, so I made a symlink to it and called it XtreemFS-tools.jar and
> it's in /usr/share/java.

True. We will change this with the upcoming release; there's going to be
another package containing the jar which both the servers and tools will
depend on, which will avoid the duplication of jars.

>
> Now the command runs, but I get a new error:
>
> java.io.IOException: Cannot run program "getfattr":
> java.io.IOException: error=2, No such file or directory
>
> I have tried this with several different files and I get the same error.

That's apparently because you don't have the 'attr' or 'libattr' package
installed on your system.

Hope this helps and best regards,
Jan

[Dennis McEntire]

Thank you for all your help. I was able to get the xtfs_tools going by
installing the attr package.

Now I have a new concern that's much more important.

For some reason, the system has become unstable and is now crashing
when doing various operations (like copying files into it).

The errors that show up are:

2009-12-10T13:43:29.000Z EMERG: xtfs_mount: crashed on unknown
exception, dumping to ./41e55070-5f37-5bb4-79b5a3bd-0e72052.dmp and
trying to send to
http://www.xtreemfs.org/dump/dump.php?f=41e55070-5f37-5bb4-79b5a3bd-0e72052.dmp

2009-12-10T13:43:34.000Z EMERG: xtfs_mount: exception trying to send
dump to the server: errno = 110, strerror = Connection timed out

It seems the only way to "clear" these errors is to RESTART the DIR
and MRC daemons.

Any thoughts on what could cause these problems?

Dennis

[Dennis McEntire]

Also, this is frustrating:

http://groups.google.com/group/xtreemfs/search?group=xtreemfs&q=crashed+on+unknown+exception&qt_g=Search+this+group

and

http://groups.google.com/group/xtreemfs/search?group=xtreemfs&q=EMERG&qt_g=Search+this+group

Both yield NO results. Am I not looking at the right place for technical issues?

Dennis

[Dennis McEntire]

Jan, just some quick answers to your questions. I downloaded and built
the source from:

http://www.xtreemfs.org/download.php?t=source

Once I did that, the xtfs_repl program was available, but the command
did not work until I did these:

cd /usr/share/java
ln -s XtreemFS.jar XtreemFS-tools.jar

I made a symbolic link to the original JAR file and then the commands worked.

Oh, yes, I had to install attr (on Mandriva, urpmi attr)

Also, regarding replication, I went to a box that has the xtreemfs
tools installed and a share mounted, and did this:

xtfs_repl --ocr_factor_set 2 test

Note that test is the mounted share. Did I do that part right? It
seemed to work, but I wanted to make sure that I am supposed to point
the command to the mounted folder, not the server(s) nor the share
name(s).


Thanks,

Dennis

[Dennis McEntire]

Additional information on the crashing issue:

I can consistently get the system to crash by trying to do anything to
a file that is ZERO size on disk.

So, this will crash the system:

1. cd /xtreem (the mount point)
2. touch testfile
3. rm testfile

That will generate the error. There seems to be other ways to generate
the error, but this particular method is pretty consistent. When the
error occurs, the way we fix it is to get out of the mounted dir, then
umount /dir and xtfs_mount the dir again and then it's ok. Funny
thing, the operation we were trying to do (like the rm) works anyway
and deletes the file.

Dennis

[Jan Stender]

Hi Dennis,

> Jan, just some quick answers to your questions. I downloaded and built
> the source from:
>
> http://www.xtreemfs.org/download.php?t=source

Ok.

>
> Once I did that, the xtfs_repl program was available, but the command
> did not work until I did these:
>
> cd /usr/share/java
> ln -s XtreemFS.jar XtreemFS-tools.jar
>
> I made a symbolic link to the original JAR file and then the commands worked.

Right, I got the problem - this is a bug in 1.1. It has been fixed in
1.2, which we are going to publish today. Sorry for the inconvenience.

> Also, regarding replication, I went to a box that has the xtreemfs
> tools installed and a share mounted, and did this:
>
> xtfs_repl --ocr_factor_set 2 test
>
> Note that test is the mounted share. Did I do that part right? It
> seemed to work, but I wanted to make sure that I am supposed to point
> the command to the mounted folder, not the server(s) nor the share
> name(s).

Yes. We were debating to let the tools communicate directly with the
services (i.e. via service UUIDs + volume names), but we came to the
conclusion that it's better to use mounts instead; this relieves us of
dealing with security and authentication in the tools.

Best regards,
Jan

[Björn Kolbeck]

Hi Dennis,

On 11.12.2009 01:07, Dennis McEntire wrote:
> Additional information on the crashing issue:
>
> I can consistently get the system to crash by trying to do anything to
> a file that is ZERO size on disk.
>
> So, this will crash the system:
>
> 1. cd /xtreem (the mount point)
> 2. touch testfile
> 3. rm testfile
>
> That will generate the error. There seems to be other ways to generate
> the error, but this particular method is pretty consistent. When the
> error occurs, the way we fix it is to get out of the mounted dir, then
> umount /dir and xtfs_mount the dir again and then it's ok. Funny
> thing, the operation we were trying to do (like the rm) works anyway
> and deletes the file.

I can reproduce this bug when I use a non-resolvable hostname for the
OSD. Please make sure the hostname which the OSD uses is correct. You
can see the mapping on the DIR status page in the "Address Mapping"
section. If the OSD does not detect its hostname correctly, you can set
it with 'hostname = my.machine.name.com' in the osdconfig.properties file.

We have fixed this bug in the upcoming release 1.2, the client now
reports a proper error message.

Bj�rn