create_open and ZoneLocal in 2.55

[Planck]

Hello,
i found some probelm with ZoneLocal plugin and 2.55 version. I have
configured two zones (center1,center2). Get_paths work OK and gives
paths from proper zone but create_open command gives paths to storage
in second zone. Anybody have also this problem ?
Regards

[dormando]

Is this an existing setup, or a test setup?

If production any chance you could use a dev host, make a test setup ie
the quickstart guide from www.mogilefs.org, and modify it to match your
production setup as closely as you can?

Someone else reported this a while back, but I wasn't able to reproduce
the issue with create_open.

Also, you should consider an upgrade (or a downgrade to 2.54), 2.55 has a
bug which hangs replications/etc until all trackers have been restarted :(

[Planck]

On Feb 8, 12:35 am, dormando <dorma...@rydia.net> wrote:
> > Hello,
> > i found some probelm with ZoneLocal plugin and 2.55 version. I have
> > configured two zones (center1,center2). Get_paths work OK and gives
> > paths from proper zone but create_open command gives paths to storage
> > in second zone. Anybody have also this problem ?
> > Regards
>
> Is this an existing setup, or a test setup?

This is existing production setup with above hundred devices (maybe
problem is connected with that?). It looks like always will be chosed
device with lower disk space utilization.

>
> If production any chance you could use a dev host, make a test setup ie

> the quickstart guide fromwww.mogilefs.org, and modify it to match your

> production setup as closely as you can?

I will try to reproduce that on test installation and give some
results.

>
> Someone else reported this a while back, but I wasn't able to reproduce
> the issue with create_open.
>
> Also, you should consider an upgrade (or a downgrade to 2.54), 2.55 has a
> bug which hangs replications/etc until all trackers have been restarted :(

For now it's stable but of course upgrade to 2.57 will be on days :)

[dormando]

> This is existing production setup with above hundred devices (maybe
> problem is connected with that?). It looks like always will be chosed
> device with lower disk space utilization.

You may be right on that; at some point I changed the filtering.

I see in ZoneLocal:

@$sorted_devs = prioritize_devs_current_zone(
$MogileFS::REQ_client_ip,

MogileFS::Worker::Query::sort_devs_by_freespace(@$devices)
);

sort_devs_by_freespace was running very slowly with large lists, because
of weighted_list, so it only looks at the first 20.

Which is then *later* sorted by current zone.

You can test a fix by finding:
my @list =
MogileFS::Util::weighted_list(splice(@devices_with_weights, 0,
20));

in Worker/Query.pm, and changing that "20" to 100 or 200, then restarting
the tracker.

So that's probably it. Need to think a bit on how to fix it though; we
probably want to do the filtering a little more intelligently... First
filter by writeable devices (not full/dead/etc), then split by zone, and
only use @other_zone if there're no local devs. Then apply the rest of the
filtering afterwards.

Would you mind trucking over to google code and opening a bug report with
basically a copy/paste of this? If you'd be so kind as to "Confirm" the
issue by making the above change on one of your trackers and seeing if
that works first.

thanks,
-Dormando