Spatial search: FindNearby documents with WithinRadiusOf

[Olle]

I'm having a bit of a problem with the radius resolution.

What I can see in the radius should be defined in "miles". 

Hoping the test will speak for itself.

Thanks in advance.

[Test]

public void FindNearbyDocs()

{

// These items is in a radius of 4 miles (approx 6,5 km)

var areaOneDocOne = new DummyGeoDoc(55.6880508001, 13.5717346673);

var areaOneDocTwo = new DummyGeoDoc(55.6821978456, 13.6076183965);

var areaOneDocThree = new DummyGeoDoc(55.673251569, 13.5946697607);

// This item is 12 miles (approx 19 km) from the closests in areaOne 

var closeButOutsideAreaOne = new DummyGeoDoc(55.8634157297, 13.5497731987);

// This item is about 3900 miles from areaOne

var newYork = new DummyGeoDoc(40.7137578228, -74.0126901936);

var session = Store.OpenSession();

session.Store(areaOneDocOne);

session.Store(areaOneDocTwo);

session.Store(areaOneDocThree);

session.Store(closeButOutsideAreaOne);

session.Store(newYork);

session.SaveChanges();

var indexDefinition = new IndexDefinition

{

Map = "from doc in docs select new { _ = SpatialIndex.Generate(doc.Latitude, doc.Longitude) }"

};

Store.DatabaseCommands.PutIndex("FindByLatLng", indexDefinition);

// Wait until the index is built

session.Advanced.LuceneQuery<DummyGeoDoc>("FindByLatLng")

.WaitForNonStaleResults()

.ToArray();

const double lat = 55.6836422426, lng = 13.5871808352; // in the middle of AreaOne

const double radius = 5.0;

// Radius >= 14.4 return 4 results

// Radius >= 14.3 return 0 results

// Expcted is that 5.0 will return 3 results

var nearbyDocs = session.Advanced.LuceneQuery<DummyGeoDoc>("FindByLatLng")

.WithinRadiusOf(radius, lat, lng)

.WaitForNonStaleResults()

.ToArray();

Assert.That(nearbyDocs != null);

Assert.That(nearbyDocs.Length == 3);

session.Dispose();

}

public class DummyGeoDoc

{

public string Id { get; set; }

public double Latitude { get; set; }

public double Longitude { get; set; }

public DummyGeoDoc(double lat, double lng)

{

this.Latitude = lat;

this.Longitude = lng;

}

}

[Matt Warren]

I think this it is hitting some kind of weird edge case or bug with
the Lucene Spatial search.

If you change the center point to from

lat = 55.6836422426, lng = 13.5871808352

to
lat = 55.6836422426, lng = 13.6271808352

Then you get all 3 matches as expected, I'll take more of a look?!

[Matt Warren]

I forgot to add that I see the same behavior when using Lucene and
Spatial.NET directly, so it's not just a Raven problem.

[Olle]

Thanks for taking your time.

I've can now also confirm that my test didn't pass in Spatial.Net either.

Seems we have the same error here http://stackoverflow.com/questions/5642435/inconsistent-search-results-using-spatial-net-and-lucene-net-2-9

I've tried running my tests from the lucene.net trunk (2.9.4) (https://svn.apache.org/repos/asf/incubator/lucene.net/trunk) with the same error. 

Issue posted to the lucene tracker. Will update this post when I know more.

[Ayende Rahien]

Thanks, as soon as it is fixed there, we would pick it up.

[Matt Warren]

I've applied the patch from here https://issues.apache.org/jira/secure/attachment/12420781/LUCENE-1930.patch
which is part of this issue https://issues.apache.org/jira/browse/LUCENE-1930.
I did it against the RavenDB version of Spatial.NET and it works.

So I guess we just need to wait until Lucene.NET/Spatial.NET applies
this patch. If I've got time I'll try and submit something over the
next few weeks.

On Jul 5, 11:17 pm, Olle <o...@brightcabin.se> wrote:
> Thanks for taking your time.
>
> I've can now also confirm that my test didn't pass in Spatial.Net either.
>

> Seems we have the same error here http://stackoverflow.com/questions/5642435/inconsistent-search-result...

[Ayende Rahien]

Hm, I have no real problem is getting fixes before they do, as long as we have a test case that ensures that we won't miss out.

[Olle]

This is great news. 

I tried to narrow down the bug during my lunch time but didn't have the time to complete it. 

[Olle]

The same patch is now also included in the 2.9.4g version of Lucene.net.

https://issues.apache.org/jira/browse/LUCENENET-431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

[Ayende Rahien]

Matt,

Would you be able to check this and maybe generate a pull request?

[Matt Warren]

Yeah I'd be happy to.

But one question, do you know if it is possible to generate multiple
Forks of the RavenDB code? I'm currently playing around with adding
FacetedSearch to Raven (see https://github.com/mattwarren/ravendb-FacetedSearch)
and I don't want to add the Spatial Fix to that.

When I tried before GitHib won't let you create multiple forks, or am
I doing it wrong? Is my only option to create another GitHub accound
and fork/branch using that?

On Jul 8, 4:45 pm, Ayende Rahien <aye...@ayende.com> wrote:
> Matt,
> Would you be able to check this and maybe generate a pull request?
>

> On Wed, Jul 6, 2011 at 8:52 PM, Olle <o...@brightcabin.se> wrote:
> > The same patch is now also included in the 2.9.4g version of Lucene.net.
>

> >https://issues.apache.org/jira/browse/LUCENENET-431?page=com.atlassia...

[Ayende Rahien]

I don't know about multiple forks, but I can do multiple branches

I usually have different directory on the disk for every different branch that i routinely work with

[Itamar Syn-Hershko]

Matt, do this:

1) Make a new, local fork for RavenDB on temp location on your HD

2) Create a new branch from master, call it "faceted"

3) Add your Faceted repo as remote named "faceted"

4) Do fetch "faceted"

5) Merge your the master branch of remote "faceted" with branch faceted

6) In github, delete Faceted repo

7) Fork ayende/ravendb, you'll get a fresh fork you can work with

8) Add that fork as remote "origin", or "mine"

9) Push "faceted" branch to origin

[Matt Warren]

Thanks for the info, I'll give it a try later on today.

BTW would either of you have some time to take a look at the approach
I'm taking with Faceted search?

The details of how I'm doing it and what the API would look like is
here https://github.com/mattwarren/ravendb-FacetedSearch/blob/master/readme.txt.

It's based around this method
http://www.devatwork.nl/articles/lucenenet/faceted-search-and-drill-down-lucenenet/
But it has as much work as possible done at indexing time rather than
query time.

Please bear in mind that I haven't finished it yet!! The code is a bit
messy.

Cheers

On Jul 13, 11:23 am, Itamar Syn-Hershko <ita...@hibernatingrhinos.com>
wrote:

[Itamar Syn-Hershko]

I'll take a look there when I can.

[Matt Warren]

Thanks, I appreciate that.

On Jul 13, 1:13 pm, Itamar Syn-Hershko <ita...@hibernatingrhinos.com>
wrote:

> I'll take a look there when I can.
>

> On Wed, Jul 13, 2011 at 2:32 PM, Matt Warren <mattd...@gmail.com> wrote:
> > Thanks for the info, I'll give it a try later on today.
>
> > BTW would either of you have some time to take a look at the approach
> > I'm taking with Faceted search?
>
> > The details of how I'm doing it and what the API would look like is
> > here

> >https://github.com/mattwarren/ravendb-FacetedSearch/blob/master/readm...
> > .

>
> > It's based around this method
>

> >http://www.devatwork.nl/articles/lucenenet/faceted-search-and-drill-d...

[Olle]

Are there any news on the Spatial issue?

[Ayende Rahien]

I think that we pushed that to the unstable branch

[Jake Scott]

Is this fixed now?

[Matt Warren]

Yeah these changes were pulled in a while ago. If the changes don't
fix your issue just post a failing test on the mailing list and I'll
take a look.

On Sep 28, 6:12 pm, Jake Scott <jake....@gmail.com> wrote:
> Is this fixed now?

[Olle]

I can confirm that all of my previous red tests got green after the fix Matt applied.

[Matt Warren]

Glad it works.

Thanks for doing the tests and reporting back.