Lucene syntax error in queries with accents after upgrade to 3.5

[Andrej Krivulčík]

After upgrading to 3.5.0-rc-35173 (both server and client), we are experiencing issues with queries containing accents. I suspect an issue with encoding detection/setting, but can't find documentation about how to specify connection encoding.

The query is the following:

((Name:žiž^5 OR Name:žiž*^2))

It fails from within the client code (web application) with the following:

[ErrorResponseException: Url: "/databases/data/streams/query/OsobaViewIndex?&query=((Name:%C5%BEi%C5%BE%5E5%20OR%20Name:%C5%BEi%C5%BE*%5E2))&sort=Temp-Index-Score"

Lucene.Net.QueryParsers.ParseException: Could not parse: '((Name:žiž^5 OR Name:žiž*^2))' ---> Lucene.Net.QueryParsers.ParseException: Syntax error, unexpected TILDA

   at Raven.Database.Indexing.LuceneQueryScanner.yyerror(String format, Object[] args) in C:\Builds\RavenDB-3.5-RC\Raven.Database\Indexing\LuceneQuery.Scanner.cs:line 83

   at QUT.Gppg.ShiftReduceParser`2.ReportError() in C:\Builds\RavenDB-3.5-RC\Raven.Database\ShiftReduceParserCode.cs:line 413

   at QUT.Gppg.ShiftReduceParser`2.ErrorRecovery() in C:\Builds\RavenDB-3.5-RC\Raven.Database\ShiftReduceParserCode.cs:line 380

   at QUT.Gppg.ShiftReduceParser`2.Parse() in C:\Builds\RavenDB-3.5-RC\Raven.Database\ShiftReduceParserCode.cs:line 279

   at Raven.Database.Indexing.QueryBuilder.BuildQuery(String query, IndexQuery indexQuery, RavenPerFieldAnalyzerWrapper analyzer) in C:\Builds\RavenDB-3.5-RC\Raven.Database\Indexing\QueryBuilder.cs:line 73

   --- End of inner exception stack trace ---

   at Raven.Database.Indexing.QueryBuilder.BuildQuery(String query, IndexQuery indexQuery, RavenPerFieldAnalyzerWrapper analyzer) in C:\Builds\RavenDB-3.5-RC\Raven.Database\Indexing\QueryBuilder.cs:line 108

   at Raven.Database.Indexing.Index.AssertQueryDoesNotContainFieldsThatAreNotIndexed(IndexQuery indexQuery, AbstractViewGenerator viewGenerator) in C:\Builds\RavenDB-3.5-RC\Raven.Database\Indexing\Index.cs:line 1248

   at Raven.Database.Indexing.Index.IndexQueryOperation.<Query>d__17.MoveNext() in C:\Builds\RavenDB-3.5-RC\Raven.Database\Indexing\Index.cs:line 1384

   at Raven.Database.Util.ActiveEnumerable`1..ctor(IEnumerable`1 enumerable) in C:\Builds\RavenDB-3.5-RC\Raven.Database\Util\ActiveEnumerable.cs:line 16

   at Raven.Database.Actions.QueryActions.DatabaseQueryOperation.Init() in C:\Builds\RavenDB-3.5-RC\Raven.Database\Actions\QueryActions.cs:line 257

   at Raven.Database.Server.Controllers.StreamsController.SteamQueryGet(String id) in C:\Builds\RavenDB-3.5-RC\Raven.Database\Server\Controllers\StreamsController.cs:line 200

   at lambda_method(Closure , Object , Object[] )

   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ActionExecutor.<>c__DisplayClass10.<GetExecutor>b__9(Object instance, Object[] methodParameters)

   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ExecuteAsync(HttpControllerContext controllerContext, IDictionary`2 arguments, CancellationToken cancellationToken)

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at System.Web.Http.Controllers.ApiControllerActionInvoker.<InvokeActionAsyncCore>d__0.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()

   at System.Web.Http.Controllers.ActionFilterResult.<ExecuteAsync>d__2.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()

   at System.Web.Http.Controllers.ExceptionFilterResult.<ExecuteAsync>d__0.MoveNext()]

   Raven.Client.Connection.<TryOperationAsync>d__34`1.MoveNext() in C:\Builds\RavenDB-3.5-RC\Raven.Client.Lightweight\Connection\ReplicationInformerBase.cs:442

   System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) +92

   System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +58

   System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task) +25

   Raven.Client.Connection.<ExecuteWithReplicationAsync>d__33`1.MoveNext() in C:\Builds\RavenDB-3.5-RC\Raven.Client.Lightweight\Connection\ReplicationInformerBase.cs:314

   System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) +92

   System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +58

   Raven.Client.Connection.Async.<ExecuteWithReplication>d__163`1.MoveNext() in C:\Builds\RavenDB-3.5-RC\Raven.Client.Lightweight\Connection\Async\AsyncServerClient.cs:0

   System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) +92

   System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +58

   Raven.Abstractions.Util.<<RunSync>b__0>d.MoveNext() in C:\Builds\RavenDB-3.5-RC\Raven.Abstractions\Util\AsyncHelpers.cs:75

   System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +26

   Raven.Abstractions.Util.AsyncHelpers.RunSync(Func`1 task) in C:\Builds\RavenDB-3.5-RC\Raven.Abstractions\Util\AsyncHelpers.cs:89

   Raven.Client.Connection.ServerClient.StreamQuery(String index, IndexQuery query, QueryHeaderInformation& queryHeaderInfo) in C:\Builds\RavenDB-3.5-RC\Raven.Client.Lightweight\Connection\ServerClient.cs:303

   Raven.Client.Document.DocumentSession.Stream(IDocumentQuery`1 query, QueryHeaderInformation& queryHeaderInformation) in C:\Builds\RavenDB-3.5-RC\Raven.Client.Lightweight\Document\DocumentSession.cs:636

   Raven.Client.Document.DocumentSession.Stream(IDocumentQuery`1 query) in C:\Builds\RavenDB-3.5-RC\Raven.Client.Lightweight\Document\DocumentSession.cs:626

... our code

When accessing the URL /databases/data/streams/query/OsobaViewIndex?&query=((Name:žiž%5E5%20OR%20Name:žiž*%5E2))&sort=Temp-Index-Score using a browser, a correct response is returned: {"Results":[]}. It works correctly also when using Studio.

Using the queries:

• ((Name:ži^5 OR Name:ži*^2))
  
• ((Name:iž^5 OR Name:iž*^2))
  

works correctly from the client code.

From the syntax error (unexpected TILDA, while no tilde is present in the query), the fact that the accented character works sometimes, and the fact that the same query works from studio and direct browser request, I suspect that the issue is related to the client connection encoding setting. Any suggestions?

[Oren Eini (Ayende Rahien)]

That is a bug that was corrected in the recent build.

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andrej Krivulčík]

I can confirm that this doesn't occur in the latest build. Thanks.

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.