Issue when querying for a value that contains an ampersand

ma...@aveni.us

unread,

Oct 28, 2013, 11:06:36 PM10/28/13

to rav...@googlegroups.com

Hey all; new here, and just getting into Raven, so be gentle ;-)

I think I have most of querying / storing / patching figured out, but when I threw a few tens of thousands of records into my dev db and queried against it, I found what I believe to be a bug (please let me know if it is not.)

I am running into this when querying for a value that contains an ampersand. It looks like the parser is treating the ampersand as a non-escapable character (the query is Sku:*\|CNVAIS\-P\&H' but it is parsed as Sku:*\|CNVAIS\-P\')

Here are the gory details:

2013-10-28 12:49:38,326 [3] DEBUG Raven.Client.Document.SessionOperations.QueryOperation [(null)] Executing query 'Vendor:STC AND Sku:*\|CNVAIS\-P\&H' on index 'dynamic/Products' in 'http://localhost:4000 (DB: ProductManagementDEV);ProductManagementDEV'

2013-10-28 12:49:38,334 [3] ERROR Avenius.DataFeed.ConsoleApp.Program [(null)] - Exception:

System.InvalidOperationException: Url: "/indexes/dynamic/Products?&query=Vendor%3ASTC%20AND%20Sku%3A*%5C%7CCNVAIS%5C-P%5C%26H&pageSize=1&cutOff=2013-10-28T16%3A49%3A38.3268847Z"

Lucene.Net.QueryParsers.ParseException: Could not parse: 'Vendor:STC AND Sku:*\|CNVAIS\-P\' ---> Lucene.Net.QueryParsers.ParseException: Cannot parse 'Vendor:STC AND Sku:*\|CNVAIS\-P\': Lexical error at line 1, column 33. Encountered: <EOF> after : "" ---> Lucene.Net.QueryParsers.TokenMgrError: Lexical error at line 1, column 33. Encountered: <EOF> after : ""

at Lucene.Net.QueryParsers.QueryParserTokenManager.GetNextToken() in c:\Work\lucene.net\src\core\QueryParser\QueryParserTokenManager.cs:line 1386

at Lucene.Net.QueryParsers.QueryParser.Jj_ntk() in c:\Work\lucene.net\src\core\QueryParser\QueryParser.cs:line 1929

at Lucene.Net.QueryParsers.QueryParser.Term(String field) in c:\Work\lucene.net\src\core\QueryParser\QueryParser.cs:line 1461

at Lucene.Net.QueryParsers.QueryParser.Clause(String field) in c:\Work\lucene.net\src\core\QueryParser\QueryParser.cs:line 1383

at Lucene.Net.QueryParsers.QueryParser.Query(String field) in c:\Work\lucene.net\src\core\QueryParser\QueryParser.cs:line 1331

at Lucene.Net.QueryParsers.QueryParser.Parse(String query) in c:\Work\lucene.net\src\core\QueryParser\QueryParser.cs:line 224

--- End of inner exception stack trace ---

at Lucene.Net.QueryParsers.QueryParser.Parse(String query) in c:\Work\lucene.net\src\core\QueryParser\QueryParser.cs:line 239

at Raven.Database.Indexing.QueryBuilder.BuildQuery(String query, IndexQuery indexQuery, RavenPerFieldAnalyzerWrapper analyzer) in c:\Builds\RavenDB-Stable\Raven.Database\Indexing\QueryBuilder.cs:line 57

--- End of inner exception stack trace ---

at Raven.Database.Indexing.QueryBuilder.BuildQuery(String query, IndexQuery indexQuery, RavenPerFieldAnalyzerWrapper analyzer) in c:\Builds\RavenDB-Stable\Raven.Database\Indexing\QueryBuilder.cs:line 64

at Raven.Database.Indexing.SimpleQueryParser.GetFieldsForDynamicQuery(IndexQuery query) in c:\Builds\RavenDB-Stable\Raven.Database\Indexing\SimpleQueryParser.cs:line 155

at Raven.Database.Queries.DynamicQueryOptimizer.SelectAppropriateIndex(String entityName, IndexQuery indexQuery, List`1 explanations) in c:\Builds\RavenDB-Stable\Raven.Database\Queries\DynamicQueryOptimizer.cs:line 50

at Raven.Database.Server.Responders.Index.GetDynamicIndexName(String index, IndexQuery indexQuery, String& entityName) in c:\Builds\RavenDB-Stable\Raven.Database\Server\Responders\Index.cs:line 555

at Raven.Database.Server.Responders.Index.PerformQueryAgainstDynamicIndex(IHttpContext context, String index, IndexQuery indexQuery, Guid& indexEtag) in c:\Builds\RavenDB-Stable\Raven.Database\Server\Responders\Index.cs:line 507

at Raven.Database.Server.Responders.Index.ExecuteQuery(IHttpContext context, String index, Guid& indexEtag) in c:\Builds\RavenDB-Stable\Raven.Database\Server\Responders\Index.cs:line 440

at Raven.Database.Server.Responders.Index.GetIndexQueryResult(IHttpContext context, String index) in c:\Builds\RavenDB-Stable\Raven.Database\Server\Responders\Index.cs:line 375

at Raven.Database.Server.HttpServer.DispatchRequest(IHttpContext ctx) in c:\Builds\RavenDB-Stable\Raven.Database\Server\HttpServer.cs:line 864

at Raven.Database.Server.HttpServer.HandleActualRequest(IHttpContext ctx) in c:\Builds\RavenDB-Stable\Raven.Database\Server\HttpServer.cs:line 609

---> System.Net.WebException: The remote server returned an error: (500) Internal Server Error.

at System.Net.HttpWebRequest.GetResponse()

at Raven.Client.Connection.HttpJsonRequest.ReadJsonInternal(Func`1 getResponse) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\HttpJsonRequest.cs:line 340

--- End of inner exception stack trace ---

at Raven.Client.Connection.HttpJsonRequest.HandleErrors(WebException e) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\HttpJsonRequest.cs:line 532

at Raven.Client.Connection.HttpJsonRequest.ReadJsonInternal(Func`1 getResponse) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\HttpJsonRequest.cs:line 346

at Raven.Client.Connection.HttpJsonRequest.ReadResponseJson() in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\HttpJsonRequest.cs:line 230

at Raven.Client.Connection.ServerClient.DirectQuery(String index, IndexQuery query, String operationUrl, String[] includes, Boolean metadataOnly, Boolean includeEntries) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\ServerClient.cs:line 1209

at Raven.Client.Connection.ServerClient.<>c__DisplayClass62.<Query>b__61(String u) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\ServerClient.cs:line 1078

at Raven.Client.Connection.ReplicationInformer.TryOperation[T](Func`2 operation, String operationUrl, Boolean avoidThrowing, T& result, Boolean& wasTimeout) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\ReplicationInformer.cs:line 494

at Raven.Client.Connection.ReplicationInformer.ExecuteWithReplication[T](String method, String primaryUrl, Int32 currentRequest, Int32 currentReadStripingBase, Func`2 operation) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\ReplicationInformer.cs:line 455

at Raven.Client.Connection.ServerClient.ExecuteWithReplication[T](String method, Func`2 operation) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\ServerClient.cs:line 174

at Raven.Client.Connection.ServerClient.Query(String index, IndexQuery query, String[] includes, Boolean metadataOnly, Boolean indexEntriesOnly) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Connection\ServerClient.cs:line 1078

at Raven.Client.Document.AbstractDocumentQuery`2.ExecuteActualQuery() in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Document\AbstractDocumentQuery.cs:line 663

at Raven.Client.Document.AbstractDocumentQuery`2.InitSync() in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Document\AbstractDocumentQuery.cs:line 645

at Raven.Client.Document.AbstractDocumentQuery`2.GetEnumerator() in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Document\AbstractDocumentQuery.cs:line 881

at System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable`1 source)

at Raven.Client.Linq.RavenQueryProviderProcessor`1.GetQueryResult[TProjection](IDocumentQuery`1 finalQuery) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Linq\RavenQueryProviderProcessor.cs:line 1572

at Raven.Client.Linq.RavenQueryProviderProcessor`1.ExecuteQuery[TProjection]() in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Linq\RavenQueryProviderProcessor.cs:line 1497

at Raven.Client.Linq.RavenQueryProviderProcessor`1.Execute(Expression expression) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Linq\RavenQueryProviderProcessor.cs:line 1470

at Raven.Client.Linq.RavenQueryProvider`1.Execute(Expression expression) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Linq\RavenQueryProvider.cs:line 155

at Raven.Client.Linq.RavenQueryProvider`1.System.Linq.IQueryProvider.Execute[S](Expression expression) in c:\Builds\RavenDB-Stable\Raven.Client.Lightweight\Linq\RavenQueryProvider.cs:line 198

at System.Linq.Queryable.FirstOrDefault[TSource](IQueryable`1 source, Expression`1 predicate)

at Avenius.ProductManagement.DataAccess.ProductDataAccess.<>c__DisplayClass8.<GetBySku>b__7(IDocumentSession session) in s:\Dev\Avenius.Products.DataAccess\DataAccess\ProductDataAccess.cs:line 51

at Avenius.RavenDataAccess.BaseRavenDataAccess`2.<>c__DisplayClass4`1.<PerformDbGetCommand>b__3(IDocumentSession session) in s:\Dev\Avenius.RavenDataAccess\BaseRavenDataAccess.cs:line 40

at Avenius.RavenDataAccess.BaseRavenDataAccess`2.PerformDbCommand(Action`1 action) in s:\Dev\Avenius.RavenDataAccess\BaseRavenDataAccess.cs:line 22

at Avenius.RavenDataAccess.BaseRavenDataAccess`2.PerformDbGetCommand[TReturnType](Func`2 action) in s:\Dev\Avenius.RavenDataAccess\BaseRavenDataAccess.cs:line 38

at Avenius.ProductManagement.DataAccess.ProductDataAccess.GetBySku(String vendorName, String sku, Boolean partialMatch) in s:\Dev\Avenius.Products.DataAccess\DataAccess\ProductDataAccess.cs:line 45

at Avenius.STC.Feed.Feeds.Incoming.DiscontinuedProductFeed.<ProcessImpl>b__0(CsvReader csv) in s:\Dev\Avenius.STC.Feed\Feeds\Incoming\DiscontinuedProductFeed.cs:line 22

at Avenius.STC.Feed.Feeds.Incoming.BaseSTCCSVFeed.HandleFile(String webPath, Boolean fileContainsHeaders, Action`1 csvHandler) in s:\Dev\Avenius.STC.Feed\Feeds\Incoming\BaseSTCCSVFeed.cs:line 24

at Avenius.STC.Feed.Feeds.Incoming.DiscontinuedProductFeed.ProcessImpl() in s:\Dev\Avenius.STC.Feed\Feeds\Incoming\DiscontinuedProductFeed.cs:line 18

at Avenius.STC.Feed.Feeds.BaseSTCDataFeed.Process() in s:\Dev\Avenius.STC.Feed\Feeds\BaseSTCDataFeed.cs:line 18

at Avenius.DataFeed.ConsoleApp.Program.DoWork(List`1 feedsToProcess) in s:\Dev\Avenius.DataFeed.ConsoleApp\Program.cs:line 163

Please let me know if I am missing something simple.

Thanks!

Mark

Oren Eini (Ayende Rahien)

unread,

Oct 29, 2013, 12:48:26 AM10/29/13

to ravendb

Huh!

How did you generate this query?

Oren Eini

CEO

Hibernating Rhinos

Cellular: +972-52-548-6969

Office: +972-4-674-7811

Fax: +972-153-4622-7811

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

ma...@aveni.us

unread,

Oct 29, 2013, 9:33:58 AM10/29/13

to rav...@googlegroups.com

Oren-

Here is my method for generating the query:

        public Product GetBySku(string vendorName, string sku, bool partialMatch = false)
        {
            return PerformDbGetCommand(session =>
            {
                var query = from p in session.Query<Product>()
                            where p.Vendor == vendorName
                            select p;


                if (partialMatch) return query.FirstOrDefault(p => p.Sku.EndsWith(string.Concat("|", sku)));
                else return query.FirstOrDefault(p => p.Sku == sku);
            });
        }

In this case, partialMatch is true.

To fill in more details, here are two more methods in play:

        protected TReturnType PerformDbGetCommand<TReturnType>(Func<IDocumentSession, TReturnType> action)
        {
            TReturnType result = default(TReturnType);
            PerformDbCommand(session =>
            {
                result = action(session);
            });
            return result;
        }


        protected void PerformDbCommand(Action<IDocumentSession> action)
        {
            using (var session = DocumentStore.OpenSession())
            {
                action.Invoke(session);
            }
        }