hi

[Kant Kodali]

Hi All,

Say I issue a query that requires that doesnt contain the rowkey I am looking for Cassandra will return 0 results but in paginated case I want to see how can I differentiate between this case and the case where it is the last page?

I am trying to avoid "IF EXISTS AND IF NOT EXISTS queries" since those require consensus.

Another way to say this would be, Say I have code for paginated queries and I want to differentiate between the following two cases.

1. The Row key that I am looking for doesnt exists so cassandra returns 0 results

2. the row key exists but now it is the last page so cassandra where cassandra return 0 (no more results available)

what does isExhausted() will do in case 1?

Thanks!

[Andrew Tolbert]

Hi Kant,

1. The Row key that I am looking for doesnt exists so cassandra returns 0 results

2. the row key exists but now it is the last page so cassandra where cassandra return 0 (no more results available)

what does isExhausted() will do in case 1?

isExhausted will return true in both case 1 and 2.

In case #2, couldn't you check isExhausted after you get the ResultSet back initially or check if you've iterated over any rows?   Since you control iterating over the rows, shouldn't you be able to discern whether or not any rows were returned or is there something I'm missing?

Thanks,

Andy

[Kant Kodali]

Hi Andy,

checking to see if I've iterated over any rows doesn't necessarily differentiate between the two cases right because say after I finish fetching the last page which involves sending the one before the last page context. Now, I go back again but this time with the final page context that is available. Now, I have iterated over zero rows but doesnt mean that the row key doesn't exists so I failed to differentiate in the two cases. if isExhausted() returns true in both cases that it isn't helpful right. any other methods in ResultSet or somewhere where I can differentiate the two cases? I open to changing the logic that I control as long as I can differentiate the two cases.

Thanks,

Kant

  

--
You received this message because you are subscribed to the Google Groups "DataStax Java Driver for Apache Cassandra User Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to java-driver-us...@lists.datastax.com.

[Andrew Tolbert]

if isExhausted() returns true in both cases that it isn't helpful right. any other methods in ResultSet or somewhere where I can differentiate the two cases? I open to changing the logic that I control as long as I can differentiate the two cases.

isExhausted isn't really meant for this purpose though.  It's for indicating if there are any rows left to consumed, not if there were ever any rows returned. 

What could you could do to distinguish the two cases is use ResultSet.getAllExecutionInfo().size() > 1, which is sort of an indirect way of determining whether or not multiple pages were fetched.  However, if you've already iterated over all rows in a result that fits in one page, don't you still have a problem?

Thanks,

Andy

[Kant Kodali]

That might help but I need little bit more information. I perhaps need to give you more context. Every http request that I get in my microservice I fetch one page by doing while(ResultSet.getAvailableWithoutFetching() > 0) so the resultsets are just local variables to my method (so resultsets are destroyed once the method call returns). I can make it global however I am not sure If I am required to do that?

Thanks

[Andrew Tolbert]

Every http request that I get in my microservice I fetch one page by doing...

Is there:
 

a) a new ResultSet per HTTP request, 

b) the same ResultSet reused for each request?  

I'm thinking a) since you mention '...resultsets are destroyed once the method call returns', but thought i'd verify.

If a, you are making the query in your method, so you should be able to use isExhausted before iterating to determine if there were any results, right?

If b, how do you correlate the ResultSet with the request being made?   Have you considered using pagingState instead? (see: Saving and reusing the paging state)  If you use this you get a few benefits:

1) You only need to maintain the paging state between requests which is a small hash that can be returned in a response header/payload and sent with the next request.

2) If a request is provided with paging state, you know it was created from a previous query, therefore you know that there was likely a row returned in a previous query.

If you are already reusing the paging state, shouldn't following the strategy i describe in #2 work?

Thanks,

Andy

[Kant Kodali]

Hi Andy,

Here is the code and yes I do use paginated state. Also I thought you said isExhausted is not meant to check if there are results but now you are stating the contrary?

public void loadSeriesInOrder(ByteBuffer byteBuffer, ByteBuffer pageContext, int fetchSize, boolean reverseOrder, AsyncMethodCallback asyncMethodCallback) throws TException {
    ListenableFuture<ResultSet> paginatedForwardReferenceSeriesFuture = new ForwardReferenceSeries().getForwardRefernceSeriesAsync(byteBuffer, pageContext, fetchSize, reverseOrder);
    ListenableFuture<LoadSeriesResponse> loadSeriesResponseFuture = Futures.transform(paginatedForwardReferenceSeriesFuture, new Function<ResultSet, LoadSeriesResponse>() {
        public LoadSeriesResponse apply(ResultSet forwardReferenceSeriesResultSet) {
            //Mapper<ForwardReferenceSeries> mapperForwardReferenceSeries = manager.mapper(ForwardReferenceSeries.class);
            Mapper<ForwardReferenceSeries> mapperForwardReferenceSeries = forwardReferenceSeriesMapper;
            Result<ForwardReferenceSeries> forwardReferenceSeriesPage = mapperForwardReferenceSeries.map(forwardReferenceSeriesResultSet);
            List<ByteBuffer> forwardReferenceSeriesValues = new LinkedList<>();
            while(forwardReferenceSeriesResultSet.getAvailableWithoutFetching() > 0) { //just return the rows from the cache instead of fetching from db
                forwardReferenceSeriesValues.add(forwardReferenceSeriesPage.one().getDataIdentity());
            }
            PagingState nextPaginatedState = forwardReferenceSeriesPage.getExecutionInfo().getPagingState(); //Paginated state represents where we are in the Result Set so we can send the context for the next iteration
            LoadSeriesResponse loadPaginatedSeriesResponse = new LoadSeriesResponse();
            if (nextPaginatedState == null) {
                loadPaginatedSeriesResponse.setNext_page_context(new byte[]{});
            } else {
                loadPaginatedSeriesResponse.setNext_page_context(nextPaginatedState.toBytes());
            }
            loadPaginatedSeriesResponse.setValues(forwardReferenceSeriesValues);
            return loadPaginatedSeriesResponse;
        }
    });

}

[Andrew Tolbert]

 Also I thought you said isExhausted is not meant to check if there are results but now you are stating the contrary?

isExhausted is to check if there are any more rows to be read .  I was distinguishing that isExhausted is for checking if there are more rows, not if there were ever rows:

isExhausted isn't really meant for this purpose though.  It's for indicating if there are any rows left to consumed, not if there were ever any rows returned. 

Couldn't you check if the pageContext passed to loadSeriesInOrder was null/empty or not?   If there is pageContext present, you could assume that you are continuing a previous query and that there were rows returned, otherwise you are starting from the beginning and if getAvailableWithoutFetching() == 0 there were no rows.

Thanks,

Andy

[Kant Kodali]

what will be the pagecontext set to after fetching the last page in cassandra? empty or null ?

Pagecontext will be an empty byte array per say during the first invocation and last invocation according to the code I sent you. so if I do the following I will still will not be able to distinguish however if we can distinguish between the first page context and the last page context that can work but if they are empty then it will be a problem.

if(Pagecontext.remaining() == 0 && getAvailableWithoutFetching() == 0 ) {

      System.out.println("Discovered case 1&2")

}

[Kant Kodali]

so question really now is what does cassandra return in the final page context? for the very first one I pass an empty byte[] array.

[Olivier Michallat]

Hi,

The paging state is null if there is no next page (see the javadocs).

I don't understand what the issue is. How do you retrieve the paging state when an HTTP request comes in? You probably encode it in a query string parameter right? Then simply don't include the parameter in the URL of the first page and test for that in your Java code.

final boolean isFirstPage = ... // was the paging state provided in the URL?

Then use that in your callback:

boolean noResult = isFirstPage && (getAvailableWithoutFetching() == 0)

--

Olivier Michallat

Driver & tools engineer, DataStax