Secure content not showing up

[Bruno]

Hello!
We're developing a feed client that pushes public and secure feeds
into the GSA, and we're experiencing some problems with some secure
content not showing up. The events leading up to that are as follows:

* We send a protected feed for which user A is authorized, with
content "foobar 0".
* After indexing, we can see it searching for "foobar" (our feed
client gets the HEAD request with user A credentials as expected).
* We then proceed to send a thousand feeds for which only user B is
authorized, with content "foobar 1" to "foobar 1000".
* Again after indexing, this time if we do the same "foobar" search
with user A, we get no results. If we check our feed client logs, we
can see that there were around 700 HEAD requests for the feeds that
only B can see, and are denied for user A. No more HEAD requests after
that, which appears to be the reason for the result not showing up.

Any idea on what might be happening? Seems like the GSA is giving up
after those permissions denied. Actually if we search for "foobar
0" (with which only one feed matches), we get the result.
If it is of any use, our GSA version is 6.0.0.G.22.

Thanks in advance!

[brianb]

Hi Bruno,

If I understand the explanation above, it sounds like it is mostly
working for you however, what you are saying is that it is stopped at
700 or so HEAD requests and not finding that one document that the
user has access to. Is this correct?
If so, the problem is likely that you are hitting a timeout. Check
your settings in Serving -> Access Control (see the timeouts at the
bottom) and you can try adjusting that. One other thing is that the
GSA will only return 1000 results for any query too so you would only
1000 max HEAD requests as long as they can finish within the timeout
limit. I think the max you can set the timeout limit is 25 seconds for
the whole result set and 5 seconds per individual request. So if your
server cannot respond to 1000 requests in 25 seconds, then you will
likely not see results. In most cases in real life searching this
would not be an issue since you would likely not have a user that has
only access to 1 single document out of 1000 results.

Hope this helps.

Brian

[Bruno]

Hi Brian,

Thanks for the answer, you got the problem right.
I already tried adjusting the authorization parameters, but didn't get
a different result.

So, this 1000 limit result also applies for the internal matches that
still require authorization? It sounds logical that that happens, but
I'm not sure of how unlikely it is to encounter that in a real life
scenario.
Do you know if that can be found somewhere in the documentation?

Thanks a lot!
Bruno.