TypeError: 'NoneType' object is unsubscriptable

[Venkatesh Rangarajan]

Hi,

I am running a simple query

   keyword = self.request.get('keyword')

   if len(keyword) <=0:
    keyword='Google Engineer'

   query = search.SearchableQuery('Visa')
   query.Search(keyword)     

 for result in query.Get(100, int(page)):
    visas.append(result)

I keep running into the following error in some occasions. Not all.

Traceback (most recent call last):
  File "/base/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 496, in __call__
    handler.get(*groups)
  File "/base/data/home/apps/payrate/3.33/Main.py", line 142, in get

    for result in query.Get(100, int(page)):
  File "/base/python_lib/versions/1/google/appengine/api/datastore.py", line 938, in Get
    return self._Run(limit, offset)._Next(limit)
  File "/base/python_lib/versions/1/google/appengine/api/datastore.py", line 885, in _Run

    *datastore_index.CompositeIndexForQuery(pb)[:-1])
TypeError: 'NoneType' object is unsubscriptable

Any idea on how to handle these and what is causing it? And why does it happen only for certain keywords

Rgds,
Venkatesh

[Venkatesh Rangarajan]

Sample Error :

http://payrate.appspot.com/?keyword=Lead+Programmer

[Marzia Niccolai]

Hi,

I believe this is related to the 'offset' that you are are using, and am curious if you could elaborate more as to with which values of 'offset' this occurs and the number of results you think you have in your results set.

-Marzia

[Venkatesh Rangarajan]

Hi Marzia,

I think this issue is not related to the offset, because the same query was working fine previously. It started failing after the volume of data increased . I think the current volume is 700K (700+ MB).

I am using offset to limit 100 records per page.

Refer to : http://payrate.appspot.com/information_architect_salary

This was working till 2 days back and failing now. I have added a Unique URL to make it Google Search friendly...basically it translate to

http://payrate.appspot.com/?keyword=information+architect

At this point the offset would be zero.

Also : http://payrate.appspot.com/?keyword=Google+Software+Development+Engineer  . This fails with the same error

While
http://payrate.appspot.com/?keyword=Google+Engineer  works fine.

I have noticed that if I increase the number of keywords passed for the search, it fails with this error.

Please help.

Rgds,
Venkatesh

[Venkatesh Rangarajan]

And yes, my model is defined as a searchable entity. Using the default bulk-loader code with no tweaking.

def HandleEntity(self, entity):
    ent = search.SearchableEntity(entity)
    return ent

[Marzia Niccolai]

Hi,

Actually, upon further inspection, this is an issue of the query you are running needing an index, but since the query doesn't need one in most cases, we can't print the definition.

Please add indexes for the queries experiencing these issues.

-Marzia

[Venkatesh Rangarajan]

Its a search able model ? The documentation says they don't need indexes. Can you please advice me what index I should add ?

Here is my query below.

def db_visas(keyword, offset):
 visas=[]

 query = search.SearchableQuery('Visa')
 query.Search(keyword)

 for result in query.Get(101, offset):
  visas.append(result)
 return visas

[lf.hl]

Hi Marzia,

it seems i have a similar problem here and thought the missing index
file is the reason for this. I have a model "Segment" with an integer
member "spd". Multiplying "segment.spd" as an instance i got from a
query on "Segment" with some int value python tells me the TypeError
with int and NoneType. I don't have that problem on 32bit Ubuntu but
on 64 bit Ubuntu and on GAE. I defined an index file manually but
appcfg tells me i don't need it for that type of query.

I got rid of the problem on 64bit Ubuntu by realligning the code. But
that code did not work in GAE.

Best, lf.hl

On 1 Okt., 00:32, "Marzia Niccolai" <ma...@google.com> wrote:
> Hi,
>
> Actually, upon further inspection, this is an issue of the query you are
> running needing an index, but since the query doesn't need one in most
> cases, we can't print the definition.
>
> Please add indexes for the queries experiencing these issues.
>
> -Marzia
>
> On Tue, Sep 30, 2008 at 2:47 PM, Venkatesh Rangarajan <
>

[Marzia Niccolai]

These types of queries don't usually need indexes - unless they are too difficult to do without the index.  In this case you would need an index that is along the lines of:
- kind: Article
  properties:
  - name: __searchable_text_index
  - name: __searchable_text_index
  - name: __searchable_text_index
etc, depending on the number of keywords you are using in your query.

Unfortunately, queries needing this type of index have a high correspondence rate with exploding indexes (http://code.google.com/appengine/docs/datastore/queriesandindexes.html#Big_Entities_and_Exploding_Indexes), so it's probable that this index can't be built.

-Marzia

[Marzia Niccolai]

Hi,

This error seems like it is different.  From your description, with out seeing any code, it seems as though your entity doesn't have any value stored in the spd field, so when you are trying to multiply it with an integer value it's throwing an error.  What happens when you just write out the field segment.spd?

For further help, code snippets would be greatly appreciated.

-Marzia

[Venkatesh Rangarajan]

Marzia,

I added a few Index. NoneType errors have reduced. Still I get those errors every once in a while.

Data-Store timeouts are happening 50% of the time. Even for queries where the result set is really small. I am retrieving only 20 entities per request, but even that is causing time-out errors.

I am doing everything as per "documentation". At this time I am at less than 30% of the actual volume of data that I can potentially upload.

Please guide me on what I can do to improve my app ? I would prefer a little latency in page loading than milli second timeouts.

Rgds,
Venkatesh