[libRETS-users] librets SearchResultSet HasNext XML parse error

[Yuriy Raskin]

Hi Librets community

I am using librets-1.5.3 version compiled for Python 2.7 and dealing with the problem of iterating through a SearchResultSet object.
The operation of
while results.HasNext():
...................
throws the following error:
def HasNext(self): return _librets.SearchResultSet_HasNext(self)
ValueError: XML parse error at 62:1233: reference to invalid character number

So is it possible to skip a corrupted record and keep looping through the rest of the records in the SearchResultSet.

Thank you for you help an advance,

-Yuriy

Data architect @ Movoto Inc.


_______________________________________________
libRETS-users mailing list
libRET...@crt.realtors.org
http://mail.crt.realtors.org/mailman/listinfo/librets-users

[Keith T. Garner]

Unfortunately, its not possible as it means (from libRETS point of view) its a corrupted XML document. If you haven't already, try changing away from the default (US-ASCII) character set. Often times people find that ISO-8859-1 or UTF-8 often make these errors go away. If those don't work, talking to the MLS to fix the broken record may help.

Keith

--
Keith T. Garner - kga...@realtors.org - 312-329-3294 - http://realtor.org/
National Association of REALTORS® - VP - Information Technology Services

[Yuriy Raskin]

Hi Keith,

Thanks a lot for your quick response! So does it mean that I will need to
set a session object with SetDefaultEncoding (RETS_XML_UTF8_ENCODING) ?

But the most importance thing is if just a one record in a resultSet is
corrupted I want to be able to keep iterating through the rest of the
records but it seems impossible using HasNext() function. It would be nice
to have a GetNextObject() function in addition to HasNext().

Best Regards,
-Yuriy

[Keith T. Garner]

On Oct 12, 2012, at 12:06 AM, Yuriy Raskin <YRA...@movoto.com> wrote:
> Thanks a lot for your quick response! So does it mean that I will need to
> set a session object with SetDefaultEncoding (RETS_XML_UTF8_ENCODING) ?

Yep

> But the most importance thing is if just a one record in a resultSet is
> corrupted I want to be able to keep iterating through the rest of the
> records but it seems impossible using HasNext() function. It would be nice
> to have a GetNextObject() function in addition to HasNext().

Because of the way XML works, its really not possible since there are breakages of XML that make the document ambiguous to where you aren't sure if the current object has ended or its in the next one. (Its one reason XML isn't good for streaming applications like this.)

Keith

[Yuriy Raskin]

Thanks a lot!
________________________________________
From: librets-us...@crt.realtors.org [librets-us...@crt.realtors.org] on behalf of Keith T. Garner [kga...@crt.realtors.org]
Sent: Friday, October 12, 2012 4:43 AM
To: For discussion and help for and by the users of the libRETS C++ library
Cc: Alik Eliashberg
Subject: Re: [libRETS-users] librets SearchResultSet HasNext XML parse error

[Yuriy Raskin]

Hi Keith,

could you please also clarify the following issue.
For some MLSes when I am looping through the ResultSet I am getting the following error:
<RETS-STATUS ReplyCode="20208" ReplyText="Maximum Records Exceeded" />.
Should I do pagination by myself? How can I get the max allows records?

Thank you,
-Yuriy

________________________________________
From: librets-us...@crt.realtors.org [librets-us...@crt.realtors.org] on behalf of Keith T. Garner [kga...@crt.realtors.org]
Sent: Friday, October 12, 2012 4:43 AM
To: For discussion and help for and by the users of the libRETS C++ library
Cc: Alik Eliashberg
Subject: Re: [libRETS-users] librets SearchResultSet HasNext XML parse error

[Vladimir Sanchez]

I suggest you partition your search to fit the result set within the maximum allowed.

You can do it by zip code, subdivision or mlsno using an offset.

-Vlad

[Mark Klein]

On Oct 12, 2012, at 3:22 PM, Yuriy Raskin wrote:

> could you please also clarify the following issue.
> For some MLSes when I am looping through the ResultSet I am getting
> the following error:
> <RETS-STATUS ReplyCode="20208" ReplyText="Maximum Records Exceeded" /
> >.
> Should I do pagination by myself? How can I get the max allows
> records?

There are a couple of ways to address this, it all depends on what the
server supports since they're not required to support all of this:

1. Do a search and just return the count of listings. Use LIMIT and
OFFSET to loop through the resultset until you've retrieved all your
records.

2. See if the server supports HasKeyIndex (RETS 1.7.2 and later) and
InKeyIndex. Check the metadata CLASS and TABLE for this. A server is
supposed to suspend limits when only the keys within InKeyIndex are
used in a query. This will allow you to retrieve all the key fields
for your query. Then again, you can loop through those results and
fetch the records up to the server specified LIMIT per query using the
keys in your query.

3. Partition the query using the ModTimeStamp. It may take a bit of
trial and error to figure out what range can be used in order to not
exceed the system imposed LIMIT.

Hope that helps.

[Yuriy Raskin]

Thanks!