Post processing?

5 views
Skip to first unread message

Jeff Stewart

unread,
Jan 28, 2010, 9:49:00 AM1/28/10
to crt-viele-r...@googlegroups.com
Does anyone know how I can trigger post-processing scripts to run after VieleRETS as successfully downloaded and synchronized?� BTW, I love the synchronization feature.

Jeff Stewart

unread,
Jan 28, 2010, 10:59:08 AM1/28/10
to crt-viele-r...@googlegroups.com
Hi Jared,

Importing directly into Open Realty.� I thought I might be delete any records and associated images for records identified as inactive after synchronization has completed.

Jeff

--- On Thu, 1/28/10, Jared Ritchey wrote:

From: Jared Ritchey
Subject: Re: [viele-dev] Post processing?
To: "'For discussion on the development of viele rets'"
Date: Thursday, January 28, 2010, 9:59 AM

Hello Jeff, what are you referring? Do you download CSV first and then wish to add listings to a DB? � ________________________________ Jared Ritchey / Design Team Leader http://www.UltimateIDX.com http://www.WPRealty.org http://www.JaredRitchey.com � This electronic mail message is intended exclusively for the individual or entity to which it is addressed. This message, together with any attachment, may contain confidential and privileged information. �The recipient is hereby put on notice to treat the information as confidential and privileged and to not disclose or use the information except as authorized by sender. Any unauthorized review, printing, retention, copying, disclosure, distribution, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. Thank you for your cooperation. -----Original Message-----
From: viele-de...@crt.realtors.org [mailto:viele-de...@crt.realtors.org] On Behalf Of Jeff Stewart
Sent: Thursday, January 28, 2010 6:49 AM
To: viel...@crt.realtors.org
Subject: [viele-dev] Post processing? � Does anyone know how I can trigger post-processing scripts to run after VieleRETS as successfully downloaded and synchronized?� BTW, I love the synchronization feature. � No virus found in this incoming message.
Checked by AVG - www.avg.com
Version: 9.0.733 / Virus Database: 271.1.1/2650 - Release Date: 01/27/10 11:36:00
-----Inline Attachment Follows-----

_______________________________________________
viele-dev mailing list
viel...@crt.realtors.org
http://mail.crt.realtors.org/mailman/listinfo/viele-dev

Jared Ritchey

unread,
Jan 28, 2010, 12:31:07 PM1/28/10
to crt-viele-r...@googlegroups.com
Hello Jeff;
The process I use is with vieleRETS is to run everything as a CSV dump
which would include all the fields I want and then I use an SQL insert
script on a CRON to update the OpenRealty DB.
This method allows me to bypass the OpenRealty process which can slow
down some updates to a crawl.

If however you don't want to do that, what I suggest is run the full
update via vieleRETS as you now plan.

Then run a separate vieleRETS extract for your property classes
(withdrawn, sold, expired etc.) as a CSV which should include ONLY the
MLS Number if possible (some require multiple queries and multi field
downloads meaning they have a minimum) It will be a short list in most
markets.

If you can do that I'll give you the scripts I use that looks at the
MLSID in the csv file and it will remove all listings based on the MLS
Number and will UNLINK all photos which if using vieleRETS will name the
files wi th 123456789_1.jpg etc.

Your daily updates simply add new or updated listings. It reduces the
daily update process to about 5% of the time. I recently did this very
thing for MOMLS and avoided downloading images all together simply by
using an addon we call "Remote Photos". Some MLS's provide a media
server so there isn't a need to download photos. In fact you never have
to deal with photo management unless you feel you need to.
confidential and privileged and to not disclose or use the informationexcept as authorized by sender. Any unauthorized review, printing,
23:36:00

Jeff Stewart

unread,
Feb 1, 2010, 7:15:53 AM2/1/10
to crt-viele-r...@googlegroups.com
Hi Jared,

Our MLS doesn't have a media server.  Also, it looks like they provide info on only active listings.  Doing a download to OpenRealty works most of the time, but I occasionally get an out of memory error during the process.  Is there anything you can suggest to keep it from running out of memory, or would going to a csv download avoid that?

If you are willing to send me your scripts to identify and delete inactive listings and photos, I would appreciate it.

Thanks,


Jeff

--- On Thu, 1/28/10, Jared Ritchey wrote:

From: Jared Ritchey
Subject: Re: [viele-dev] Post processing?
To: "'For discussion on the development of viele rets'"
Date: Thursday, January 28, 2010, 12:31 PM

Hello Jeff; The process I use is with vieleRETS is to run everything as a CSV dump which would include all the fields I want and then I use an SQL insert script on a CRON to update the OpenRealty DB. This method allows me to bypass the OpenRealty process which can slow down some updates to a crawl.   If however you don’t want to do that, what I suggest is run the full update via vieleRETS as you now plan.   Then run a separate vieleRETS extract for your property classes (withdrawn, sold, expired etc…) as a CSV which should include ONLY the MLS Number if possible (some require multiple queries and multi field downloads meaning they have a minimum) It will be a short list in most markets.   If you can do that I’ll give you the scripts I use that looks at the MLSID in the csv file and it will remove all listings based on the MLS Number and will UNLINK all photos which if using vieleRETS will name the files with 123456789_1.jpg etc.   Your daily updates simply add new or updated listings. It reduces the daily update process to about 5% of the time.  I recently did this very thing for MOMLS and avoided downloading images all together simply by using an addon we call “Remote Photos”. Some MLS’s provide a media server so there isn’t a need to download photos. In fact you never have to deal with photo management unless you feel you need to.   ________________________________ Jared Ritchey / Design Team Leader http://www.UltimateIDX.com http://www.WPRealty.org http://www.JaredRitchey.com   This electronic mail message is intended exclusively for the individual or entity to which it is addressed. This message, together with any attachment, may contain confidential and privileged information.  The recipient is hereby put on notice to treat the information as confidential and privileged and to not disclose or use the information except as authorized by sender. Any unauthorized review, printing, retention, copying, disclosure, distribution, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. Thank you for your cooperation.   -----Original Message-----

From: viele-de...@crt.realtors.org [mailto:viele-de...@crt.realtors.org] On Behalf Of Jeff Stewart
Sent: Thursday, January 28, 2010 7:59 AM
To: For discussion on the development of viele rets
Subject: Re: [viele-dev] Post processing?   Hi Jared,

Importing directly into Open Realty.  I thought I might be delete any records and associated images for records identified as inactive after synchronization has completed.

Jeff

--- On Thu, 1/28/10, Jared Ritchey wrote:
From: Jared Ritchey
Subject: Re: [viele-dev] Post processing?
To: "'For discussion on the development of viele rets'"
Date: Thursday, January 28, 2010, 9:59 AM Hello Jeff, what are you referring? Do you download CSV first and then wish to add listings to a DB?   ________________________________ Jared Ritchey / Design Team Leader http://www.UltimateIDX.com http://www.WPRealty.org http://www.JaredRitchey.com   This electronic mail message is intended exclusively for the individual or entity to which it is addressed. This message, together with any attachment, may contain confidential and privileged information.  The recipient is hereby put on notice to treat the information as confidential and privileged and to not disclose or use the information except as authorized by sender. Any unauthorized review, printing, retention, copying, disclosure, distribution, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. Thank you for your cooperation. -----Original Message-----

From: viele-de...@crt.realtors.org [mailto:viele-de...@crt.realtors.org] On Behalf Of Jeff Stewart

Jared Ritchey

unread,
Jan 28, 2010, 9:59:01 AM1/28/10
to crt-viele-r...@googlegroups.com

Jared Ritchey

unread,
Feb 1, 2010, 8:09:56 AM2/1/10
to crt-viele-r...@googlegroups.com
Batch processing, by breaking the downloads down into smaller chunks is
what I use. I sometimes download data in one extract and images in one
or many extracts. Once you download images it becomes a maintenance
download from that point on so it should be faster to make the data
download and image download separate.


________________________________
Jared Ritchey / Design Team Leader
http://www.UltimateIDX.com
http://www.WPRealty.org
http://www.JaredRitchey.com

This electronic mail message is intended exclusively for the individual
or entity to which it is addressed. This message, together with any
attachment, may contain confidential and privileged information. The
recipient is hereby put on notice to treat the information as
confidential and privileged and to not disclose or use the information
except as authorized by s ender. Any unauthorized review, printing,

retention, copying, disclosure, distribution, retransmission,
dissemination or other use of, or taking of any action in reliance upon,
this information by persons or entities other than the intended
recipient is prohibited. Thank you for your cooperation.
-----Original Message-----
From: viele-de...@crt.realtors.org
[mailto:viele-de...@crt.realtors.org] On Behalf Of Jeff Stewart
Sent: Monday, February 01, 2010 4:16 AM
To: For discussion on the development of viele rets
Subject: Re: [viele-dev] Post processing?


Hi Jared,

Our MLS doesn't have a media server. Also, it looks like they provide
info on only active listings. Doing a download to OpenRealty works most
of the time, but I occasionally get an out of memory error during the
process. Is there anything you can suggest to keep it from running out
of memory, or would going to a csv download avoid that?

If you are willing to send me your scripts to identify and delete
inactive listings and photos, I would appreciate it.

Thanks,

Jeff

--- On Thu, 1/28/10, Jared Ritchey wrote:

From: Jared Ritchey
Subject: Re: [viele-dev] Post processing?
To: "'For discussion on the development of viele rets'"

Date: Thursday, January 28, 2010, 12:31 PM
Hello Jeff;
The process I use is with vieleRETS is to run everything as a CSV dump
which would include all the fields I want and then I use an SQL insert
script on a CRON to update the OpenRealty DB.
This method allows me to bypass the OpenRealty process which can slow
down some updates to a crawl.

If however you don't want to do that, what I suggest is run the full
update via vieleRETS as you now plan.

Then run a separate vieleRETS extract for your property classes
(withdrawn, sold, expired etc.) as a CSV which should include ONLY the

MLS Number if possible (some require multiple queries and multi field
downloads meaning they have a minimum) It will be a short list in most
markets.

If you can do that I'll give you the scripts I use that looks at the
MLSID in the csv file and it will remove all listings based on the MLS
Number and will UNLINK all photos which if using vieleRETS will name the
files with 123456789_1.jpg etc.

Your daily updates simply add new or updated listings. It reduces the
daily update process to about 5% of the time. I recently did this very
thing for MOMLS and avoided downloading images all together simply by
using an addon we call "Remote Photos". Some MLS's provide a media
server so there isn't a need to download photos. In fact you never have
to deal with photo management unless you feel you need to.

________________________________
Jared Ritchey / Design Team Leader
http://www.UltimateIDX.com
http://www.WPRealty.org
http://www.JaredRitchey.com

This electronic mail message is intended exclusively for the individual
or entity to which it is addressed. This message, together with any
attachment, may contain confidential and privileged information. The
recipient is hereby put on notice to treat the information as
confidential and privileged and to not disclose or use the information
except as authorized by sender. Any unauthorized review, printing,
retention, copying, disclosure, distribution, retransmission,
dissemination or other use of, or taking of any action in reliance upon,
this information by persons or entities other than the intended
recipient is prohibited. Thank you for your cooperation.

-----Original Message-----
From: viele-de...@crt.realtors.org
[mailto:viele-dev-bounces@crt. realtors.org] On Behalf Of Jeff Stewart
Sent: Thursday, January 28, 2010 7:59 AM
To: For discussion on the development of viele rets
Subject: Re: [viele-dev] Post processing?


Hi Jared,

Importing directly into Open Realty. I thought I might be delete any
records and associated images for records identified as inactive after
synchronization has completed.

Jeff

--- On Thu, 1/28/10, Jared Ritchey wrote:

From: Jared Ritchey
Subject: Re: [viele-dev] Post processing?
To: "'For discussion on the development of viele rets'"

Date: Thursday, January 28, 2010, 9:59 AM
-----Inline Attachment Follows-----
_______________________________________________
viele-dev mailing list
viel...@crt.realtors.org
http://mail.crt.realtors.org/mailman/listinfo/viele-dev

No virus found in this incoming message.
Checked by AVG - www.avg.com
Version: 9.0.733 / Virus Database: 271.1.1/2650 - Release Date: 01/27/10
23:36:00

-----Inline Attachment Follows-----
_______________________________________________
viele-dev mailing list
viel...@crt.realtors.org
http://mail.crt.realtors.org/mailman/l istinfo/viele-dev

No virus found in this incoming message.
Checked by AVG - www.avg.com
Version: 9.0.733 / Virus Database: 271.1.1/2660 - Release Date: 01/31/10
11:35:00

Rama Schneider

unread,
Feb 1, 2010, 7:28:01 AM2/1/10
to crt-viele-r...@googlegroups.com
I ran into that same issue. I was able to get around the problem by
breaking the downloads up in to smaller pieces and then running them one
by one. In my case I was able to break it down by city/town.

Rama Schneider


On 2/1/2010 7:15 AM, Jeff Stewart wrote:
> Hi Jared,
>
> Our MLS doesn't have a media server. Also, it looks like they provide
> info on only active listings. Doing a download to OpenRealty works most
> of the time, but I occasionally get an out of memory error during the
> process. Is there anything you can suggest to keep it from running out
> of memory, or would going to a csv download avoid that?
>
> If you are willing to send me your scripts to identify and delete
> inactive listings and photos, I would appreciate it.
>
> Thanks,
>
> Jeff
>
> --- On *Thu, 1/28/10, Jared Ritchey //* wrote:
>
>
> From: Jared Ritchey
> Subject: Re: [viele-dev] Post processing?
> To: "'For discussion on the development of viele rets'"
>
> Date: Thursday, January 28, 2010, 12:31 PM
>
> Hello Jeff;
>
> The process I use is with vieleRETS is to run everything as a CSV
> dump which would include all the fields I want and then I use an SQL
> insert script on a CRON to update the OpenRealty DB.
>
> This method allows me to bypass the OpenRealty process which can
> slow down some updates to a crawl.
>
> If however you don’t want to do that, what I suggest is run the full
> update via vieleRETS as you now plan.
>
> Then run a separate vieleRETS extract for your property classes
> (withdrawn, sold, expired etc…) as a CSV which should include ONLY

> the MLS Number if possible (some require multiple queries and multi
> field downloads meaning they have a minimu m) It will be a short list
> in most markets.
>
> If you can do that *I’ll give you the scripts* I use that looks at
> *From:* viele-dev-bounces@ crt.realtors.org
> [mailto:viele-de...@crt.realtors.org] *On Behalf Of *Jeff Stewart
> *Sent:* Thursday, January 28, 2010 7:59 AM
> *To:* For discussion on the development of viele rets
> *Subject:* Re: [viele-dev] Post processing?

>
> Hi Jared,
>
> Importing directly into Open Realty. I thought I might be delete any
> records and associated images for records identified as inactive
> after synchronization has completed.
>
> Jeff
>
> --- On *Thu, 1/28/10, Jared Ritchey //* wrote:
>
>
> From: Jared Ritchey
> Subject: Re: [viele-dev] Post processing?
> To: "'For discussion on the development of viele rets'"
>
> Date: Thursday, January 28, 2010, 9:59 AM
>
> Hello Jeff, what are you referring? Do you download CSV first and
> then wish to add listings to a DB?
>
> ________________________________
>
> Jared Ritchey / Design Team Leader
>
>
> This electronic mail message is intended exclusively for the
> individual or entity to which it is addressed. This message,
> together with any attachment, may contain confidential and
> privileged information. The recipient is hereby put on notice to
> treat the information as confidential and privileged and to not
> disclose or use the information except as authorized by sender. Any
> unauthorized review, printing, retention, copying, disclosure,
> distribution, retransmission, dissemination or other use of, or
> taking of any action in reliance upon, this information by persons
> or entitie s other than the intended recipient is prohibited. Thank

> you for your cooperation.
>
> -----Original Message-----
> *From:* viele-de...@crt.realtors.org
> [mailto:viele-de...@crt.realtors.org] *On Behalf Of *Jeff Stewart
> *Sent:* Thursday, January 28, 2010 6:49 AM
> *To:* viel...@crt.realtors.org
> *Subject:* [viele-dev] Post processing?

>
> Does anyone know how I can trigger post-processing scripts to run
> after VieleRETS as successfully downloaded and synchronized? BTW, I
> love the synchronization feature.
>
> No virus found in this incoming message.
> Checked by AVG - www.avg.com
> Version: 9.0.733 / Virus Database: 271.1.1/2650 - Release Date:
> 01/27/10 11:36:00
>
>
> -----Inline Attachment Follows-----
>
> _______________________________________________
> viele-dev mailing list
Reply all
Reply to author
Forward
0 new messages