Automations, DynamicLists, and Segmentations

Stephen Powis

unread,

Jul 29, 2013, 3:30:12 PM7/29/13

to newfeatures

Berg and I have slowly been transitioning us over from our old Solr based Automations, DynamicLists, and Segmentations (I'll refer to all three just as Automations to keep things simple) to our new rules engine that uses queues, and now is probably a good time to explain to everyone how it works, and how its different from our old way.

The way it used to work for Automations is we had background jobs that ran every so often and matched everyone in the account all at once. As a result we were able to show in the Pardot interface when these jobs last ran. This approach worked great for small and medium sized accounts, but completely fell apart for larger accounts.

The new way we process Automations we no longer have massive background jobs that run in each account. Instead whenever a prospect changes (this could be anything from a prospect viewing a web page, a CRM sync, an opportunity changed that is related to a prospect etc..), we push the prospect into a Queue. We then have a cluster of workers which pull prospects off of the other end of the queue, process all the relevant Automations and takes the required actions based on the out come of each (add/remove from dynamic list, or apply matched automation/segmentation actions).

Generally the time from when a prospect change happens until that prospect has had all the relevant rules processed, is very quick. For internal reference, most cases it is well within 10 seconds, however If there is a mass influx of changes to a specific shard, there can be longer delays. At this point I don't think we want to express any specific time of how quickly automations will run to clients, in the future we may have some kind of SLAs, but for now no specific time. I'll be adding tools to the app so support people can check the status of the Queue for any backlogs in the system if questions of slow rules come up.

Since there are no longer massive jobs processing accounts every so often, we can no longer show a 'last processed' or 'last run at' timestamp in the app. I believe I've removed all of these from the app, if you see any left around please let me know and I can remove it.

What if you want to force a refresh of a dynamic list? On the dynamic list read screen there should be a "refresh list" link that you can click. This will queue everyone in the account and start processing them. Once its completed processing all of the prospects it queued, it should send out a notification email saying it has completed.

What if you want to force a refresh of an automation? At this point time, there is no tool in the app to do this. Soon, it will work exactly like the dynamic list one. You click a link, you wait some time, and then an email shows up in your inbox saying it has completed. In the mean time you can PM me and we can make something happen.

What else is coming soon related to this?

- Graphs showing Queue backlogs per shard or per account. I'll send out another email when these are ready for support.

- When you create a new or edit an existing Automation, DynamicList, or Segmentation, the first time it "runs" you'll probably eventually see some fancy progress bar showing how far along it is processing everyone in the account.

I think that covers it, if you have any questions please reply and we can have a Q/A.

Stephen

Jeff Elrod

unread,

Jul 29, 2013, 3:33:06 PM7/29/13

to Stephen Powis, newfeatures

YES. All yes.

Jeff Elrod :: Support Engineer :: 404.492.6845 x 311
Pardot - A salesforce.com Company

Helpful Links :: Knowledge Base | Ideas

Jacquie Hauth

unread,

Jul 29, 2013, 3:33:25 PM7/29/13

to Stephen Powis, newfeatures

This is great, Powis! THANK YOU for all the amazing work you've been doing!! Does this mean that we're getting rid of the "real time" option for automation rules... since they ALL run in real time now?

Jacquie Hauth

Client Solutions Manager | Pardot - A salesforce.com Company

(404) 492-6845 x235 | @PardotJacquie

Get Answers: http://pardot.com/help | Share Best Practices: http://forum.pardot.com

On Mon, Jul 29, 2013 at 3:30 PM, Stephen Powis <stephe...@pardot.com> wrote:

Vincent Migliore

unread,

Jul 29, 2013, 3:39:53 PM7/29/13

to Jacquie Hauth, Stephen Powis, newfeatures

Good stuff. Thanks for the ridiculous amount of work on this and this
explanation.

Gracias

Mark Larson

unread,

Jul 29, 2013, 3:49:54 PM7/29/13

to Vincent Migliore, Jacquie Hauth, Stephen Powis, newfeatures

Sweeeeeeet! Appreciate all the details and the labor here, y'all.

-Mark

--

Mark Larson

Solutions Coordinator

Pardot Marketing Automation

404-492-6845 x351

Training | Knowledge Base | Idea Exchange | Blog

Razid Ahmad

unread,

Jul 29, 2013, 4:05:07 PM7/29/13

to Mark Larson, Vincent Migliore, Jacquie Hauth, Stephen Powis, newfeatures

All hail StevieP & Berg!

--
______________________________
Razid Ahmad
Email Engineer
Pardot - No Hassle Marketing Automation
950 E. Paces Ferry Rd.
Suite 3300
Atlanta, GA 30326
razid...@pardot.com

Anna Sims

unread,

Jul 29, 2013, 4:48:28 PM7/29/13

to Razid Ahmad, Mark Larson, Vincent Migliore, Jacquie Hauth, Stephen Powis, newfeatures

getting choked up over this great explanation of the new automation process. hi-fives all around!

Anna Sims :: Product Instructional Designer:: 404.492.6845 x279
Pardot - An salesforce.com Company
Helpful Links :: Knowledge Base | Ideas | Connections 2013! |

anigif_enhanced-buzz-15344-1369069121-11.gif

Robert Phillips

unread,

Jul 29, 2013, 4:55:12 PM7/29/13

to Anna Sims, Razid Ahmad, Mark Larson, Vincent Migliore, Jacquie Hauth, Stephen Powis, newfeatures

Someone get these guys some big a$$ chains!

--

Rob Phillips
The Pardot Solutions Team | http://www.pardot.com/help
(404) 492 6845 x 314

image.png

anigif_enhanced-buzz-15344-1369069121-11.gif

Reply all

Reply to author

Forward