RapidPro 5.2 Release Candidate

[Nic Pottier]

Hi all,

We are pleased to announce that we have pushed the first release candidate for the 5.2 release (v5.2.0) to the https://github.com/rapidpro/rapidpro repo. We consider this release stable but are entering a code freeze over the next week and will fix any critical bugs found during the next two weeks. Expect a final announcement of the official stable 5.2 release on November 4th as well as what we are planning for the 5.4 release due in March.

As always, this release wouldn’t be possible without the continued support of Nyaruka’s customers at UNICEF and TextIt (https://textit.in). A big thanks from the Nyaruka team for choosing us for your RapidPro hosting, the continued development of RapidPro wouldn’t be possible without it. Big thanks to Eric, Leah, Norbert and Rowan for their hard work in putting this release together.

Big updates in this release:

• You can now start a set of contacts in a flow based on a contact search. This works both when manually starting a flow from the user interface as well as the start flow action. This addresses a long standing use case of wanting to notify the contacts attached to a clinic / family / region from a flow. Previously your only choice was to create a group per clinic which didn’t scale well. Lots of interesting use cases are unlocked by this and we are looking forward to seeing what kinds of new things will be built by it.
• Contact pages now contain a detailed log of the events that occurred in recent flows the contact participated in. These are currently visible for 7 days after a session has ended. This makes it much easier to reason about what has happened in a flow run as you can see the individual events that occurred on that contact. The ability to do this kind of logging is one of the many big wins of the new flow engine.
• We have added support for Natural Language Understanding within flows. You can now connect a classifier from Wit, LUIS or Bothub from your account page and classify text within a flow using the “Split by Intent” action. You can pull out entities extracted during the classification using @results.[result name].extra.[entity name]. For example say you built a classifier for booking flights which extracted the destination and you split by that intent in a flow saving the result as intent, you could pull out the destination entity using @results.intent.extra.destination. Thanks to Ilhasoft for their contribution to add Bothub support.
• All templating is now in the new engine format, including campaign events, manual broadcasts and scheduled broadcasts. Previously these were still using the old templating format (IE: old @contact.age vs new @fields.age).
• We have moved the new editor out of beta and now default all new flows to use the new editor. We will be removing the option of creating new flows in the old editor in 5.4 and removing the old editor entirely for 6.0. (old flows will be automatically migrated to the new editor)

Other important work:

• A large part of our focus continues to be cleanup from the old engine. Rowan took on the Herculean task of ripping out the old engine while maintaining test coverage and we now have a much simpler codebase that no longer contains all the old (uncalled) engine code. To give you an idea of that, we’ve removed some 20,000 lines from the Python RapidPro codebase for this release.
• A related goal has been to move all messaging and flow related async activities into Mailroom. All broadcasts are now created and sent by Mailroom instead of the old Python code and scheduled flows and broadcasts are also now executed by Mailroom. This decreases the complexity of the RapidPro side and removes duplication while also making all these tasks more scalable for large installs. We have a few more small items to move over but we expect to have all async non-routine tasks done exclusively within Mailroom for 5.4.
• We are moving to using web components for the more complicated widgets used on the RapidPro frontend. Eric put in a big effort here in rewriting the alias editor, completion component, omnibox (used to select contacts and groups) and other base components for use on the frontend. While there has been a significant learning curve here, especially in making web components work with older browsers, we are excited about what they will enable in the future and the better user experience they provide.
• We have dusted off airtime transfer support, which was previously done via a platform named TransferTo. This is now called DTOne and we’ve integrated with their new APIs and airtime transfers are now available in the new engine.

While the above are the big pieces that went into this release it has been a tremendously busy four months for the Nyaruka team. We have made over 1,500 commits in that time in enhancements, cleanup, bug fixes and optimizations. To give you an idea of the pace we’ve been moving at, these are the commit counts since the 5.0 release in July:

• 789 RapidPro commits
• 221 Mailroom commits
• 277 GoFlow commits
• 164 FlowEditor commits
• 61 Courier commits
• 4 Archiver commits
• 0 Indexer commits

Summary of some of the smaller changes:

• Send test email when the SMTP server config are set
• Unify max Contact Fields count across system
• Don’t allow deletion of Flow labels with children
• Rework airtime transfers to have separate HTTP logs
• Sync intents on Classifier creation
• Trim HTTP logs older than 3 days
• Remove fragile Africa’s Talking links to configuration pages
• HTTPLog models and views, start of unified HTTP logging
• Add ThinQ Channel Type
• Fire schedule Flows and Broadcasts in Mailroom instead of Celery
• Update docs on flow start extra to be accessible via @trigger
• Fix issues with web components in Microsoft Edge
• List Contact fields with new expression syntax and fix how campaign dependencies are rendered
• Use Mailroom to interrupt runs when archiving or releasing a flow
• Allow viewers to do GETs on some API endpoints
• Fix rendering of campaign and event names in UI
• Allow viewing of Channel logs in anonymous orgs with URN values redacted
• Fix importing/copying flows when flow has group dependency with no name
• Remove all IVR flow running functionality which is now handled by Mailroom
• Update to Django 2.2
• Send credit expiration emails to all administrators
• Fix format_number to be able to handle decimals with more digits than current context precision
• Populate account SID and and auth token on Twilio callers when added

Cheers,

Nic

--

Nic Pottier                                    ni...@nyaruka.com
Nyaruka Ltd                                  http://nyaruka.com/