Sending batches of appeal notifications

[Laurent Savaëte]

Hi all,

As part of setting up our appeal logic in madada.fr, we've hit a little annoyance in the law (yes, again) around batch requests.

When an appeal is made on a batch request sent to more than 5 public bodies, the requester has to notify all those public bodies that an appeal is made against them, by sending them individual emails. This is easy for small batches of a few entities, but we have users sending out batches to over 1000 public bodies, and we know from statistics that they will need to appeal about 80% of these. At that point, we hit the same emailing problems as with sending out the initial batch itself.

I have implemented the appeal logic with minimal info in the database (for privacy/gdpr reasons, we don't want to save all the PII that users have to provide to send their appeal), so I just save a "status_update" event with a new state of "cada_appeal", and no schema changes to limit problems with upgrades down the line. So I need to pass the list of requests/public bodies that need to be notified from the web request creating the appeal to some cron job that will actually send out those notifications.

I can see 2 distinct ways of doing this without modifying the db:

- I could add some transient marker like "appeal_notification_pending" in the params of the status_update event to identify requests from the cron job, and then remove that marker once the notification is sent. It feels a bit weird to modify an event object though. I'd also like to avoid adding another "appeal_notification_sent" event, because it would not be required in all cases and all the logic behind would become more convoluted.

- Alternately, I could use a tag on the request, which is removed once the notification is sent. Although this tag would live on the request only for a few minutes (in theory), there is always the risk that a user could remove it before it's processed (or worse, add this tag manually on some random request) , leading to weird situations. Obviously, I could special case that one tag everywhere, but it just feels clunky.

Are there any other solutions I could use that do not involve any changes in the db structure, or weird side effects?

Thanks for any ideas you may have!

Laurent / Ma Dada

[Gareth Rees]

The simpler option is to go with the tags. Users can't (yet) modify them, and the only way for users to tag requests is via the URL params [1] so that's unlikely.

I'd like to make tags more public/editable in future, but if we do that I'd want to make them "prominenceable" (i.e. able to be viewed by "anyone", "requester only", or "hidden" – so admin only). If we ever did that you could convert these tags to "hidden" prominence, and we could add some reserved words for tags that users can't add.

Agree that event data should be immutable.

Can't think of a better way than the tags off hand without adding database stuff.

Best,

Gareth

[1] https://alaveteli.org/docs/developers/api/#starting-new-requests-programmatically

[Laurent Savaëte]

Hi Gareth,

Thanks for this, it's helpful. I'll stick to tags as you suggest, and see if I can add some safeguards around them, maybe check which user created the tag or something along those lines to prevent abuse.

Cheers,

Laurent

--
You received this message because you are subscribed to the Google Groups "Alaveteli Dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to alaveteli-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/alaveteli-dev/853d8c88-db81-4479-93de-d31532fca84fn%40googlegroups.com.

[Gareth Rees]

I don't think we record the user who creates the tag at the moment, but you could use the tag value (tags can be like `foo:bar` where foo is the tag name and bar is the value) to create a random digest/uuid that's unlikely to ever be guessed (e.g. `appeal_notification_pending:d55ed616cdf9`).