fyi-cli/mcp: Tool for Working with FOI Requests (https://github.com/edithatogo/fyi-cli)

[Dylan Mordaunt]

Hi all,

I'm a physician, economist and executive by background, from Australia and NZ. It is in this context that I oversee and am interested in "Freedom of Information" requests.

I wanted to study FOI requests at scale, therefore access the Alaveteli API. I couldn't find an equivalent tool, so I created one: https://github.com/edithatogo/fyi-cli.

fyi-cli / fyi-mcp is a privacy-first client for Alaveteli-based platforms. It was prototyped for New Zealand in Python, but subsequently adapted to Rust (Python still available, but I won't be maintaining it). Because of the nature of the work I'm doing, I needed to be able to have robust, scalable memory management, hence the choice.

I have subsequently extended it to all English-speaking countries. It should work for non-English speaking countries, and I've developed it to be, but this would need validation from others to make more robust..

The initial output from this pipeline is a modelling exercise, to create a global ontology for Freedom of Information (FOI-O). The first version, based on NZ and modelled using FOI (OIA) data, is described here: 

FOI-O: An NZ-first ontology and verification methods package for Freedom of Information process modelling. https://doi.org/10.48550/arXiv.2607.02947; https://github.com/edithatogo/foi-o.

As this work progresses, I'll "harden" the tool for use with other jurisdictions. I will also scale the methods I've been working on to enable improvement and innovation around FOI.

The tool is permissively licensed (MIT) and could easily be adopted if this group felt that it was appropriate- happy to add others as contributers as a starting point and depending on this community's governance, perhaps others may be interested in adopting it as a community project.

This is my personal email, but it's probably best to contact me at dylan.mordaunt [at] vuw.ac.nz.

Kind regards,

Dylan

[Oliver Lineham]

Kia ora Dylan,

I run FYI.org.nz, the Alaveteli site for NZ which you refer to here. As far as I know, you've not made any attempt to get in touch about this work. If you have, and I've missed it, I'm sorry.

While I love it when people do interesting things with the information released through the service - I must ask you to immediately stop using automated tools with the site for the time being. Please contact me privately (oliver at fyi) to discuss the way forward. I see you're at VUW. If you're in Wellington maybe we can arrange a time to meet in person.

Please also be aware that I intend to implement countermeasures to block access by most automated tools, such as scrapers and bots like this one, unless pre-authorised. 

In principle I have no objection to AI using information from the site (this is about freedom of information after all). However, many of our users will know that the platform has become very unstable over the last year. That's largely because AI scrapers and bots now account for over 95% of the system load, and have caused the service to frequently be completely unavailable to its human users. 

Please understand that FYI receives no funding - it is personally costing me $1500-$2000 and countless hours per month to just barely keep the site online. If I can't get the costs down, stability up, and deal with some other abuse problems, then it is likely that the site will close.

There are infrastructure improvements which could be made to reduce the impact; nevertheless that is the current situation. The behaviour of many of these tools (such as only backing off once a 429/5xx is received), in contrast to traditional search engine crawlers, has been indistinguishable from layer 7 denial-of-service attacks we have also been hit with this year.

Regards,

Oliver

FYI.org.nz administrator

[Dylan Mordaunt]

Hi Oliver,

No problems. Yes, I did reach out but didn't receive a response. I subsequently connected with someone at the university who said they would connect us, but unfortunately that didn't arise.

The initial work wasn't research- it's probably best not to explain in an open forum. The tool isn't AI nor are the outputs intended for training LLMs, I appreciate what you are saying but this is quite different and aligned with a separate "rules as code" project.

Kind regards,

Dylan

--
You received this message because you are subscribed to a topic in the Google Groups "Alaveteli Community" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/alaveteli-users/d5kw-_UUMJg/unsubscribe.
To unsubscribe from this group and all its topics, send an email to alaveteli-use...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/alaveteli-users/b873a4a5-2742-46e1-ade8-1d99db8d2a0fn%40googlegroups.com.

[Ben Fairless]

Hi Dylan,

I'm Ben, the CEO of the OpenAustralia Foundation. We run Right to Know, the Alaveteli site for Australia, and I want to echo Oliver's concerns.

This work is interesting and clearly well-intentioned, and we would promote the free and open access to information. But I'd repeat Oliver's request plainly: please pause the use of automated tools without the consent of the site operator.

We are a small charity running multiple public services on an extremely tight budget. Instead of prioritising new features or fixing bugs, I have to divert financial, development and leadership resources into dealing with the significant increase in traffic caused by automated bots/scrapers/AI. This increase in traffic has caused multiple real outages, which have a direct impact on the people who rely on our services: the public, journalists and advocates making and reading FOI requests.

When automated tools create the risk that our services will be unavailable for real people, we consider that no different to a denial of service attack. We have already taken the regrettable step of implementing countermeasures to block access by automated tools, such as scrapers and bots like this one.

One idea might be to contribute to upstream Alaveteli to create APIs that can grant access to the data you seek without causing service outages. That way, when Alaveteli instances update to the latest version, you can have endpoints you can target (and that site operators can choose to enable).

Kind Regards,

Ben

--

Ben Fairless (He/Him)

Chief Executive Officer, OpenAustralia Foundation

b...@oaf.org.au

openaustralia · planningalerts · righttoknow · theyvoteforyou

Donate today

My working hours might not match yours — so reply whenever it's convenient for you, not when it's convenient for me. We acknowledge the Traditional Custodians of the lands on which we work, and pay our respects to Elders past and present.

You received this message because you are subscribed to the Google Groups "Alaveteli Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to alaveteli-use...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/alaveteli-users/CA%2BD7Coyo9ua4XXC5QBh5-LkhS%2B%3DqAQDOKU6%3Demge447Un5LpkA%40mail.gmail.com.

[Dylan Mordaunt]

Hi Ben,

I'm hearing what you are saying. I haven't yet used this outside NZ even though I'm based in Australia, and even then, there were rate controls in place. The existing controls can be advanced and dynamic approaches added.

If you're able to point me to areas of Alvateli that would be helpful to contribute to, I'm more than happy to. Do you know the origins of the scraping? I have some thoughts on how that could be addressed, depending on the origins.

Cheers, 

Dylan

---

From: alavete...@googlegroups.com <alavete...@googlegroups.com> on behalf of Ben Fairless <b...@oaf.org.au>
Sent: Thursday, July 9, 2026 9:25:13 pm
To: alavete...@googlegroups.com <alavete...@googlegroups.com>
Subject: Re: [alaveteli-community] Re: fyi-cli/mcp: Tool for Working with FOI Requests (https://github.com/edithatogo/fyi-cli)

To view this discussion visit https://groups.google.com/d/msgid/alaveteli-users/CAKnAbpbHqmTEjPYx_LZ1LEmLge3MAphmDtA6d_PSZmpv-uBMLg%40mail.gmail.com.

[Oliver Lineham]

Hi Dylan,

Whatever the tool, please don't do scraping until we've had a chat - this is just to protect the stability of the somewhat fragile site. I'm on holiday but back in town from the 20th. I put this on the mailing list just in the hope that others thinking of doing a similar thing might see it and also get in touch.

I can also explain some of the less obvious limitations of the request metadata which may help your project.

Oliver

You received this message because you are subscribed to the Google Groups "Alaveteli Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to alaveteli-use...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/alaveteli-users/CA%2BD7Coyo9ua4XXC5QBh5-LkhS%2B%3DqAQDOKU6%3Demge447Un5LpkA%40mail.gmail.com.

[Dylan Mordaunt]

Hi Oliver,

Again, I'm hearing what you're saying. It sounds like I've walked into something wider here. For what I'm doing this isn't wildly unconstrained use, it is relatively specific and with clear constraints on rate etc. 

Per Ben, I'll take a look at the underlying Alvateli code and see what suggestions might help. I'll ensure there are stronger controls in the API wrapper. Having had a quick look, I've got some immediate thoughts.

Around unconstrained use, if you are able to share logs it may help me clarify some of the options available, which wouldn't be expensive to implement.

It also sounds like at least Australia and NZ have challenges with fundraising. Happy to help with a strategy around that. 

Cheers,

Dylan

To view this discussion visit https://groups.google.com/d/msgid/alaveteli-users/CALGijv50eYTMYqBxx5xyL%3Dq53zTfuMScyof6hsewzx9Qvn5dwQ%40mail.gmail.com.

[Oliver Lineham]

Dylan, can you confirm you'll stop automated tool use until approved? Your replies have not clearly said so.

Thanks, 

Oliver

To view this discussion visit https://groups.google.com/d/msgid/alaveteli-users/CA%2BD7CoyLUQvYotO0eQ5EVuDPSqBO09yXc%2B3usH8WHnHpVjTk9g%40mail.gmail.com.

[Dylan Mordaunt]

Hi Oliver,

At this point I'm going to disengage. I've acknowledged your feedback..Whomever is scraping your server at a high tempo is very unlikely to reach out to a public forum and acknowledge it. As I have said, my work is completed.

If you pass on the logs, I'm happy to help troubleshoot. I've added additional controls to the CLI, so that people who aren't aware they should rate and volume control, are far more constrained in their choices. I've also set aside some time to make several changes to Alvateli that I think would be quick wins in terms of mitigation of what is presumably bot-overload.

I can see that there is a misunderstanding here but I can't see further engagement is going to be helpful. If your intention isn't for people to use the API, I would either restrict or turn it off. The risk you run in doing that is that whomever is doing this may simply overload you through browser scripting, and if they want to, Cloudflare and the like aren't terribly difficult to bypass. I have some suggestions for that.

Regards,

Dylan

To view this discussion visit https://groups.google.com/d/msgid/alaveteli-users/CALGijv7g__UWtUTUOnqSVsgKkhvNn3xJLescsbmaBRKG9PNZzg%40mail.gmail.com.