Update large amount of outdated data

[matthew reed]

Hi,

I'm looking at grabbing all the latest data from https://www.aip.net.nz/ to update some very outdated data in New Zealand. I will automate this process so it can be done regularly. Every cycle or something like that.

How can I find out where the openAIP auto generated data source comes from? It's seems to be incorrect where openAIP is sourcing this data from, maybe this point needs to be updated?

I'm planning to scrape all the latest data to supply to AvareX EFB app along with updating OpenAIP

The limitation with the API is you can only update a single airport at a time, maybe I can update the complete JSON file found here https://storage.googleapis.com/29f98e10-a489-4c82-ae5e-489dbcd4912f/nz_apt.json and then supply back to OpenAIP?

There is also new airspace that will also need to be updated. That goes in effect next month so will update then. 

I would create a new JSON and compare with existing for data consistency and only update what's required. 

Thanks,

Matthew 

[webmaster]

Hi Matthew,

this is great news. We appreciate your effort very much! I just checked the update history of NZ data. The last update was 03/2024 and we received data from a contributor. Though the data was updated 03/2024, it may be older than 2024 depending on the time the data was sourced - unfortunately I don't know the exact date. So having the dataset updated with recent data and even getting an automated updated process in place would be awesome! 

OpenAIP provides update endpoints only for single documents. Each update process is always an isolated operation. What you can easily do is create a JSON file that contains all updated data, then iterate over the items and update each item one by one. This works if you update the data inside a openAIP derived JSON file which will contain the document ID. This way you can target the specific documents via the OpenAIP API. 

If you want to automate the process, the easiest way to have a full clean update is: wipe all current data, replace with new one. This is how UK is updated automatically at the moment. For smooth updates, we need to integrate that a bit more into OpenAIP I suppose. We can have a private chat about that and figure out a way to achieve this.

Cheers,

Stephan

[matthew reed]

Hi Stephan,

After reading through the disclaimer on AIP site, it looks like I can only use the data from there for personal use only. So looks like this option is out. There are some other sources I was looking into that do have approval if we can discuss would be great.

What's your email address? Or do you want to send me an email and will discuss some options.

Thanks,

Matthew