Third Party Access

[Med Dyer]

I recently released a user-friendly photogrammtery app for iOS and a companion 3-D model creation app for MacOS. Together these apps -- ReefScan and ReefBuild -- are intended to provide a free option for easy model creation, for groups that are not ready for the whole Agisoft route. I am working on adding an orthomosaic export option to ReefBuild, to allow single-button generation of a transect's orthomosaic, and I would like to include some level of AI coral ID. Is it possible to access the CoralNet algorithm and data for this use?

ReefScan for iOS in the App Store

ReefBuild for MacOS in the App Store

[Stephen Chan]

Hi there, those are quite useful looking apps! Thanks for sharing.

The closest thing to what you're looking for would be the CoralNet API. You can submit an API request with 2D images, point-position data, and an ID-number for a CoralNet automated classifier; and you'd later get an API response with coral-ID labels for the specified point positions.

API documentation: https://coralnet.ucsd.edu/pages/help/api/

Blog post with more discussion: https://coralnet.ucsd.edu/blog/automatically-annotating-175000-images-with-the-coralnet-api/

However, limitations to note:

1) We only support 2D images (JPEG or PNG), not 3D models.

2) Our pipeline isn't designed to process huge 2D images which might represent an entire orthomosaic; the main upload interface on CoralNet restricts single images to 8000 x 8000 pixels. I think this is a likely point of improvement for CoralNet's future, since a lot of folks have been wanting to analyze orthomosaics more easily, but for now we have this restriction.

3) Our API doesn't return results in real time. What you actually get from your initial API request is a request ID number, and then you'd periodically query the API until the response says the results are ready. With normal server load, you'd probably expect to wait on the order of 20 minutes.

[Med Dyer]

Thank you for that information. It looks like I would need to to specify a set of specific points in each image for analysis, which is what I recall the CoralNet website doing as well. So I specify some number of specific points and then get a return data set that indicates what might be pictured at those points. This is useful but I am looking towards coral coverage (% coverage) as well as species ID, across the full image. It sounds like I would still need some manual input to achieve that goal.

The huge size of orthomosaics (OM) is a problem. In my very limited experience, a resolution of 16kx16k is needed to cover a typical 5m-radius transect start. That resolution provides enough detail for coral ID and also coral health evaluation. The resulting TIFF though is gigantic, over 1GB and sometimes a multiple of that if the full transect length is included. For CoralNet though I can easily break my OM into smaller sizes. I'm going to experiment with the API and see how it goes. My goal is an app system that is mostly "press and go" for each step of the process. The processing delay for CoralNet is not an issue as every step of this process takes a long time.

[Stephen Chan]

Sounds like it may work out then - good luck!