Picture tagging and uploading to Database

[Henk Troost]

Hi all,

I'm new to the forum so i will just shoot.

Simple question: is what i am asking for possible?

If yes: which way should i be looking?

I have been googling and searching for ways to upload images to a database and came across google app engine.

The background

For my business (export of fresh products) i want to use daily images from my suppliers to enhance our productoffering to our clients.
These images are coming in through email (we are using exchange server & outlook 2013).
At this moment we are having built a custom ERP-solution (Program X): to my knowledge webbased and working under SQL.
(i am not familiar with programming etc)

The idea
I want to tag image content and upload them to a database.
Then i want to be able to generate a newsletter for clients including these images.
Also an offer for clients from a sinlge supplier is possible
In the future the output will be extended to web / whatsapp and/or other apps)

Tagging: 
Product: we have a product structure hierarchical in the custom ERP-solution (Fruit - Apple - Apple Fuiji - Apple Fuiji New Zealand - Apple Fuiji New Zealand 18 kg box size 120)
Supplier: the supplier is also a tag

Speed:
Everyday there is a constant timelimit and stress (Imagine 20 calls a minute with suppliers/clients for 5-6 hours in a row for example) 
The imageupload and tagging has to be done very fast and user friendly. Ideally just on the image Copy Command (Ctrl-C) and Paset into uploader. Product and Supplier tag. Done

Example
1) tagging picture with "supplier A"  and  'Apple Fuiji New Zealand 18 kg box size 120'
2) entry in Program X:  supplier A  has the product 'Apple Fuiji New Zealand 18 kg box size 120'   costing EUR 12

Output
1) Total offer for clients with numerous products (hundreds) form several supplliers (50-100) illustrated with several pictures
2) Single offer for clients: including the tagged picture and individual price setting (Automatically done by program X)

Well that's it
Tnx for shooting

Henk Troost
Netherlands

[Ani Hatzis]

Hello Henk,

you could also use other GCP products to achieve this workflow, but considering your work-load peaks and the described use-cases App Engine could be a good platform choice. Here comes the shooting :)

Incoming emails

I'm not sure what caveats come for copy&paste of images between Outlook and a web UI, but I think it should be possible to automate some of these steps. If you don't have a dedicated email address for such messages, Exchange / Outlook could submit POST requests to the GAE app and upload the images on request of the receiving Outlook user. If all suppliers send their messages to a dedicated address (e.g. upl...@example.com), you could configure GAE to handle all incoming mail messages and process the images. Python standard environment has InboundMailHandler for doing this. (Note: Although this feature might not be available for all supported GAE environments or languages, you can combine multiple environments/languages in the same app)

If there are not many suppliers you might consider the burden of managing supplier-side users with authorization to upload and tagging images directly into your app.

However, the basic steps of the inbound mail handler could be:

1. Retrieve image data from the email (and ignore all other binaries / attachments?) and move the images into a dedicated Cloud Storage bucket.
2. For each image, create a "pending tag-job" in Cloud Datastore.
3. Save mail body and meta data into Cloud Datastore (a very scalable no-sql database) as "email", and link it to all related tag-jobs (i.e. images).
4. Maybe resize the images (some GAE versions have features for that or use a 3rd party lib) or you could use a Cloud Function that automatically resizes all images that are added to a certain bucket.
5. Let the app suggest some tags based on the sender of the email (easy with a map of sender addresses and supplier tags), and maybe even the product tags if the emails are supposed to be structured.
6. Note: There is a maximum size of inbound/outbound mail messages, including attachments: 31.5 MB.

Tagging

In a web UI of the same GAE app, authorized users (of your company) can browse all pending (or completed) tag-jobs, and validate / edit the product and supplier tags that were automatically suggested by the app. (If you are lucky, you can train a machine learning product to tag the images by itself or at least suggest a smaller set of tags, e.g. "Apple Fuji", together with sender addresses it might be possible to reduce the manual tagging effort significantly)

Output

This sounds a little bit like a shop or catalogue. The web pages for that could be done in GAE, maybe with links to Program X pages that also contain the prices? I also don't expect any issues if you want to send the output in different formats through other channels, at least not if there is some kind of HTTP / REST API available. There is also SMS and voice service support in some GAE versions. If the images can be public, you could use the Cloud Storage URLs for caching (faster and cheaper), also in newsletters.

The newsletters however could be a little tricky. GAE supports sending mails only in small quotas (if I remember correctly), bulk emails (i.e. same mail to many recipients), or by a 3rd party SMTP server. But there is so much to consider (bounce handling, opt-in / double opt-in etc.), it might be easier to send emails through a mail service provider. Mailgun, Mailjet and SendGrid have special pricing and maybe even free tier for GCP customers, but any provider should work if your app can use their API.

Administrative UI

Maybe also some functions to delete images/emails, mark tag-jobs as completed, edit mappings used for auto suggested tags etc. Maybe also an import of product / supplier data from Program X (CSV?). User authorization. (Authentication probably best via oAuth 2 since there is so much that could go wrong)

I hope my 2 cents were helpful.

Groet uit Duitsland.

Ani

[George (Cloud Platform Support)]

Hello Henk, 

What you desire is certainly attainable. As in similar cases, there is more than one solution possible, so you have a wealth of choices at your disposal. To store your images you may use Datastore and Cloud Storage in conjunction, where the storage it meant to store the actual images. One possible implementation is described in the reply to the "Storing images and structured data together (Google Cloud Platform)" question in stackoverflow.  

For the actual tagging effort, your app may use the Vision API to assign tags based on certain features detected in the image. Label Detection detects broad sets of categories within an image, which range from modes of transportation to animals. You may get a feel of how this feature works from the "Detecting Labels" documentation page. 

For coding issues specifically, you reach a larger pool of specialized programmers if you publish your question on Stackoverflow.