How to build a simple search engine on Google cloud platform(GCP)?

[Yifan Ye]

we  have built a website, the backend is based on fastapi, deployed in cloud run. currently, it reads data from google BigTable. The schema of the BigTable is really simple: row_key is the item_id, and columns are some info of the item (item name, item description etc).

Currently the logic of our website is very simple, when you click an item in the website, the website backend will query the bigtable based on the item_id, and the website will display all the info of this item.

But we want to add search functionality to the website , which enables user to search by item name. The first solution came into my mind is: promote the item_name to the row_key of bigtable, and use regex string match to "search" the gived substring in the bigtable. But it will not work, since:

1. the item name can be in different languages , while bigtable row_key only support ASCII string.
2. I feel this is not an efficient design, there must be a better way to support string match based search engines.

So my question: how to build a search engine for my website in gcp, in the context of the above.

[garye...@google.com]

Hi Yifan,

Are you tied to using Bigtable as your only data store? If not, there might be a better option.

How do new Items get created in the database? Would it be possible to also write new Items to a search engine such as Elasticsearch (https://www.elastic.co/partners/google-cloud)? In this model, search requests go to Elasticsearch which can return the relevant item_id(s) which can then be looked up very quickly in Bigtable. 

Thanks,

gary