A common approach is to use connection pooling in the client application.
It avoids the need to re-establish connections (which can be expensive in terms of latency) and allows you to limit the total amount of connections in use at any time. It works well in practice.
In case it doesn't, then you may need to introduce caching in your application or apply other techniques (for example creating read replicas and spreading load to them or sharding the data amongst multiple database instances). Things get a lot more complicated once you start going down this path so it's best to see if connection pooling will suffice.
If performance is a concern, I suggest setting up a load test to see how well the application performs.