Yes, I have found the touch sense to be a little buggy at times. What I get is cross-talk. It is hard to tell if the issue is the ESP32 or my own touch pad design, it is a fairly involved science apparently. I seem to recall that the WiFi also confuses the internal cap sense, which wouldn't be the case with an external IC.
I have another project which uses an external IC for cap sense and it is a lot more reliable, but the tradeoff is that you don't get interrupts as easily, so you would need to poll the cap sense IC quite fast to avoid latency, or research an IC that provides interrupt-on-change if there is one.