I am trying to build an WebApp, and this is my first one. do we need to use external router like gorilla/mux for routing ?
You certainly don't "need" to use one. Go has a router built into the net/http package. A simple example of a trivial "web app" is available here: https://gobyexample.com/http-servers -- it uses the built-in router.
That said, many folks do prefer to use gorilla/mux or other advanced routers for the extra features they offer over the built-in ones. Others prefer to use more complete web-app frameworks like Gin or Beego. YMMV based on your exact needs.