By default, the C++ standard library is very minimal.
You need to set APP_STL in your Application.mk file.
I use:
APP_STL := gnustl_static
but you could have used system, stlport_static, stlport_shared, or gnustl_static.
It's documented under $NDK/docs/CPLUSPLUS-SUPPORT.html, and it's a little hidden, because the $NDK/documentation.html index file doesn't list it.