Arun,
They have different roles. A Reference is a handle to ANY value, and you don't necessarily know what that is.
An Object is the root of all vectors and maps, but not of scalars. Scalars are just themselves.
This separation is such that any functions that look up a new value can return a Reference, and that all code that figures out what type something is can be in Reference, and is not duplicated among the various kinds of containers.
So to implement your object visitor pattern, you can implement a dispatcher on Reference, and not on anything else.
We could make Reference a subclass of Object purely in the sense that they share similar instance variables, but really they are different things. Maybe I should make Object private, since I don't see any reason for anyone to be needing to use it.