Part of this stems from the fact that an OAuth 2 access token is not
directly related to a user session. The client application receives
both the access token and the refresh token (if being used), but not
necessarily anything specific to a user session. Of course, you can
include user info in the access token attribute set. But that doesn't
necessarily imply the ability for the Resource Server to know how to
validate the user's login session.
You might take a look at how OpenID Connect is addressing this
specifically through the addition of an ID token and session info.
While the specs are not yet fully baked, it does promise to add an
identity layer on top of OAuth 2 that may help resolve these types of
issues (assuming I understand your issue!). If that works for you,
you would have a standards based way to address session management
within OAuth2 use cases.