I this that the behavior is correct. We are really wanting to test if the element is visible to the user.
You should use the other keyword, Page Should Contain Element.
(Selenium/SeleniumLibrary is a tool to "simulate" the real user usage of the browser, not like the new tools, Cypress and Playwright)