Yes (kind of).
The top section in PSI is the CrUX data and is what really matters in our opinion. It's what real users experienced. If they're happy, then that's what's important.
The bottom section is a diagnostic run from Lighthouse under very specific conditions—that may be slower than your typical real user. Maybe all your real users have much better devices or networks? Maybe they are often repeat visitors with part of the site cached so it's faster than the cold load Lighthouse does.
So in that sense, yes if real users are mostly happy then don't panic over the diagnostic test.
In the other sense though, the diagnostic test can highlight potential issues and further improvements even if it is slower than the typical user to your website. Maybe if you're close to the Core Web Vitals thresholds then you might want a bit more head room so you can use that diagnostic data to help you get there. Or maybe you want your website experience to be excellent even for users who happen to use much slower devices (maybe you're missing out on such users who just abandon your site because it's too slow for them?). Otherwise a slight shift in traffic could easily knock you into the poor categories.
Also when you have a huge discrepancy (2 seconds - 10 seconds is a lot more than we usually see!), then it might be worth understanding why. One reason could be the page you're testing (your home page I presume?) is really slow, but the "Real User section" doesn't have enough data for that specific page and so falls back to the origin (see screenshot below). And that slow page is being masked by lots of faster pages so overall your site looks fast. That would also explain why it looks good in Search Console overall which similarly groups pages.
So yes real users are what matters. But the slower lab test can still be useful data (hence why we show it!), so don't ignore it completely.