While you can override and set permissions at an individual Pipeline Stage level, pipeline groups are the main way to control who can see and operate pipelines. I believe the permissions here just override who can "operate" (trigger) the stage.
Pipeline stage permissions are not very easy to maintain, and are mainly used for special situations like "I want only special people/roles to have permissions to trigger the "deploy-to-prod" stage, but other stages anyone in the role can operate".
So you are usually best to model your pipeline groups around teams, rather than services in a situation where you have the same service worked on with different pipelines by different teams that you don't want to see each other.
Are you using pipelines-as-code, APIs or the user interface to manage/define your pipelines/pipeline groups?
-Chad