You can navigate between entries without leaving the quickfix list using :cnext and :cprev commands. Both of these take a count if you want to skip a few entries. But going to a specific entry in the middle of the list somewhere would be difficult without a mapping-based solution like John and Karthick gave.