One reason is that there
has to be a monarch, whereas there does not have to be an heir to a peerage: it is perfectly acceptable for a peerage to become extinct.
The system of male-preference primogeniture which governed the British monarchy until the Succession to the Crown Act developed over centuries precisely to deal with the situation of needing a monarch in the absence of male heirs. It was not initially clearly defined. William the Conqueror was succeeded by two of his sons in turn, William II and Henry I (bypassing his eldest son, Robert Curthose, who succeeded instead as Duke of Normandy until losing even that territory to Henry I). But on Henry I's death in 1135, there were no remaining male-line heirs of William the Conqueror. The uncertainty over who had the right to succeed led to the Anarchy: should the new monarch be the Empress Matilda, Henry I's only surviving child, who had what we would consider the genealogically senior claim but who was a woman; or should it instead be Stephen of Blois, who had a genealogically junior claim (he was the son of one of the daughters of William the Conqueror) but who was a man? Stephen's claim relied on the idea that it was possible for the Crown to pass through a woman (in his case, his mother), but not to a woman. In the event, Matilda was never universally recognised as monarch (and only ever styled herself "Lady of the English", rather than "Queen"), but the Crown ended up passing to her son, Henry II.
The Crown stayed in the male line of the Plantagenets for some time after that, and when Henry IV usurped Richard II in 1399, he bypassed Edmund Mortimer, 5th Earl of March, the senior descendant (but through a female line) of Lionel of Antwerp, Duke of Clarence, elder brother of Henry IV's father, John of Gaunt, Duke of Lancaster. (Both Lionel and and John were sons of Edward III.) That came back to bite the Lancastrian line because the Yorkist line (descendants of Edmund of Langley, Duke of York, a younger brother of Lionel and John) married into the Mortimer line: through this marriage, although their male-line Plantagenet descent was junior to that of the Lancastrians, they had acquired a more senior female-line descent. This was relied on by Richard, Duke of York, and subsequently by his son, who became Edward IV, when challenging and then usurping Henry VI in the Wars of the Roses. But still, at that stage no one was necessarily arguing that a woman could become monarch herself, merely that a claim to the throne could pass through her.
The unambiguous system which existed until the recent legislation did not clearly establish itself until the 16th century, when Mary I and then Elizabeth I acceded upon Edward VI's death leaving Henry VIII with no remaining male heirs. Elizabeth I was then succeeded relatively uncontroversially by James I, again relying on female-line descent (through Margaret, eldest daughter of Henry VII, who had married into the Stewart Kings of Scots; and again through Mary, Queen of Scots, who had married into a junior Stewart/Stuart line). It has succeeded in ensuring that an heir always exists, and has avoided issues like the throne passing to a distant line due to the male-line extinction of the main branch, as happened e.g. in France on the death of Henry III in 1589.
On the contrary, what most commentators do not seem to realise is that hereditary peerages were (except in rare cases, like the Dukedom of Marlborough) effectively designed to have at least a high risk of becoming extinct. Monarchs (and governments once the constitutional monarchy developed) have always wanted to be able to create peers, but also have good reason for not wanting the number of peers to keep on increasing unmanageably. Limiting descent to heirs male ensures that the Peerage is continually thinned out by extinction. I don't know how many hereditary peers there would now be if every peerage had been created with limitation to heirs general without division, but I would guess many thousands.
Of course, we are not in that position now, as no new hereditary peerages are being created. One might say "well, then, now that extinctions are no longer desirable, why not allow titles to survive by passing to female heirs?". But all that does is continue to limit peerages to those in the already established noble families. If it is a laudable aim to preserve the numbers of the Peerage through generations to come, wouldn't it make much more sense instead to resume the creation of hereditary peerages, once again establishing the Peerage as something to which everyone can aspire, rather than simply making it easier for existing noble families to extend their status as such?