Comments on the documentation for BeautifulSoup

[Robert Morelli]

I'd like to bring to your attention some small frustrations I encountered learning BS from the standard documentation. For context, much of my experience programming is with languages that have a more formal and standardized style of documentation than Python,  compared to Python's more chatty and example based style, so that is probably contributing to my confusion.

I'll focus, as an example, on one specific issue that came up, which is representative.

In the section on "find_all()", first of all, there is no visual cue as to what level of the document we are at. "find_all()" appears in my browser as bold red text, about the same size as an <h3> heading. I realize in the HTML, it's actually an <h2>, but that's not visually apparent, because there's no other text in the heading. Also, the navigation sidebar is a bit flaky. I'm not nitpicking. I actually find it difficult to navigate the document, and frequently lose my place.

Then, the signature of the method is given as 

Signature: find_all(name, attrs, recursive, string, limit, **kwargs)

Since no default values are indicated, I assume (as a relatively inexperienced Python programmer), that these are positional arguments. But, the examples make it clear they aren't.

My confusion is intensified by one of the examples:

soup.find_all("p","title")

If "title" is a positonal argument, it is the value of attrs. Now, attrs doesn't have a dedicated section like the other arguments (which are also presented out of order). It's seems, from poking around, that attrs is expected to be a dictionary object. But then, "title" makes no sense as the value of attrs.

Sure enough, the documentation poses the rhetorical question, "Why does find_all("p", "title") find a <p> tag with the CSS class “title”? Let’s look at the arguments to find_all()." Good question. However, as far as I can tell, it's never answered. But, even if it is, and I missed it, I really don't like that kind of rhetorical question in documentation. In my experience, as often as not, when a question like that appears in documentation, the author forgets to actually answer it. But, even if they do, I don't read documentation for enjoyment. I'm trying to get the information as quickly as possible, so I appreciate a link to where it's answered, if it isn't immediate.

After a while, I decided to look at the archived documentation for BS 3, on the assumption that this might be a vestige of a deprecated feature, even though BS 3 seems to have been phased out 6 years ago. Indeed, this seemed to answer the question, but not in the way I was expecting. 

First, the BS 3 documentation gives the signature of find_all() with their default values. That was reassuring. Then, it stated explicitly below: "You could search by CSS class with soup.find("tagName", { "class" : "cssClass" }), but that's a lot of code for such a common operation. Instead, you can pass a string for attrs instead of a dictionary. The string will be used to restrict the CSS class."

So, that resolved the mystery. But, that explanation, as far as I can tell, was dropped from the BS 4 documentation. I don't understand why. If this is a deprecated feature, fine, then remove the example. My personal preference would be for this feature to be deprecated and removed from BS, since it only saves a small amount of typing at the risk of creating a lot of confusion. But, that's your all. If this is a kosher feature, then it should be explained.

In any case, this example is typical of the difficulty I've had learning BS. It probably reflects my preference for languages that are more explicit than Python, but I am, after, not the only person to learn Python as a second language.