We generally fill it with terms from the units ontology, but if that's not sufficient we will need either request new terms or look to another source such as QUDT.
Length unit is defined in the unit ontology. If you browser
http://purl.obolibrary.org/obo/UO_0000001 you will see the current subclasses of length unit. Generally if you wanted to use "inch" as a length unit, you would make a term request to UO add it and then use the URI of the new term as an instance in the OWL, the object of the has_measurement_unit_label relation.
The unit ontology doesn't have a coherent ontological commitment as to what a unit is. From an IAO perspective each unit label is a particular (an instance). The unit label and what it refers to are established by social processes - that certain masses are to be called a kilogram was established by a standardization committee.
What makes you uncomfortable about having the object measurement unit label be an information entity . From an ontological standpoint, there are the recordings of measurements which result in information entities, and the recording has as part the name of the unit. One might argue that that part - the unit label - is about (a defined class of qualities?). We haven't done that yet, though it is true in some sense. Instead we say that the combination of the value and the label is information about a quality (length) of the thing that was measured.