I do see your point Melvyn but my use case only requires to save different types of geometries and visualize them. Anything more usefull is just extra. This works fine when i build the map myself but the OpenLayersWidget has this geomtype restriction.
The complexity in saving geometries of different type to different fields instead of one comes from the additional logic of checking the geometry type and choosing the field/model accordingly, retrieving the geometry requires some additional logic, require one geometry to exist but only one --> additional logic.... I also agree that this might not be too hard or too complex but as the django website tells me "a framework for perfectionist with deadlines" where I a perfectionist on meeting deadlines feel like doing it the simplest way possible. But for now I solved the issue creating my own view and adding a link "view geometry" to the change list (the work amount mostly copy paste old stuff and simple examples).
Thanks for your help! At least now I know that the GeometryField is not supposed to work in the admin and the reasons behind it = we cannot do anything usefull with it.