I think it should obey the registrar. In some sense it should cascade delete the entire gradebook for the semester that the course is no longer related to. There is no warning for this, so just having the grades orphaned is probably fine.
In such an event though the teacher should no longer have any way to see those orphaned grades exists.