Putting '{{$:/core/images/spiral}} description’ in the caption field should in fact work, and it’s a bug that it doesn’t.
The problem lies in the template $:/core/ui/EditorToolbar/stamp-dropdown which builds the dropdown for the stamp tool. To display the description of each stamp it currently uses:
<$view tiddler=<<snippetTitle>> field="caption" mode="inline">
<$view tiddler=<<snippetTitle>> field="title" mode="inline"/>
</$view>
The <$view> widget displays a field as plain text. If the field doesn’t exist then the content of the <$view> widget is displayed instead.
However, the convention in TiddlyWiki is to transclude the caption field, rather than just viewing it as plain text. Transcluding the content of the field causes wikitext such as the image to work normally.
I’ve fixed the problem here:
The fix will be live on the prerelease in about 10 minutes:
We don't recommend using the prerelease for anything important, though. Until v5.1.22 is released you’d probably be better off using the Unicode trick (which does in any case carry some advantages).
Best wishes
Jeremy.