Divide Legend into Two Columns, Each with Its Own Title

[eipi10]

I recently worked on a Stack Overflow question in which the poster wanted to have the legend in two columns, controlling which factor levels were in each column, and also putting a separate legend title over each column. I was able to get a custom split of the factor levels by adding a dummy factor level. However, I had two issues in trying to create separate legend titles. First, I had to resort to grob manipulation, which worked, but which required a lot of extra code and trial and error placement of the grobs. Second, even though I turned off clipping for all pre-existing grobs, the titles I created with the text grobs are still clipped by the pre-existing legend grob.

So, my questions are:
1) Is there a way to do this within ggplot2, without having to resort to grobs manipulation?
2) If not, is there an easier way to create and position the text grobs for the custom legend titles?
3) Why is the legend still clipping the text grobs and how can I fix this so that the text grobs are "on top" of the legend grob?

Please feel free to post an answer on Stack Overflow in addition to any response you provide here. Alternatively, I'll be happy to add any responses I receive here to my SO answer, with credit of course.

Here's the code for the plot, using the diamonds data frame:

library(gtable)

# Add dummy factor level to achieve desired column split in legend
diamonds$cut = factor(diamonds$cut, levels=c("Fair", "Good", " ", "Very Good", 
                                             "Premium", "Ideal"))

# Initial plot with two-column legend and extra margin space between plot area and legend where we'll place
# the two custom legend titles
p = ggplot(diamonds, aes(color, fill=cut)) + geom_bar() + 
  scale_fill_manual(values=c(hcl(seq(15,325,length.out=5), 100, 65)[1:2], 
                             "white",
                             hcl(seq(15,325,length.out=5), 100, 65)[3:5]),
                    drop=FALSE) +
  guides(fill=guide_legend(ncol=2)) +
  theme(legend.position=c(0.5,-0.26),  
        plot.margin=unit(c(1,1,7,1),"lines")) +
  labs(fill="") 

# Add two text grobs
p = p + annotation_custom(
    grob = textGrob(label = "First\nGroup", 
                    hjust = 0.5, gp = gpar(cex = 0.7)),
    ymin = -2200, ymax = -2200, xmin = 3.45, xmax = 3.45) +
  annotation_custom(
    grob = textGrob(label = "Second\nGroup",
                    hjust = 0.5, gp = gpar(cex = 0.7)),
    ymin = -2200, ymax = -2200, xmin = 4.2, xmax = 4.2)

# Override clipping and draw plot
gt <- ggplot_gtable(ggplot_build(p))
gt$layout$clip <- "off"
grid.draw(gt)

Thanks,
Joel