King,
The factor used to identify your values is made in the ggplot2 call and does not contain the levels that you are assigning for the colors so those extra levels are ignored. Best to define these from the beginning and use them.
Data %>%
mutate(value = lvls_expand(as.factor(value), as.character(0:6))) %>%
ggplot(., aes(x=sample, y=gene)) +
geom_tile(aes(fill=value), colour = "black") +
scale_fill_manual(values = colvalue, drop=FALSE)
forcats::lvls_expand makes adding those levels pretty easy, but needs to be a character vector. You still need drop = FALSE in the scale_*_manual() call.
HTH,
B