If you are streaming a video to red5, you can use any of the supported codecs. Look up on the red5 GitHub page.
You can then record it to flv from there. Irrespective of the codec used - h264 + speex or h264 + aac, or Sorenson + nellymoser etc you will end up saveing into a flv container. If you need a mp4 you need to post process it.
Another neat trick is that if you have 2 or 4 streams at a time max, you can use ffmpeg to live record the stream as mp4 or anything you want while transcoding the live stream on the fly.