Yes, FAT32 is fine and 8GB is fine (though you'll never fill it). Preferably use the SD Association's SD Card Formatter. Then just copy the files to the SD card on your Windows machine. If you hadn't saved a copy of the files, you can get them from the disks directory of this archive:
https://github.com/dhansel/Altair8800. If you've added additional customized disks, you'll have to find backup copies of them and manually copy them to the SD card, as well as edit the DISKDIR.TXT, HDSKDIR.TXT, TDISKDIR.TXT, and/or CDISKDIR.TXT as appropriate to whatever disks you've added.
If you've modified any of the disks on the corrupted card and don't have a backup, then you'll have lost those modifications.