Hi Neeraj,
This is a great question and one I cover with my students in my classes. I break the development of this encoding into a few steps. First, note how the opcode field is consistent across all instruction classes. Then note the rd, rs1, and rs2 fields always occupy the same bits of the instruction, whenever they are present. This situation helps with the instruction decoding and the design of the control logic for the read and write ports of the register file. However, these conditions quickly begin to place restrictions on the encoding of the immediates.
At this point, I have the students recall how the "sign"-bit of a two's complement number is critical in sign-extending a two's complement number. I then ask them to note where the msb is for each of the immediate fields in each instruction class. The surprising, and clever, result is that the msb of each immediate field is always in bit 31 of an instruction. This can be used to simplify the logic that sign extends the immediates.
Finally, I point out that the tendency of us to want to keep all of the immediate bits grouped together in a more human readable form, is certainly not necessary, and can even make our design and implementation more complex.
I hope this is helpful. Let me know if you have any questions.
Ray Simar
Professor in the Practice
Department of Electrical and Computer Engineering
Rice University