Overstating the problem a lot, unless your farm is so immense that to load each corner is to fully unload all other corners. I've only experienced it as an edge case, so perhaps I've not seen exactly what you've seen. However, this might be worth looking in to. I'm guessing this is Realistic Biomes (still getting acquainted with the code behind the mod soup we've got going on here)?
Can you also fully describe a reproducible setup that reliably illustrates the issue, including plant type, orientation, size in chunks, location of player, etc. so others can reproduce and confirm?