You could mark the "builds" subdirectories as ignored in source control?
Or if you go to Manage Jenkins > Configure System and click "Advanced"
at the top of the page, you'll be able to alter where build data is
stored ("Build Record Root Directory").
If you change this to "${JENKINS_HOME}/builds/${ITEM_FULLNAME}", for
example, then all of the build data will be stored there.
At this point, the only thing left in ${JENKINS_HOME}/jobs/<name> (i.e.
${ITEM_ROOTDIR}) will be the job config.xml, and the metadata about
build numbers and last success/failure symlinks etc..
Note that changing this value will not automagically migrate all of your
existing data to the new location you've specified.
Regards,
Chris