First the API you mention above will not set the folder icon. Its returning a dictionary with the current icon in the kGMUserFileSystemCustomIconDataKey key.
Second if you include the cocoa libraries and link against the cocoa library you should be able to call the function from C with out any issues. If thats not what you want, then I am at a loss to help. Try search carbon headers and hope that apple has not deprecated with no replacement functions.