I do not know if you have found an answer to this, but a crude way of doing this would be to iteratively generate random data using beta distributions where a and b vary. Then test each generated distribution against your random data using a Kolmogorov-Smirnov test.
The Kolmogorov-Smirnov test is called using ks.test([your data], [generated data OR a cumulative distribution function])
This should at least give you a range of values for a and b within which your data falls.
Hope this helps,
Varun