Testing updates

[Kevin Campbell]

The LookupError which was raised earlier is due to a previous test (test_alias_queue) processing items, resulting in the in-memory queues getting populated. Those items were then removed from the database, but the in-memory queues not cleared. I've updated the test_alias_queue code now to clear the queues.

I've updated Jenkins to pull in the secrets.py during the build time. I've also added in missing items to the providers_test_proxy command. Heather, I've had to restore the old test/data directory, as the replacement in extras wasn't covering the same things. In some cases there is more than one item per provider. The old files are pretty messy, but restoring was the quickest way I could get things working again.

The issues with the recent failures failing to parse XML are due to the BeautifulSoup version.

======================================================================
FAIL: test.unit_tests.providers.test_crossref.TestCrossRef.test_extract_biblio
----------------------------------------------------------------------
Traceback (most recent call last):
 File "/usr/lib/pymodules/python2.6/nose/case.py", line 183, in runTest
   self.test(*self.arg)
 File "<http://orchid.vm.bytemark.co.uk:8500/job/total-impact/ws/test/unit_tests/providers/test_crossref.py",> line 37, in test_extract_biblio
   assert_equals(biblio, expected)
AssertionError: {'authors': 'None, None, None, None', 'journal': None, 'title': None, 'year': None} != {'journal': u'PLoS ONE', 'title': u'Sharing Detailed Research Data Is Associated with Increased Citation Rate', 'year': u'2007', 'authors': 'Piwowar, Day, Fridsma, Eisen'}


I've moved orchid from 3.1.0.1 to 3.2.1 now, which have resolved this. BeautifulSoup4 is also available as bs4. I would recommend using XML parsers for XML though where possible, it will probably be more robust in the long term.