site request

[scout78]

Hi!

I was wondering if you'd would be willing to add support for alldanielfic.com and csi-forensics.com. I've already tried to write these adapters myself, but since both sites don't seem to use the standard div containers, my meager programming skills haven't gotten me very far.

~scout

[Jim Miller]

On 9/27/2014 11:20 AM, scout78 wrote:
> I was wondering if you'd would be willing to add support for

> alldanielfic.com <http://alldanielfic.com> and csi-forensics.com
> <http://www.csi-forensics.com/index.php>. I've already tried to write

> these adapters myself, but since both sites don't seem to use the
> standard div containers, my meager programming skills haven't gotten me
> very far.

If it's not working out to create a regular adapter, you might try
making one using the eFiction Base adapter.

For examples, look at:
adapter_fannation.py
adapter_themaplebookshelf.py

They both use base_efiction_adapter.py. It doesn't collect metadata as
well as a fully custom adapter, but it may be good enough.

[scout78]

On Saturday, September 27, 2014 6:56:59 PM UTC+2, Jimm wrote:

If it's not working out to create a regular adapter, you might try
making one using the eFiction Base adapter.

For examples, look at:
adapter_fannation.py
adapter_themaplebookshelf.py

They both use base_efiction_adapter.py.  It doesn't collect metadata as
well as a fully custom adapter, but it may be good enough.

As you suggested, I tried using the eFiction Base adapter for both sites but I got mixed results.

csi-forensics.com: The adapter seems to work fine so far, but for some reason it doesn't extract the rating from the metadata (probably because the rating is not in a <span>).

example URL: http://www.csi-forensics.com/viewstory.php?sid=4518
the code: https://docs.google.com/file/d/0B84VOuT8FeFMSzRKQlZNMXp1QkE/

alldanielfic.com: That one doesn't work at all. It already fails to extract the title of the story because here, the title isn't a link.

example URL: http://alldanielfic.com/viewstory.php?sid=31
the code: https://drive.google.com/file/d/0B84VOuT8FeFMbDZGNFpBNmtYQ2c/

Is there by any chance some kind of tutorial for the eFiction Base adapter?

[Jim Miller]

On 9/27/2014 2:27 PM, scout78 wrote:
>
> As you suggested, I tried using the eFiction Base adapter for both sites
> but I got mixed results.

That's not a surprise. It's an attempt to make a 'one-size fits all'
solution to all the eFiction sites--and that's a highly customizable
system. So if the rating is the only thing missing, that's pretty much
as good as the example sites.

> Is there by any chance some kind of tutorial for the eFiction Base adapter?

Not really. The author of that part, John Doe, popped up for about a
week last month, added the eFiction Base adapter and then disappeared again.

Jim

[scout78]

On Saturday, September 27, 2014 11:11:41 PM UTC+2, Jimm wrote:

Not really.  The author of that part, John Doe, popped up for about a
week last month, added the eFiction Base adapter and then disappeared again.

That's a shame, but I guess sooner or later I would have had to learn some Python and how to use BeautifulSoup anyway.

In the meantime I have 2 more adapters for you, if you want them, and I've also updated the sheppardweir adapter with the review counts.

csiforensicscom: https://docs.google.com/file/d/0B84VOuT8FeFMTVdhdDhyN212eWc/
samandjacknet: https://docs.google.com/file/d/0B84VOuT8FeFMdEg4QUNMZ0Fha28/
sheppardweircom: https://docs.google.com/file/d/0B84VOuT8FeFMZjlIRURPdlFOQ1k/

The csiforensics adapter is still missing the rating & categories, I have however figured out that Wraithbait uses a similiar layout. However, I have no idea how to get the appropriate lines of code to work with the eFiction Base adapter:

        pt = soup.find('div', {'id' : 'pagetitle'})
        rating=pt.text.split('[')[1].split(']')[0]
        self.story.setMetadata('rating', rating)


        info = soup.find('div', {'class' : 'small'})
        cats = info.findAll('a',href=re.compile(r'browse.php\?type=categories&id=\d'))
        for cat in cats:
                self.story.addToList('category',cat.string)

Could you maybe take a quick look and tell me if it's even possible to get that to work?

Thanks.
~scout

[Jim Miller]

On 9/28/2014 5:02 PM, scout78 wrote:
> In the meantime I have 2 more adapters for you, if you want them, and
> I've also updated the sheppardweir adapter with the review counts.

Sure, I'll include them in the next version.

> The csiforensics adapter is still missing the rating & categories, I
> have however figured out that Wraithbait uses a similiar layout.
> However, I have no idea how to get the appropriate lines of code to work
> with the eFiction Base adapter:

There's actually a bug in base_efiction_adapter.py--it should be setting
'category' not 'categories'. (It's my fault, really, for not being
consistent with singular vs plural names.)

But csiforensics also needs custom handleMetadataPair code because it
uses '>' to separate instead of ','.

It's not really possible to get the rating for csiforensics using
eFiction Base adapter because csiforensics doesn't include that
information in the print page that the eFiction Base adapter gets all
its data from. It would have to be a regular adapter instead. (Or
extend extractChapterUrlsAndMetadata to fetch another page and parse it
from there, I suppose.)

I've posted a test version of the plugin with these adapters, including
'Categories code' for csiforensics:

http://www.mobileread.com/forums/showthread.php?p=2936068#post2936068

Jim

[scout78]

On Monday, September 29, 2014 6:04:03 PM UTC+2, Jimm wrote:

There's actually a bug in base_efiction_adapter.py--it should be setting
'category' not 'categories'.  (It's my fault, really, for not being
consistent with singular vs plural names.)

But csiforensics also needs custom handleMetadataPair code because it
uses '>' to separate instead of ','.

Thank you for taking a look at it. What's kinda interesting for me to see, is that you added the same code that I finally came up with in the wee hours of the morning. Only, you probably didn't need 3-4 hours of trial and error do do it. Maybe there's hope for me, yet. ;)

It's not really possible to get the rating for csiforensics using
eFiction Base adapter because csiforensics doesn't include that
information in the print page that the eFiction Base adapter gets all
its data from.  It would have to be a regular adapter instead.  (Or
extend extractChapterUrlsAndMetadata to fetch another page and parse it
from there, I suppose.)

Ah, so that's why I kept running into a wall. I guess turning csiforensics into a regular adapter can be my next project then. It just might take me a while. I've still got a lot of learning to do.

Again, thank you so much for your help.
~scout