Run Button greying out

[Darrell Smith]

Hi

Since lunch time I've found that the run button is greying out and an oops something went wrong error message is displaying.

The run button reappears after 6 - 12 refreshes but then goes again after a couple of runs

Darrell

[Zarino Zappia]

Seems to be working for me.

Can anyone else confirm?

What browser are you using, Darrell?

Zarino Zappia

VP of Product, ScraperWiki

@zarino

--
You received this message because you are subscribed to the Google Groups "ScraperWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scraperwiki...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Darrell Smith]

I'm using chrome.

[Francis Irving]

Strange - if it happens again, can you copy and paste the error
message?

Meanwhile, most of our development attention is on the new Beta
ScraperWiki.

https://beta.scraperwiki.com/

Please do try it out!

Francis

[Darrell Smith]

I've tried the beta version but I laways get 

PHP Fatal error:  Class 'scraperwiki' not found in /home/code/scraper on line 6

When trying to copy over my old code and I haven't had time to fix them yet.

Is there some kind of migration tool or some docs on the best methods?

[Francis Irving]

Ah, there isn't a PHP compatibility library, at least yet!

I think one could be made on top of ReadBean PHP fairly easily.
http://www.redbeanphp.com/

Which functions are you using? save_var, get_var, save and select it
looks like.

It's probably worth making if somebody has a large number of PHP
scrapers on ScraperWiki that they care about.

The Kohovolit people in the Czech Republic probably have the most.
https://scraperwiki.com/tags/kohovolit.eu

Francis

> > an email to scraperwiki...@googlegroups.com <javascript:>.

[Darrell Smith]

That explains it then :)

I'll take a look at redbean when I've a bit of free time. 

I'm only using the basic functions so hopefully it wont be too much to flip them over to the beta version

cheers

D

[ph...@civicagency.org]

I have about 30 php scrapers in classic that I would really like to migrate. In fact, I already paid for a premium account before I realized the php class was no longer available. I'm wondering if there's anyone who can help point me in the right direction to start using Redbean to connect to the datastore. If I can wrap my head around it enough I can try to build a new library, but can't promise anything. At the very least I'd just like to get sense of what options there are to read/write to the datastore from php, so any pointers would be much appreciated!

[ph...@civicagency.org]

I'd still appreciate any input on this I can get, but I did finally notice a little snippet of sample code when you create a fresh new php tool in the browser editor. Since I didn't see this anywhere that could be captured by I search engine, I figured I'd paste it here for others who might be searching like I was:

// RedBeanPHP makes the database table for you http://redbeanphp.com/
require 'rb.php';

R::setup('sqlite:scraperwiki.sqlite');
$animal = R::dispense('animal');
$animal->name = 'Violet';
$animal->age = 7;
$id = R::store($animal);

[Francis Irving]

Yes, the below is as far as I got!

I know the people from Kohovolit.eu (Czech Parliamentary hackers) will
need it too.

There's a Github repository where this should go.
https://github.com/scraperwiki/scraperwiki-php
I guess in a "scraperwiki.php" file.

What's your Github username Phil, and I can add you to it.

Francis

[Philip Ashlock]

Thanks. I'm philipashlock on github.

I'm getting more familiar with redbean, but it seems like I might still need to write sql like this

                    R::exec('INSERT or UPDATE INTO ? (?) VALUES (?) WHERE ?',$parameters);

You received this message because you are subscribed to a topic in the Google Groups "ScraperWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/scraperwiki/QR3J4KVQg5M/unsubscribe.
To unsubscribe from this group and all its topics, send an email to scraperwiki...@googlegroups.com.

[Francis Irving]

OK, I've given you write access to that repository.

There's a discussion about upserts in redbean here...
https://github.com/gabordemooij/redbean/issues/160

With some hacks to do it.

Francis

[Philip Ashlock]

Thanks. I saw that discussion about upserts, but unless I misunderstand something it still doesn't seem like there's a better way to do it than this - which is maybe too sqlite specific?

        $parameters['table_name'] = $table_name;
        $parameters['keys'] = join(", ", array_keys($data));
        $parameters['values'] = join(', ', array_fill(0, count($data), '?')); // adds the ? placeholder for values

        $sql = vsprintf('INSERT or REPLACE INTO %s (%s) VALUES (%s)', $parameters);
        R::exec($sql,array_values($data));

I checked in a first pass at the most basic functionality of a new scraperwiki.php at:
https://github.com/philipashlock/scraperwiki-php/blob/master/scraperwiki.php#L30

I'll see if I can start refactoring/migrating more from the old version, but if anyone has any feedback/contributions it'd be much appreciated. I'm not sure I feel too confident I can get a fully production ready version of this done by myself.

[Philip Ashlock]

I think I've actually been able to implement most of the main features of the classic scraperwiki.php using RedBean now - https://github.com/philipashlock/scraperwiki-php/blob/master/scraperwiki.php

[Philip Ashlock]

So I've started using this on my own scrapers and it seems to be working well enough, but it is kind of annoying to have to include this whole block of code in each scraper rather than have it included automatically as it was on Scraperwiki Classic. Are there any special considerations I should make to get this ready to be officially included or should I just go ahead and submit a pull request?

On Sunday, September 22, 2013 3:04:52 AM UTC-4, Philip Ashlock wrote:

I think I've actually been able to implement most of the main features of the classic scraperwiki.php using RedBean now - https://github.com/philipashlock/scraperwiki-php/blob/master/scraperwiki.php

> > an email to scraperwiki+unsubscribe@googlegroups.com.

> > > For more options, visit https://groups.google.com/groups/opt_out.
> >
> > --
> > You received this message because you are subscribed to a topic in the
> > Google Groups "ScraperWiki" group.
> > To unsubscribe from this topic, visit
> > https://groups.google.com/d/topic/scraperwiki/QR3J4KVQg5M/unsubscribe.
> > To unsubscribe from this group and all its topics, send an email to

> > scraperwiki+unsubscribe@googlegroups.com.

> > For more options, visit https://groups.google.com/groups/opt_out.
> >
>
> --
> You received this message because you are subscribed to the Google Groups "ScraperWiki" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to scraperwiki+unsubscribe@googlegroups.com.

> For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the Google Groups "ScraperWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/scraperwiki/QR3J4KVQg5M/unsubscribe.

To unsubscribe from this group and all its topics, send an email to scraperwiki+unsubscribe@googlegroups.com.

[Francis Irving]

Yep, go ahead and submit a pull request!

On Thu, Oct 03, 2013 at 05:10:24PM -0700, Philip Ashlock wrote:
> So I've started using this on my own scrapers and it seems to be working
> well enough, but it is kind of annoying to have to include this whole block
> of code in each scraper rather than have it included automatically as it
> was on Scraperwiki Classic. Are there any special considerations I should
> make to get this ready to be officially included or should I just go ahead
> and submit a pull request?
>
> On Sunday, September 22, 2013 3:04:52 AM UTC-4, Philip Ashlock wrote:
> >
> > I think I've actually been able to implement most of the main features of
> > the classic scraperwiki.php using RedBean now -

> > https://github.com/philipashlock/scraperwiki-php/blob/master/scraperwiki.php<https://github.com/philipashlock/scraperwiki-php/blob/master/scraperwiki.php#L30>

> >>> > > You received this message because you are subscribed to a topic in
> >>> the
> >>> > > Google Groups "ScraperWiki" group.
> >>> > > To unsubscribe from this topic, visit
> >>> > >
> >>> https://groups.google.com/d/topic/scraperwiki/QR3J4KVQg5M/unsubscribe.
> >>> > > To unsubscribe from this group and all its topics, send an email to

> >>> > > scraperwiki...@googlegroups.com.
> >>> > > For more options, visit https://groups.google.com/groups/opt_out.
> >>> > >
> >>> >
> >>> > --
> >>> > You received this message because you are subscribed to the Google
> >>> Groups "ScraperWiki" group.
> >>> > To unsubscribe from this group and stop receiving emails from it, send
> >>> an email to scraperwiki...@googlegroups.com.
> >>> > For more options, visit https://groups.google.com/groups/opt_out.
> >>>
> >>> --

> >>> You received this message because you are subscribed to a topic in the
> >>> Google Groups "ScraperWiki" group.
> >>> To unsubscribe from this topic, visit
> >>> https://groups.google.com/d/topic/scraperwiki/QR3J4KVQg5M/unsubscribe.
> >>> To unsubscribe from this group and all its topics, send an email to

> >>> scraperwiki...@googlegroups.com.
> >>> For more options, visit https://groups.google.com/groups/opt_out.
> >>>
> >>
> >>
> >
>
> --
> You received this message because you are subscribed to the Google Groups "ScraperWiki" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to scraperwiki...@googlegroups.com.

[Francis Irving]

Looks really good!

I've pulled it into scraperwiki/scraperwiki-php

I'm finding on the first run it doesn't seem to save, then does second
time. Any idea why?

Francis

On Thu, Oct 03, 2013 at 05:10:24PM -0700, Philip Ashlock wrote:

> So I've started using this on my own scrapers and it seems to be working
> well enough, but it is kind of annoying to have to include this whole block
> of code in each scraper rather than have it included automatically as it
> was on Scraperwiki Classic. Are there any special considerations I should
> make to get this ready to be officially included or should I just go ahead
> and submit a pull request?
>
> On Sunday, September 22, 2013 3:04:52 AM UTC-4, Philip Ashlock wrote:
> >
> > I think I've actually been able to implement most of the main features of
> > the classic scraperwiki.php using RedBean now -

> > https://github.com/philipashlock/scraperwiki-php/blob/master/scraperwiki.php<https://github.com/philipashlock/scraperwiki-php/blob/master/scraperwiki.php#L30>

> >>> > > You received this message because you are subscribed to a topic in
> >>> the
> >>> > > Google Groups "ScraperWiki" group.
> >>> > > To unsubscribe from this topic, visit
> >>> > >
> >>> https://groups.google.com/d/topic/scraperwiki/QR3J4KVQg5M/unsubscribe.
> >>> > > To unsubscribe from this group and all its topics, send an email to

> >>> > > scraperwiki...@googlegroups.com.
> >>> > > For more options, visit https://groups.google.com/groups/opt_out.
> >>> > >
> >>> >
> >>> > --
> >>> > You received this message because you are subscribed to the Google
> >>> Groups "ScraperWiki" group.
> >>> > To unsubscribe from this group and stop receiving emails from it, send
> >>> an email to scraperwiki...@googlegroups.com.
> >>> > For more options, visit https://groups.google.com/groups/opt_out.
> >>>
> >>> --

> >>> You received this message because you are subscribed to a topic in the
> >>> Google Groups "ScraperWiki" group.
> >>> To unsubscribe from this topic, visit
> >>> https://groups.google.com/d/topic/scraperwiki/QR3J4KVQg5M/unsubscribe.
> >>> To unsubscribe from this group and all its topics, send an email to

> >>> scraperwiki...@googlegroups.com.
> >>> For more options, visit https://groups.google.com/groups/opt_out.
> >>>
> >>
> >>
> >
>
> --
> You received this message because you are subscribed to the Google Groups "ScraperWiki" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to scraperwiki...@googlegroups.com.