Phirehose proposed development roadmap & RFC

91 views
Skip to first unread message

Fenn Bailey

unread,
Jul 21, 2011, 9:54:24 PM7/21/11
to phireho...@googlegroups.com
Hey Guys,

First up, I'd like to thank everyone who's contributed to Phirehose to help make it better/more useful/etc - I haven't had a lot of time to personally contribute large amounts of updates to it, so it's fantastic to see the community pick it up and continue development.

I thought it would be useful to let everyone know how I'd like to see it evolve and what I think is a good direction.

The main features that are obviously missing and should be implemented/unified are:
  • OAuth based authentication
  • Heartbeat notifications
  • UserStreams
  • SiteStreams
Some of this has been implemented via patches but I think it'd be great to get it rolled up and unified into a single authoritative version. My initial thoughts on how to do this are:

  1. Refactor the base Phirehose class into an abstract BasePhirehose class and main Phirehose class;
  2. Add support for OAuth & heartbeat into the base class as a core features;
  3. Refactor all common code (network connectivity, logging, reconnects, etc) into the BasePhirehose class, leaving only the filter and sample implementations in Phirehose;
  4. Create UserStreamPhirehose and SiteStreamPhirehose subclasses of BasePhirehose to support associated streams (and methods);
The idea with this is that it should be very clear/simple how to connect to each type of stream, ie:

$myPublicStream = new Phirehose(Phirehose::METHOD_SAMPLE);
$myUserStream = new UserStreamPhirehose(...);
$mySiteStream = new SiteStreamPhirehose(...);

Naturally, each stream type can/should be subclassed appropriately to let developers consume the stream type, ie:

class MyFunkyWebsiteStream extends SiteStreamPhirehose {
   ...
}

$myStream = new MyFunkyWebsiteStream();
$myStream->consume();

So, 3 questions:
  1. How does that sound to the existing Phirehose users out there in terms of providing a platform they can build on?
  2. Does the solution sound suitable/any comments/suggestions?
  3. Who feels like they have the skills and time to help undertake the work described?
Cheers!

  Fenn.

Adam Green

unread,
Jul 21, 2011, 10:00:12 PM7/21/11
to phireho...@googlegroups.com
Fenn, I'd like to add the idea of a support library of useful
functions. One function I need badly is a solid linkifier based on
tweet entities that can handle UTF8 correctly. This seems like it is
needed by lots of Twitter display sites. I know it is fine line
keeping this focused on collecting API streams, but every tweet
collected needs to be linkified as well. Maybe there can be a plugin
architecture for this type of enhancement.

--
Adam Green, 140...@gmail.com
Twitter API Consultant and Analyst
Developer of the First Presidential Twitter Debate
http://140TownHall.com
http://140dev.com, @140dev
http://2012twit.com, @2012twit
781-879-2960

Faustino Forcén

unread,
Jul 22, 2011, 2:03:24 AM7/22/11
to phireho...@googlegroups.com
El 22/7/11 04:00, Adam Green:


I agree with the suggested roadmap. I'd add more detailed documentation and
samples for beginners. I could help with that.


As for the linkifier needed by Adam, here's the function I use for the task.
Notices two things:
- you need to have PCRE compiled with full unicode support
- In my service I include in hashtags more characters than the officially
supported. Adjust the regexp to your needs.


/**

linkify function with support for utf-8 characters.

@param $text is the raw text of the tweet
@returns the same tweet with links, users and hashtags linkified

warning: you need to have PCRE compiled with "Unicode properties support"
run pcretest -C on your console. the result should include this lines:

Compiled with
UTF-8 support
Unicode properties support

*/

function procesaTwit($text) {
return preg_replace(array(
'@(https?://([-\w\.]+)+(/([\w/_\.\-]*(\?\S+)?(#\S+)?)?)?)@',
'/@(\w+)/',
'/#([\d\w\/\pL]+)/',
),

array( '<a href="$1" target="_blank">$1</a>',
'<a href="http://twitter.com/$1">@$1</a>'
'<a
href="http://twitter.com/#!/search?q=%23$1">#$1</a>'
),

$text);
}

ff

Gene

unread,
Jul 24, 2011, 3:10:53 PM7/24/11
to Phirehose Users
This would be an amazing upgrade to Phirehose!

On Jul 21, 8:54 pm, Fenn Bailey <fenn.bai...@gmail.com> wrote:
> Hey Guys,
>
> First up, I'd like to thank everyone who's contributed to Phirehose to help
> make it better/more useful/etc - I haven't had a lot of time to personally
> contribute large amounts of updates to it, so it's fantastic to see the
> community pick it up and continue development.
>
> I thought it would be useful to let everyone know how I'd like to see it
> evolve and what I think is a good direction.
>
> The main features that are obviously missing and should be
> implemented/unified are:
>
>    - OAuth based authentication
>    - Heartbeat notifications
>    - UserStreams
>    - SiteStreams
>
> Some of this has been implemented via patches but I think it'd be great to
> get it rolled up and unified into a single authoritative version. My initial
> thoughts on how to do this are:
>
>    1. Refactor the base *Phirehose* class into* *an abstract *BasePhirehose
>    *class and main *Phirehose* class*;*
>    2. Add support for OAuth & heartbeat into the base class as a core
>    features;
>    3. Refactor all common code (network connectivity, logging, reconnects,
>    etc) into the *BasePhirehose* class, leaving only the *filter *and *
>    sample* implementations in *Phirehose*;
>    4. Create *UserStreamPhirehose* and *SiteStreamPhirehose* subclasses of *
>    BasePhirehose* to support associated streams (and methods);
>
> The idea with this is that it should be very clear/simple how to connect to
> each type of stream, ie:
>
> *$myPublicStream = new Phirehose(Phirehose::METHOD_SAMPLE);*
> *$myUserStream = new UserStreamPhirehose(...);*
> *$mySiteStream = new SiteStreamPhirehose(...);*
> *
> *
>
> Naturally, each stream type can/should be subclassed appropriately to let
> developers consume the stream type, ie:
>
> *class MyFunkyWebsiteStream extends SiteStreamPhirehose {*
> *   ...*
> *}*
> *
> *
> *$myStream = new MyFunkyWebsiteStream();*
> *$myStream->consume();*
> *
> *
>
> So, 3 questions:
>
>    1. How does that sound to the existing Phirehose users out there in terms
>    of providing a platform they can build on?
>    2. Does the solution sound suitable/any comments/suggestions?
>    3. Who feels like they have the skills and time to help undertake the
>    work described?
>
> Cheers!
>
>   Fenn.
Reply all
Reply to author
Forward
0 new messages