[SciPy-User] timeseries forwardfill

[Chris Clarke]

Hi
I haven't used this library in a while but i seem to recall you could
forward fill 2d arrays and set initial starting values etc.??
Am i correct ?? If so, any special reasons were they removed??
Regards
Chris

_______________________________________________
SciPy-User mailing list
SciPy...@scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user

[Pierre GM]

On Nov 18, 2009, at 9:04 AM, Chris Clarke wrote:

> Hi
> I haven't used this library in a while but i seem to recall you could
> forward fill 2d arrays and set initial starting values etc.??
> Am i correct ?? If so, any special reasons were they removed??
> Regards
> Chris

What do you mean, removed ? You can find `forward_fill` in scikits.timeseries.lib.interpolate.
Am I answering your question ?
P.

[Chris Clarke]

Sorry for the later reply. Yes forward_fill is still there and it
works!!!
But it seemed to have some more capability (initial values, 2d arrays)
when it was in the sandbox??
I may be wrong and mixing up with some other library.
I just wanted to be sure that if i do my own patch i'm not
reinventing the wheel!!
As this is why we are standardizing on scikists.timeseries
Regards
Chris

[Pierre GM]

On Nov 18, 2009, at 5:18 PM, Chris Clarke wrote:

> Sorry for the later reply. Yes forward_fill is still there and it
> works!!!

Good

> But it seemed to have some more capability (initial values, 2d arrays)
> when it was in the sandbox??
> I may be wrong and mixing up with some other library.

That does sound familiar, but i don't think it was part of scikits.timeseries...
A patch for 2D would be welcome, I'm not quite sure what you mean by initial value, though

[Christopher Clarke]

Hi

Something seems to have gone wrong with my initial reply!!

Anyway, I often encounter the "initial values" use case when I am creating business day time series out of RDBMS tables using a subset of the observations in the table.  For example i have a SQL query fragment like

WHERE symbol='SFC' and dateix BETWEEN '2009-01-01' AND '2009-09-01'

Now suppose that 2009-01-02 and 2009-01-05 are missing (trading is sparse on many of the exchanges i'm dealing with) i am supposed to forward_fill using the last traded value for SFC which may or may not be 2008-12-31.  Hence i have a query that find the last traded values and i use these as the "initial values". 

Anyway here is by forward_fill wrapper. Its not very efficient as i'm copying and forward_fill is copying etc but..

 I'm actually starting to have reservation about the usefulness forward_fill on 2d as opposed to  the individual the individual series arrays as i am finding that i've often got to do loads of transformations and checking on the individuals arrays before i can combine them into  a single ma array for filling anyway

def forward_fill2(marr,maxgap=None,init_vals=None):

  """

  init_vals a list with  the same no. of elements as marr.shape[1]

  """

    arr=ma.array(marr,copy=True

    if arr.ndim == 1:

        if init_vals:

            if arr.mask.any() and arr.mask[0]:

                if init_vals:

                    arr[0] = init_vals[0]

                

        return forward_fill(arr,maxgap)

    else:

        n = arr.shape[1]

        if init_vals:

            mask=ma.getmask(arr)

            if len(init_vals) != n:

                raise ValueError, 'Initial Values sequence does no match number of columns'

            for c in range(len(init_vals)):

                if arr.mask.any() and mask[0,c]:

                        if init_vals[c]:

                            arr[0,c]=init_vals[c]

        arr = ma.hsplit(arr, n)[0]

        return ma.column_stack([forward_fill(np.squeeze(a),maxgap) for a in arr])

   

On Thu, Nov 19, 2009 at 5:07 AM, Chris Clarke <ccl...@chrisdev.com> wrote:

Hi 

The "initial value"