Consecutive Events Below Threshold

I have SPI timeseries of length 324 and values ranging from -3 to +3. I want to get the indices of places where 3 or more consecutive timesteps are more below threshold -1 I have t

Solution 1:

With np.logical_and.reduce + shift, checking for consecutive rows that are below the threshold. Then a groupby to get all of the aggregations you need:

import numpy as np
import pandas as pd

defget_grps(s, thresh=-1, Nmin=3):
    """
    Nmin : int > 0
        Min number of consecutive values below threshold.
    """
    m = np.logical_and.reduce([s.shift(-i).le(thresh) for i inrange(Nmin)])
    if Nmin > 1:
        m = pd.Series(m, index=s.index).replace({False: np.NaN}).ffill(limit=Nmin-1).fillna(False)
    else:
        m = pd.Series(m, index=s.index)

    # Form consecutive groups
    gps = m.ne(m.shift(1)).cumsum().where(m)

    # Return None if no groups, else the aggregationsif gps.isnull().all():
        returnNoneelse:
        return s.groupby(gps).agg([list, sum, 'size']).reset_index(drop=True)

---

get_grps(pd.Series(a))
#               listsumsize#0[-1, -2, -5]-83#1[-3, -3, -1, -2]-94get_grps(pd.Series(a), thresh=-1, Nmin=1)
#               listsumsize#0[-3]-31#1[-1, -2, -5]-83#2[-3, -3, -1, -2]-94get_grps(pd.Series(a), thresh=-100, Nmin=1)
#None

Solution 2:

Here is a commented step-by-step recipe.

a = [-3,4,5,-1,-2,-5,1,4,6,9,-3,-3,-1,-2,4,1,4]
th = -1
a = np.array(a)

# create mask of events; find indices where mask switches
intervals = np.where(np.diff(a<=th, prepend=0, append=0))[0].reshape(-1,2)

# discard short stretches
intervals = intervals[np.subtract(*intervals.T) <= -3]

intervals
# array([[ 3,  6],#        [10, 14]])# get corresponding data
stretches = np.split(a, intervals.reshape(-1))[1::2]

stretches
# [array([-1, -2, -5]), array([-3, -3, -1, -2])]# count events
-np.subtract(*intervals.T)
# array([3, 4])# sum events
np.add.reduceat(a, intervals.reshape(-1))[::2]
# array([-8, -9])

Solution 3:

Since you tagged pandas:

s = pd.Series([-3,4,5,-1,-2,-5,1,4,6,9,-3,-3,-1,-2,4,1,4])

# thresholdinga = (s<1)

# blocksb = (a!=a.shift()).cumsum()

# groupbydf = s[a].groupby(b).agg([list,'size','sum'])
df = df[df.size>=3]

Output

           list      size   sum
3[-1, -2, -5]3   -85[-3, -3, -1, -2]4   -9