storing event in an array and identifying x y coords

[roman crooke]

Hi, 

So my plan is to modify the accumulating Events from camera code given by dv processing, and after capturing the event i want to store 4 copies of it into an array, so that i can identify the coordinates and then display it so that say, (x odd, y odd = green) and so on. 

Are the events able to be stored in an array similar to what is suggested here?:

https://www.pluralsight.com/guides/importing-image-data-into-numpy-arrays

And are there any examples of how to identify the coordinates of an event?

hope you are doing well, 

Roman :)

[jonatan....@gmail.com]

Hi, 

Sounds interesting! 

I think you should look at dv accumulator documentation: https://dv-processing.inivation.com/rel_1.7/accumulators.html#accumulating-event-from-a-camera

Although I don't really get what you want to achieve, my best guess would be that you want to use: frame = accumulator.generateFrame()

Hope this helps!

Best,

Jonatan

[roman crooke]

brilliant thank you, Ill try and work on it a bit more and try and ask a more specific question if I have any! 

Hope you are doing well,

Roman :)

[roman crooke]

Hi, so I think I have some code that works for identifying the events as (odd,odd) (even,odd) (odd,even) and (even,even) and have assigned them to an array for each, in theory i think i can feed these to the generateFrame function you mentioned before but i wanted to check if an array of coordinates in the form [(x,y), (x,y)] could work? Ive attached some screenshots of the event accumulation code. Any tips would be great for generating the frames from these arrays would be great! thanks :) 

[jonatan....@gmail.com]

Hi Roman, 

I am happy that you made some progress! It's hard to test the code and give feedback. Can you do a minimal code snippet that we can help you with? What are you loading on line 63? Dv-processing doesn't have a function like you are looking for (except generateFrame), but I think you would be able to write one yourself. 

Have a nice weekend!

best,

Jonatan

[roman crooke]

hi so the  code im using is :

    # Receive events

    events = capture.getNextEventBatch()

    im = cv.imread(events)

    blue = [255,0,0]

    red = [0,0,255]

    green = [0,255,0]

    X,Y = np.where(np.all(im==blue,axis=2))

    #id if its even or odd components

    coord_X= np.array(X)

    coord_Y= np.array(Y)

    # used to cat for bayer filter

#range for the X coord, then check for even and odd

    for i in range(len(X)):

        if (coord_X[i]%2 !=0 and coord_Y[i]%2 !=0):

            red_coord.append ((coord_X[i], coord_Y[i]))

        elif (coord_X[i]%2 !=0 and coord_Y[i]%2 ==0):

            green1_coord.append ((coord_X[i], coord_Y[i]))

        elif (coord_X[i]%2 ==0 and coord_Y[i]%2 !=0):  

            green2_corrd.append ((coord_X[i], coord_Y[i]))

        elif(coord_X[i]%2 ==0 and coord_Y[i]%2 ==0):

            blue_coord.append ((coord_X[i], coord_Y[i]))

Im currently trying to work in event accumulation, however it was suggested to be to work on it in the store event code, creating a function called Bayer Pattern. but im not sure how i would then feed the events into these categories 

[jonatan....@gmail.com]

Thank you for the code snippet! 

I think you want to look at the slicing callback again. You should probably use the multistreamslicer. Please take a look at the code snippet attached. 

Then in your while loop if you are able to extract the (for example) event_red, you can call:

slicer.accept("red",event_red). 

Don't forget to add all events to the main "events" stream (slicer.accept("events",events)) since the sequence is sliced after this one.

Hope this helps!

Best,

Jonatan

--------------------------------------------

# Initialize an event visualizer per color
visualizer_red = dv.visualization.EventVisualizer(capture.getEventResolution(), dv.visualization.colors.white(),
dv.visualization.colors.green(), dv.visualization.colors.red())

# Initialize preview window
cv.namedWindow("Preview", cv.WINDOW_NORMAL)

# Initialize a multi stream slicer with 5 streams (all events + red->blue)
slicer = dv.EventMultiStreamSlicer("events")
slicer.addEventStream("red")
slicer.addEventStream("green1")
slicer.addEventStream("green2")
slicer.addEventStream("blue")

# Declare the callback method for slicer
def slicing_callback(data):
all_events_not_used = data.getEvents("events")
events_red = data.getEvents("red")
#events_green1 = data.getEvents("green1")
#events_green2 = data.getEvents("green2")
#events_blue = data.getEvents("blue")

# Visualise red channel
cv.imshow("Preview", visualizer_red.generateImage(events_red))
cv.waitKey(2)

# Register a callback every 33 milliseconds
slicer.doEveryTimeInterval(timedelta(milliseconds=33), slicing_callback)

.....

[roman crooke]

Brilliant thank you so much for your help i really appreciate it! Enjoy your weekend! :)

[roman crooke]

Hi Jonatan, 

I just wanted to check something with you, in the create and store code given be dv processing, theres an ev.x and ev.y and i wanted to check how to properly import those functions into another script. does it need to be a for loop like the event create and store? 

..........

for ev in store: 24 print(f"Sliced event [{ev.timestamp()}, {ev.x()}, {ev.y()}, {ev.polarity()}]").........

As i think if i make the x and y's in that if, else if, these specfic versions, verses the numpy array ive been using, Ill be able to directly add them better to each of the events i want to create. also just wanted to double check that i can ".add " events to the associated store as well. 

Thanks for all your help, 

Roman :)

[jonatan....@gmail.com]

Hi again, 

If you call store.numpy(), you will get a numpy array with the length number of events. Element has data with [timestamp, x_coordinate, y_coordinate, polarity].

Instead of your for loop, you can then directly work with this data. 

Best,

Jonatan

[roman crooke]

Brilliant, Thank you! 

Best, 

Roman

[roman crooke]

Hi Jonatan 

Im close to getting the code working, thanks to all your help! but i wanted to check how to create events that can be declared whilst not calling them just "events" as I have a version of the code that almost works but wont recognize the "events_red", usually at the slicer line 119. Any advice would be great ! 

Thank again!

Roman 

import dv_processing as dv

import cv2 as cv

from datetime import timedelta

import time

# Open any camera

capture = dv.io.CameraCapture()

#genera store

gen_store = dv.EventStore()

#declare the dif stores and events for colours

# Make sure it supports event stream output, throw an error otherwise

if not capture.isEventStreamAvailable():

    raise RuntimeError("Input camera does not provide an event stream.")

# Initialize an accumulator with some resolution

visualizer = dv.visualization.EventVisualizer(capture.getEventResolution())

# Apply color scheme configuration, these values can be modified to taste

visualizer.setBackgroundColor(dv.visualization.colors.white())

visualizer.setPositiveColor(dv.visualization.colors.iniBlue())

visualizer.setNegativeColor(dv.visualization.colors.darkGrey())

#visualisers

visualizer_red = dv.visualization.EventVisualizer(capture.getEventResolution(), dv.visualization.colors.white(),

                                                  dv.visualization.colors.red(), dv.visualization.colors.red() )

visualizer_green1 = dv.visualization.EventVisualizer(capture.getEventResolution(), dv.visualization.colors.white(),

                                                  dv.visualization.colors.green(), dv.visualization.colors.green() )

visualizer_green2 = dv.visualization.EventVisualizer(capture.getEventResolution(), dv.visualization.colors.white(),

                                                  dv.visualization.colors.green(), dv.visualization.colors.green() )

visualizer_blue = dv.visualization.EventVisualizer(capture.getEventResolution(), dv.visualization.colors.white(),

                                                  dv.visualization.colors.blue(), dv.visualization.colors.blue() )

# Initialize a preview window

cv.namedWindow("Preview", cv.WINDOW_NORMAL)

# Initialize a slicer

slicer = dv.EventStreamSlicer()

# Declare the callback method for slicer

def slicing_callback(events_red: dv.EventStore):

    # Generate a preview frame

    frame = visualizer_red.generateImage(events_red)

    # Show the accumulated image

    cv.imshow("Preview", frame)

    cv.waitKey(2)

# Register callback to be performed every 33 milliseconds

slicer.doEveryTimeInterval(timedelta(milliseconds=33), slicing_callback)

# Run the event processing while the camera is connected

while capture.isRunning():

    # Receive events

    events = capture.getNextEventBatch()

    # Check if anything was received

    if events is not None:

        low = events.getLowestTime()

        high = events.getHighestTime()

        for i in range(high - low):

        # in here, fill in the 3 or 4 new stores for each colour

            for ev in gen_store:

                if (ev.x(i)%2 !=0 and ev.y(i)%2 !=0):

                    events_red = [{ev.timestamp()}, {ev.x(i)}, {ev.y(i)}, {ev.polarity()}] # do ev.x(i) and ev.y(i)

                    cv.imshow("Preview", visualizer_red.generateImage(events_red))

                elif (ev.x(i)%2 !=0 and ev.y(i)%2 ==0):

                    events_green1 = [{ev.timestamp()}, {ev.x(i)}, {ev.y(i)}, {ev.polarity()}]

                    cv.imshow("Preview", visualizer_green1.generateImage(events_green1))

                elif (ev.x(i)%2 ==0 and ev.y(i)%2 !=0):

                    events_green2 = [{ev.timestamp()}, {ev.x(i)}, {ev.y(i)}, {ev.polarity()}]

                    cv.imshow("Preview", visualizer_green2.generateImage(events_green2))

                elif (ev.x(i)%2 ==0 and ev.y(i)%2 ==0):

                    events_blue = [{ev.timestamp()}, {ev.x(i)}, {ev.y(i)}, {ev.polarity()}]

                    cv.imshow("Preview", visualizer_blue.generateImage(events_blue))

                #end if

            #end for ev in gen store

        #endd for i in range

       

       # cv.waitKey(2)

        slicer.accept(events_red) # this is where the error is

    else:

        # No data has arrived yet, short sleep to reduce CPU load

        time.sleep(0.001)

[jonatan....@gmail.com]

Hi roman, 

Great with the progress!

However, there is a couple of mistakes in your code. The main reason why slicer.accept(events_red) is failing is because it's empty. You never initialise events_red in your case. You are iterating gen_store, which is empty. I suggest moving everything to the slicer_callback function. Then you can do the filtering there. I updated your code a bit. Give it a try, and please provide feedback if it's not working. 

Hope this helps!

Best,

Jonatan

----------------------------------------------------------

import dv_processing as dv

import cv2 as cv

from datetime import timedelta
import time

# Open any camera
capture = dv.io.CameraCapture()

capture.setDavisColorMode(capture.DavisColorMode.Color)

# Make sure it supports event stream output, throw an error otherwise
if not capture.isEventStreamAvailable():
raise RuntimeError("Input camera does not provide an event stream.")

# visualisers
visualizer_red = dv.visualization.EventVisualizer(capture.getEventResolution(), dv.visualization.colors.white(),
dv.visualization.colors.red(), dv.visualization.colors.black())
visualizer_green1 = dv.visualization.EventVisualizer(capture.getEventResolution(), dv.visualization.colors.white(),
dv.visualization.colors.green(), dv.visualization.colors.black())
visualizer_green2 = dv.visualization.EventVisualizer(capture.getEventResolution(), dv.visualization.colors.white(),
dv.visualization.colors.green(), dv.visualization.colors.black())
visualizer_blue = dv.visualization.EventVisualizer(capture.getEventResolution(), dv.visualization.colors.white(),
dv.visualization.colors.blue(), dv.visualization.colors.black())

# Initialize a preview window
cv.namedWindow("Preview", cv.WINDOW_NORMAL)

# Initialize a slicer
slicer = dv.EventStreamSlicer()

# Declare the callback method for slicer

def slicing_callback(events):

# Check if anything was received

events_red = dv.EventStore()
events_green1 = dv.EventStore()
events_green2 = dv.EventStore()
events_blue = dv.EventStore()

if events is not None: # check that there is events
for ev in events: # for each event
if (ev.x() % 2 != 0 and ev.y() % 2 != 0): # if the event is red
x = int(ev.x() / 2) # divide by 2 to get the correct coordinates
y = int(ev.y() / 2)
events_red.push_back(ev.timestamp(), x, y, ev.polarity())
elif (ev.x() % 2 != 0 and ev.y() % 2 == 0): # if the event is green 1
x = int(ev.x() / 2) # divide by 2 to get the correct coordinates
y = int(ev.y() / 2 + 0.5)
events_green1.push_back(ev.timestamp(), x, y, ev.polarity())
elif (ev.x() % 2 == 0 and ev.y() % 2 != 0): # if the event is green 2
x = int(ev.x() / 2 + 0.5) # divide by 2 to get the correct coordinates
y = int(ev.y() / 2)
events_green2.push_back(ev.timestamp(), x, y, ev.polarity())
elif (ev.x() % 2 == 0 and ev.y() % 2 == 0): # if the event is blue
x = int(ev.x() / 2 + 0.5) # divide by 2 to get the correct coordinates
y = int(ev.y() / 2 + 0.5)
events_blue.push_back(ev.timestamp(), x, y, ev.polarity())

# Generate a preview frame for color channel
frame = visualizer_red.generateImage(events_red)
frame_green1 = visualizer_green1.generateImage(events_green1)
frame_green2 = visualizer_green2.generateImage(events_green2)
frame_blue = visualizer_blue.generateImage(events_blue)
# Tile all frames from above to one
frame[0:int(frame.shape[0] / 2), int(frame.shape[1]/2):int(frame.shape[1])] = frame_green1[0:int(frame.shape[0]/2), 0:int(frame.shape[1]/2)]
frame[int(frame.shape[0]/2):int(frame.shape[0]), 0:int(frame.shape[1] / 2)] = frame_green2[0:int(frame.shape[0]/2), 0:int(frame.shape[1]/2)]
frame[int(frame.shape[0]/2):int(frame.shape[0]), int(frame.shape[1]/2):int(frame.shape[1])] = frame_blue[0:int(frame.shape[0]/2), 0:int(frame.shape[1]/2)]

# Show the accumulated image
cv.imshow("Preview", frame)

cv.waitKey(1)

# Register callback to be performed every 33 milliseconds
slicer.doEveryTimeInterval(timedelta(milliseconds=33), slicing_callback)

# Run the event processing while the camera is connected
while capture.isRunning():
# Receive events
events = capture.getNextEventBatch()

if(events is not None): # check that there is events
slicer.accept(events)

else:
# No data has arrived yet, short sleep to reduce CPU load
time.sleep(0.001)

----------------------------------------------------------

[roman crooke]

Thank you so much for the advice !! I really appreciate it !!!