Simple protocol

[Mikkel Roald-Arbøl]

My initial goal is to just activate/deactivate a camera - no actual tasks, just monitoring. I know it's not utilizing the potential of the software, but as I am hoping to rely on autopilot for most/all of my experiments (to streamline the workflow), I would like to also do the simple things from here. But from reading the docs, I can't figure out how to do this - or if it is even possible to do without having a task.

Now the quesiton/request:  So would you/anyone be able to write a simple example protocol(/task?) which would be able to do that? I assume this is something that many people doing behaviour would be interested in. 

Cheers!

[jlsaun...@gmail.com]

Yes! it is possible to use hardware objects outside of tasks!

It would be something like

from autopilot.hardware import cameras

cam = cameras.Camera_CV(0)

# to write to a file

cam.write('output_file.mp4')

# to capture indefinitely

cam.capture()

# or to capture for 10 seconds

# cam.capture(10)

Am i understanding your question correctly?

[jlsaun...@gmail.com]

PS good idea! added an example like this to the documentation queue.

[Mikkel Roald-Arbøl]

Not quite, but that's perfectly fine. :-D I really like that you can still use the hardware classes outside of tasks, great feature!

Was actually more thinking of a simple monitoring task that can still be started from the Terminal (so no closed-loop needed here), and where the camera/sensor can be specified in the protocol - but you've gotten me in the right direction, so I'll try to write one and will send it your way when it's done. (Y) 

Cheers!

[jlsaun...@gmail.com]

Aha -- yes this is also related to another development goal, In addition to the tasks/pilot pattern, I think it would be useful for situations like this to have an 'observer' agent: so something that either can run some continuous observation, trigger observation every x interval, trigger observation on some input (like a motion detecting camera). Also would be good to have this agent be smart about data reporting -- for something eg. deployed as a wildlife observation camera, would want to save data locally and upload when it could rather than needing to continuously stream.

To do that, the long-term goal would be to refactor what are currently the "Pilot" and "Terminal" classes into a single "Agent" inheritance tree, and make a more formal description of the methods and attributes that should make up an Agent.

For now, yes a simple task that starts a camera should work, but doesn't necessarily have the ux you might want for continuous observation

[Mikkel Roald-Arbøl]

That sounds great! I actually named my task 'Observer', so I can tell we're aligning here. I can see how that creating a new agent isn't a small job, so for now I'll make a Task. (Y) I don't necessarily think you would need a completely new agent, but perhaps just expanding the features of the Pilot would be sufficient without complicating things too much. Perhaps just having a binary option to have tasks/protocols that evaluate input, and tasks/protocols that merely observe. Does that make sense?

In terms of making features for supporting wildlife camera traps I wouldn't put in too much effort. I've actually delved into that topic a bit lately, and because of the relatively high power consumption of the Pi's (even if they 'sleep') would be too high as they need to be battery powered, so people tend to use Arduino's or other PCBs with very low power consumption even with good power banks. However, an Observer agent like that would still be immensely valuable in the lab - I do experiments on sleeping insects, so most of the time it might not be worth actually collecting data, e.g. DLC-tracking an animal all the time, even if it doesn't move for hours.

[Mikkel Roald-Arbøl]

Ah, things just got a bit more complicated when I realised the Camera_CV of course don't support the native RPi camera modules. There's a library for those (PiCamera), and I've tried to fidget with writing a 'Camera_PI' class in cameras.py, but I really don't have enough of an overview to do so currently. Would be awesome though, e.g. I'm using an IR camera (https://shop.pimoroni.com/products/night-vision-camera-module-for-raspberry-pi?variant=12516582752339). And now with their HQ camera which, along with the right lens, works well with microscopy!

If you can put me int the right direction regarding which arguments and attributes it would need I'd be down to write such a class.

[Mikkel Roald-Arbøl]

So, a little progress! Turns out that the native CSI camera can be made available to OpenCV with: 

sudo modprobe bcm2835-v4l2       (see https://stackoverflow.com/a/37530016/13240268)

Additionally, it turned out I didn't actually have cv2 in my virtualenv (checked wity 'pip freeze'. Maybe not at all on my Pi...), so simply installed it inside the virtualenv:

pip3 install opencv-python

Additionally, in cameras.py, there are references to prefs.TERMINALPORT which gave me errors, but I assume should now be PUSHPORT - at least that worked. So now it sort of works... I've made a short observe.py task in which I'm currently hardcoding the destination and I'm sure it will have to be done more elegantly. I'm also not sure I'm recording at 30fps eventhough it's specified as I can tell from my Pi terminal that the fps changes throughout. 

But HEY! I'll celebrate this victory for now! :-D  

[jlsaun...@gmail.com]

Sorry for the delay, time is extremely scare nowadays.

Several people have asked about this and I think adding a real wrapper for the pi camera would be a really good idea. from what I understand the native library is really quick, so would be good to not have a ton of opencv overhead on top of it. that would just consist of subclassing the Camera class and overriding the relevant methods for initialization, capture, property setting, etc. The metaclass documentation should have descriptions of the relevant methods and properties, and i'll be able to write that more fully after i finish this batch of writing on my desk.

good to know about the opencv trick though, the more I think about it the more I think i need to actually rent a server for the site and deploy a wiki on it -- there are so many little tricks like these for working with the pi that are scattered throughout the docs and my notes, and it woudl be good to have a central place for people to stash tricks like these.

More detail on the framerate issue? I've had a lot of trouble with reliable control of capture fps through OpenCV.

congratulations on the little wins :)

[Mikkel Roald-Arbøl]

No worries, just appreciative you do at all. :-)

I'll give a go at writing a native PI camera class when I have time. And I'll update on the framerate issue once I know more.

Cheers!

[Mikkel Roald-Arbøl]

Once you have more time on your hands it would be great to chat, I'd be more than happy to contribute some of my time to Autopilot backend and/or documentation. :-)

[jlsaun...@gmail.com]

Fabulous!!! should be finishing up a new experiment, and then I need to actually submit the paper to a journal, but then I plan on setting up another big development push and would love to have you along for the next ride :)