Image processing locally

[Bastian]

Hi,
I’ve been playing with color tracking of objects through openCV. Running my scripts remote on a Windows 8 PC, works really fine and after testing several ways of morphologic transformations in several different orders, I found a way that gives very satisfactory results. Even with less optimal light conditions. The script I use (color_detection_with_neck_movements.py) can be found here: https://github.com/Bas-Huizer/Rpi-camera-robot-car.git . When I run the script local on the Rpi-bot, the script stalls after connecting to the bot through websockets without any processing. I tried using the small images class, but that didn’t work either. I’ve seen several examples of color tracking, using the same approach, working on Raspberries . So I’m wondering what happens (and where did I go wrong)?. Is it a combination of the different layers (callback streaming, websockets, openCV)? Is my script to complex/slow? (E.g. All examples I’ve seen were written in C++, without the websockets layer. ) Am I just overlooking the root cause?
I would be very obliged if someone could give me some hints here!

[Alan]

Hi Bastien,

I haven't done much image processing on the robot, but I have got it to work a bit in the past (although I remember the framerate being very slow).

I think that there may be a bug or two in py_websockets_bot which is causing the image retrieval to be slow but I haven't had a chance to look at it yet.

When your script stalls, do you know if it's crashed or just running really slow? Could you try putting in some print statements to trace the program flow?

One thing that you mind find helpful is to put a time.sleep statement for maybe 0.1 to 0.5 seconds once every loop. I have seen the robot web server get swamped by commands if they're sent too quickly (I need to think of a solution for this) and this may be contributing to your problem now that all processing is taking place on the Pi.

Hope something there helps. Please let me know how you get on.

Regards

Alan

[Bastian]

Hi Alan,
Thnx for your response! I tried some adjustments: get_the_latest_image without call back, used the small image routine and inserted and changed several time.sleep and print statements. It didn’t make a difference. Even the first print-statement didn’t show up. When breaking the runs, it seems the script sticks in the __init__.py (a couple of times it also looked as reinitializing the bot: flashing red power light of the Rpi, all IR sensors kept flashing and I needed a full HW reboot).
The trace back shows like this:
File “/home/pi/py_websockets_bot/examples/Routine_search_sign_incl_neck_local_version.py”,
line 77, in <module> image, image_time = bot.get_latest_camera_image()
File “/usr/local/lib/python2.7/dist~ packages/ py_websockets_bot/__init__.py, line 303, in get_latest_camera_image
Return self._get._latest_streaming_data (“image”, “start_streaming_camera_images”, max_image_age)
File “/usr/local/lib/python2.7/dist~ packages/ py_websockets_bot/__init__.py, line 437, in get._latest_streaming_data time.sleep(0.001)

Looks to me as if the script stalls in getting an image and gets a time interrupt. Since not even the print statement is executed, I suspect it to be the first image pull.
Not sure if this makes sense?

[Alan]

Hi Bastian,

A couple of things to try.

Does a control script work if it doesn't try to stream images? i.e. if it just tries to drive the motors?

The line 437 in /home/pi/py_websockets_bot/init.py that reads

Code: Select all

time.sleep(0.001)

may be causing your Pi to run slow as it may not give enough time for images to arrive. Can you try changing the sleep to something like 0.1 or 0.01 to see if it improves things? You'll need to reinstall py_wesockets_bot after that though by going to /home/pi/py_websockets_bot and running

Code: Select all

sudo python setup.py install

Regards

Alan

[Bastian]

Hi Alan,
That did the trick! ThnX very much for that one. Changing the sleep timer into 0.01 was enough. Changing it to 0.1 gave the same result, so I kept it at 0.01. (Maybe it’s an update for a next release of your software).
I already experienced the impact of the sleep timers (scripts runs much faster than the bot is able to respond) and use them every time by trial and error. Do you have any logic or thumb rule by any chance?
The routine works now and the script can detect sign images (full routine with neck movements and motor shuffle on my Github ‘search_sign_routine.cpl’. Maybe it’ll become handy sometime).
Running local works noticeable slower than running remote. I hope I can tune/tweak the performance a bit. (The objective is running autonomous at the end.)
Next sprint will be moving towards the sign, taking the neck position as bearings and comparing the centroid coordinates to keep track. After that it’s reading the sign by comparison.
ThnX for helping me out!
Cheers,
Bas

[Alan]

Hi Bas,

That's great news. In terms of sleeping between updates, I think that you should be able to push it as low as 0.01 seconds. The problem seems to be, that if you don't have a sleep, then messages can get backed up on the web server running on the robot, and the server becomes unresponsive as it churns through them.

Interesting to see that you're drawing inspiration from http://roboticssamy.blogspot.nl/. I thought his RS4 robot was really cool and the sign reading code is incredibly elegant. I've done a partial conversion of the sign reading code to Python myself and was amazed at how straightforward the algorithm was once you understand it. I think I have a tendancy to overcomplicate things but I probably would have ended up with something much less efficient, and much more complicated if I'd tried to solve that problem myself.

Very interested to see how your project progresses. Please keep us updated.

Regards

Alan