We have a system in which we make many small captures in quick succession of each other. The captures are split because they require different settings. Currently, we open the stream to take pictures, and then when we have enough we try to stop (or abort) the stream. Often this goes well, but sometimes it gets stuck in stream.try_stop(). The code just ‘freezes’ without Exception, also doesn’t terminate.
Above also tested for try_abort, and multi/continuous mode) It happens after a random amount of captures, in either of the cameras we’re using (no consistencies found yet). We use two cameras that are coupled in a “master-slave” configuration
How/when can this happen?
Is there a better way to start/stop the acquisition? We currently use a MultiFrame for capturing. It would be better to keep the stream running, and just like in Genicam use AcquisitionStart to reinitiate a capture, but I haven’t figured out a way of doing this through the python API?
Copy that! The same happend to me. I finally got it reasonable good by putting cam start/stop in a daemon thread. This is my flow:
Main program starts a “recording thread” (daemon thread)
Recording thread starts and stops cam/stream once before starting the actual recording.
Main program waits a while for a “CAM OK” state from the recording thread.
If the “CAM OK” state doesn’t show up we know the recording thread is hanging.
The main program makes a system exit() which kills the daemon thread. (I haven’t found a good way of killing a hanging thread in Python?)
A third “defibrillator” program checks if my main program is running and restarts if necessary.
It looks crazy complicated and I totally agree. It shouldn’t have to be like this but it works pretty well now. I’ll attach part of the defibrillator code so you can save some time on those annoying things😉
-------- “defibrillator” code --------
I removed a few things so it might not be running directly😊
I start the Python scripts from .bat files hence the CMD stuff and 2 processes (cmd & python).
running_state = DefibrillatorState.NORMAL
class DefibrillatorState(Enum):
NORMAL = 0
RESTARTED = 1
XXX_FAIL = 2
def check_processes():
global running_state
try:
python_rec_proc=None
cmd_rec_proc=None
# Find processes of interest
for proc in psutil.process_iter(['pid', 'name', 'cmdline', 'cpu_percent']):
# Get CMD processes
if proc.name().startswith("cmd") and len( proc.cmdline() ) > 2:
if "run_REC.bat" in proc.cmdline()[2]:
cmd_rec_proc = proc
# Get Python processes
if proc.name().startswith("python") and len( proc.cmdline() ) > 1:
if "recording.py" in proc.cmdline()[1]:
python_rec_proc = proc
# Do the process validation
if python_rec_proc==None or running_state == DefibrillatorState.XXX_FAIL:
# Python recording has died. Do a safety kill on CMD and PY and restart
kill_proc(python_rec_proc, "Python recorder")
kill_proc(cmd_rec_proc, "CMD RECORDING")
time.sleep(1)
os.system("start cmd /c run_REC.bat")
running_state = DefibrillatorState.RESTARTED
else:
if running_state == DefibrillatorState.RESTARTED:
running_state = DefibrillatorState.NORMAL
except Exception as e:
logging.warning(f"Exception during checking of processes: {e}")
except KeyboardInterrupt:
logging.info("Defibrillator received CTRL-C. Exiting.")
sys.exit(2)
def kill_proc(proc, proc_name):
try:
if proc == None:
logging.debug(f"---{proc_name} was not running")
else:
logging.debug(f"---Killing {proc_name}")
proc.kill()
except Exception as e:
logging.debug(f"Exception during killing of {proc_name}: {e}")
if __name__ == '__main__':
logging.debug(f"DEFIBRILLATOR is running...")
while True:
time.sleep(15)
check_processes()
have you tried to use the try_abort()?
The reason is, that the try_stop() waits for the current image, if this is not delivered, it will wait for ever.
try_abort() on the other hand simply cancels any ongoing waits.
The timeout after which a possible active wait will be canceled is infinite per default (exception: CompositeStreamBase Streams).
The new acquisition stack has an overload for the try_stop() to pass a timeout after which the acquisition will be cancelled.
Gen 3 acq stack
(C++ examples only, python works pretty much the same)
Hi Johang! Thx for sharing your knowledge and code. I had the same consideration but I’m hoping there is a simpler approach before spending some time on it . If not will definitely try to implement it!
@Chris I have tested try_abort(), abort(), stop(), try_stop() independently, in combination with a stream.wait, and a stream.wait_for(10).
Execution still freezes on the try_abort(). That was tested yesterday in combination with the stream_wait_for(10). Next week I’ll have some time to extract the class from our system and I’ll try to build a simpler testable script in hope of reproducing the problem.
Any other ideas of what could be the problem in the meantime?
. If not will definitely try to implement it!