Grabbing frames from camera and stream to NGINX rtmp module

Has anyone setup a connection between the CVBpy library and streamed the incoming images to nginx rtmp module? Have been scouting the interwebs and havent found any great solutions, I want to stream the live view from the camera to NGINX rtmp and deliver it to a web UI with HLS, the NGINX part is something that i know how to do, i’m only having trouble with the part between the camera and streaming to the server.

Is there anyone that could point me in the right direction, I could share the results here when done for others if wanted.

You probably want to access the image data directly to stream it. In that case, have a look at the to_array method where you can get the image as numpy array:

 np_array = cvb.as_array(image, copy=False)

Thanks for replying, i guess that i already had implemented that array conversion method into my code, i’m just having troubles with sending it from my code to a server, would i go for ffmpeg or something like g-streamer after that part?

I think so, yes. Maybe try ffmpeg. You should be able to pipe the image to ffmpeg and stream it. Sadly I have not much experience in this area as we mainly support CVB. There is an upcoming project to stream image data though. It is part of CVB 13.04.000 but not documented and currently only in C. If you have a business case for this, you can let us know. It might help prioritize the python bindings for web streaming.

Hi @antonkristensen ,

the following code uses Socket.IO to stream a CVB image converted to JPEG over a websocket, maybe this helps.
If you need the client side as well (HTML with JS) let me know.

Cheers
Chris

from aiohttp import web
import socketio
import time
import asyncio
import cvb
import os
import json
import numpy as np
import io
import PIL.Image as Image
import base64

connected = 0

sio = socketio.AsyncServer(async_mode='aiohttp')
app = web.Application()
sio.attach(app)

async def index(request):
    with open('index.html') as f:
        return web.Response(text=f.read(), content_type='text/html')

app.router.add_get('/', index)
app.router.add_static('/static/', path='static/')

@sio.on('connect')
async def connectionEstablished(sid, session):
    global connected
    connected += 1
    print("Connected to: " + sid)

@sio.on('disconnect')
async def connectionClosed(sid):
    global connected
    connected -= 1
    print("Disconnected: " + sid + " Connections left: " + str(connected))        

@sio.on("clientToServer")
async def receiveData(sid, data):
    print("received: " + data) 

async def sendData(dataOut):
    await sio.emit('serverToClient', dataOut)

async def main():
    try:
        global connected
        print("Waiting for clients")
        while connected == 0:
            await asyncio.sleep(1)
            pass
        with cvb.DeviceFactory.open(os.path.join(cvb.install_path(), "drivers", "CVMock.vin"), port=0) as device:
            print("Driver loaded")
            stream = device.stream
            stream.start()
            while True:
                result = await device.stream.wait_async_for(2000)
                image, status = result.value
                if status == cvb.WaitStatus.Ok:
                    npyArray = cvb.as_array(image, copy=False)
                    np_img = Image.fromarray(npyArray)
                    buffered = io.BytesIO()
                    np_img.save(buffered, format="jpeg")
                    img_str = base64.b64encode(buffered.getvalue())
                    data = str(img_str)[2:-1]
                    await sendData(data)
    except:
         print("Cancelled acquisition")

    finally:
         print("finished, exiting now")

sio.start_background_task(main)
print("Starting web-server")

if __name__ == '__main__':
    web.run_app(app)

Thanks, trying to find all other alternatives over a web app since i don’t want and don’t need the overhead of it built into my acquisition code…

Well in the while loop, you are left with the CVB image interpreted as jpeg… so you only need to find a way to send this to you server, any web application should then be able to interprete/display the images.

If your server is running on the same machine or you have access to the directories on your server (which should easily be configured) you might simply save the jpeg to a folder that is configured in NGINX.

At least thats what I understand you want to do.

Cheers
Chris

Yeah in theory its a similar thing, but i don’t wan’t to write to the hard-drive when its only configured for live view to browser, i’ll look more into the ffmpeg route

Thanks for this recommendation and tips.