Skip to main content

Getting started with the OAK-D camera in Touchdesigner

The OAK-D Pro camera is an IR depth camera compatible with both Mac and Windows, making it a great alternative to the Kinect. This guide will show you how to set it up in TouchDesigner.

Connect the camera to your laptop and launch Touchdesigner. From the Operators panel find the OAK Device CHOP and Activate the chop, refresh the sensor list to find your camera. Click Initialize and wait for it to mark "Ready" (1) then press Start. This CHOP runs the code in Text DAT to capture RGB data from the camera, using the DepthAI library. To display this data, from the Operators panel find the OAK Select TOP. Open its parameter window, drag and drop your CHOP into the OAK Select TOP parameter 'OAK Device CHOP', then click on the arrow next to the Stream parameter and select 'rgb'. Your camera should now be running! 

# me - this DAT
# oakDeviceOp - the OP which is cooking

import depthai as dai

def onInitialize(oakDeviceOp, callCount):
	return 0

def onInitializeFail(oakDeviceOp):
	parent().addScriptError(oakDeviceOp.scriptErrors())
	return

def onReady(oakDeviceOp):
	return
	
def onStart(oakDeviceOp):
	return

def whileRunning(oakDeviceOp):
    disparityQueue = oakDeviceOp.device.getOutputQueue(name="disparity", maxSize=4, blocking=False)
    disparityFrame = disparityQueue.get()
    disparityData = disparityFrame.getFrame()  # Converts disparity to an OpenCV format
    # Normalize the disparity map for visualization
    disparityData = (disparityData - disparityData.min()) / (disparityData.max() - disparityData.min()) * 255
    disparityImage = disparityData.astype('uint8')
    # Send disparityImage to a TOP in TouchDesigner
    return


def onDone(oakDeviceOp):
	return


def createPipeline(oakDeviceOp):
    # Create pipeline
    pipeline = dai.Pipeline()

    # Define sources and outputs
    monoLeft = pipeline.create(dai.node.MonoCamera)
    monoRight = pipeline.create(dai.node.MonoCamera)
    depth = pipeline.create(dai.node.StereoDepth)
    xout = pipeline.create(dai.node.XLinkOut)

    # Set stream name
    xout.setStreamName("disparity")

    # Properties
    # Set the resolution for the mono cameras to 1280x800 (800P)
    monoLeft.setResolution(dai.MonoCameraProperties.SensorResolution.THE_800_P)
    monoLeft.setBoardSocket(dai.CameraBoardSocket.LEFT)
    monoRight.setResolution(dai.MonoCameraProperties.SensorResolution.THE_800_P)
    monoRight.setBoardSocket(dai.CameraBoardSocket.RIGHT)

    # Configure depth node
    depth.setDefaultProfilePreset(dai.node.StereoDepth.PresetMode.HIGH_DENSITY)
    depth.initialConfig.setMedianFilter(dai.MedianFilter.KERNEL_7x7)
    depth.initialConfig.setConfidenceThreshold(240)
    depth.setLeftRightCheck(True)
    depth.setExtendedDisparity(False)
    depth.setSubpixel(False)

    # Linking
    monoLeft.out.link(depth.left)
    monoRight.out.link(depth.right)
    depth.disparity.link(xout.input)

    return pipeline

Back to top