Skip to main content

Depth camera for Mac users

Anyone who's worked with TouchDesigner before would have surely stumbled across some Kinect tutorial, which unfortunately is only compatible with Windows. The Creative Technology Hub now offers students the chance to borrow an OAK-D Pro camera. These are IR depth cameras compatible with both Mac and Windows, making it a great alternative to the Kinect. This page will briefly guide you through installing the prerequisites and show you how to set the camera up in TouchDesigner.

1. Installing Dependencies

Before using the OAK-D Pro with TouchDesigner, you'll need to install some dependencies:

  • Python 3.8+ 

  • DepthAI Library

Assuming you've already installed Python and the pip library for it, you can check your Python version by opening the terminal and typing in: 

pyhton --version

If your version is older than 3.8, make sure to upgrade it. Next install the DepthAI library by typing in:

pip install depthai

At this point it's good practice to restart your laptop before heading to TouchDesigner.

2.1 Setting up TouchDesigner

Connect the camera to your laptop and launch Touchdesigner.

From the Operators panel find the OAK Device CHOP and Activate the chop, refresh the sensor list to find your camera. Click Initialize and wait for it to mark "Ready" (1) then press Start. This CHOP runs the code in Text DAT to capture RGB data from the camera, using the DepthAI library. 

To display this data, from the Operators panel find the OAK Select TOP. Open its parameter window, drag and drop your CHOP into the OAK Select TOP parameter 'OAK Device CHOP', then click on the arrow next to the Stream parameter and select 'rgb'. Your camera should now be running! 

2.2 Depth Map

To gather both 'disparity' and 'rgb' data, replace the code in Text DAT with the following: 

import depthai as dai

def onInitialize(oakDeviceOp, callCount):
	return 0

def onInitializeFail(oakDeviceOp):
	parent().addScriptError(oakDeviceOp.scriptErrors())
	return

def onReady(oakDeviceOp):
	return
	
def onStart(oakDeviceOp):
	return

def whileRunning(oakDeviceOp):
    return

def onDone(oakDeviceOp):
	return

def createPipeline(oakDeviceOp):
    # Create pipeline
    pipeline = dai.Pipeline()

    # Define nodes for RGB stream
    camRgb = pipeline.create(dai.node.ColorCamera)
    xoutRgb = pipeline.create(dai.node.XLinkOut)
    xoutRgb.setStreamName('rgb')

    # Define nodes for Stereo Depth stream
    monoLeft = pipeline.create(dai.node.MonoCamera)
    monoRight = pipeline.create(dai.node.MonoCamera)
    depth = pipeline.create(dai.node.StereoDepth)
    xoutDepth = pipeline.create(dai.node.XLinkOut)
    xoutDepth.setStreamName('disparity')

    # Configure RGB camera node
    camRgb.setPreviewSize(1280, 720)
    camRgb.setBoardSocket(dai.CameraBoardSocket.RGB)
    camRgb.setFps(30)

    # Configure Stereo Depth nodes
    monoLeft.setResolution(dai.MonoCameraProperties.SensorResolution.THE_800_P)
    monoLeft.setBoardSocket(dai.CameraBoardSocket.LEFT)
    monoRight.setResolution(dai.MonoCameraProperties.SensorResolution.THE_800_P)
    monoRight.setBoardSocket(dai.CameraBoardSocket.RIGHT)

    # Configure Depth node
    depth.setDefaultProfilePreset(dai.node.StereoDepth.PresetMode.HIGH_DENSITY)
    depth.initialConfig.setMedianFilter(dai.MedianFilter.KERNEL_7x7)
    depth.initialConfig.setConfidenceThreshold(240)
    depth.setLeftRightCheck(True)
    depth.setExtendedDisparity(False)
    depth.setSubpixel(False)

    # Linking RGB camera to output stream
    camRgb.preview.link(xoutRgb.input)

    # Linking Mono cameras to depth node
    monoLeft.out.link(depth.left)
    monoRight.out.link(depth.right)

    # Linking depth output to disparity stream
    depth.disparity.link(xoutDepth.input)

    return pipeline

At this stage you can once again click the arrow next to the Stream parameter in OAK Select TOP and select 'disparity'. 

You can create a second OAK Select TOP, should you need the 'rgb' channel also displayed.

3. Conclusion 

The OAK-D Pro is a fantastic alternative to the Kinect for Mac users working with TouchDesigner. At this stage you should be able to set this camera up and follow any Kinect tutorial along. Happy coding!

Back to top