LEGO Building Robot

End-to-end robot arm system for detecting, picking, and stacking custom LEGO-style blocks.

For Berkeley EECS 106A, our team built an end-to-end LEGO-style block assembly robot using a UR5 arm, wrist-mounted camera, custom 3D-printed blocks, ROS 2, OpenCV, ArUco calibration, and MoveIt motion planning.

The system detects square and rectangular blocks in a low-light workspace, transforms camera-frame detections into the robot base frame, and executes a repeatable pick-and-place routine to build multi-step structures from high-level instructions.

My strongest contributions were the low-light perception pipeline, perception-to-control integration, camera/table/arm transforms, and end-to-end testing. The final stack connected real-time block segmentation, instruction processing, inverse kinematics, and robot execution into a working robotics pipeline.

Strengths

  • End-to-end perception-to-action robotics pipeline.
  • Low-light OpenCV block detection using LAB color features and contour geometry.
  • ROS 2 topics/services connecting perception, instructions, and arm movement.
  • MoveIt-based inverse kinematics and pick-and-place execution.
  • Custom block hardware iterated across four versions for reliable stacking.

Gallery

UR5 arm placing a custom block
UR5 arm executing a block pick-and-place motion.
System design flow from calibration through perception and pick-and-place
System flow connecting calibration, camera transforms, block detection, and pick-and-place.
Vision pipeline stages for low-light block detection
Vision pipeline stages for low-light block detection.
Small and long block detection results
Small and long block selection from the same camera view.

Links