PITTSBURGH – Researchers at Carnegie Mellon University’s School of Computer Science have created BrickGPT, an AI-powered tool that uses text prompts to bring ideas to life using Lego brick designs. The demo currently produces step-by-step instructions for 21 Lego structures.
The research team – Ava Pun, Kangle Deng, Ruixuan Liu, Deva Ramanan, Changliu Liu, and Jun-Yan Zhu – describe BrickGPT as “the first approach for generating physically stable interconnecting brick assembly models from text prompts.”
Researchers say the project sets a precedent for AI innovation, as making tangible items from text prompts could one day reduce roadblocks and delays in construction projects.
“It takes a long time to turn ideas into a physical design and prototype,” said Changliu Liu. “But if you can integrate generative AI into the process, it can significantly improve efficiency when kicking off projects.”
The current BrickGPT demo accounts for structural stability through a score-based physics reasoning algorithm. The AI model tracks stability by considering previous brick placement.
“BrickGPT generates brick structures incrementally by predicting one brick at a time given a text prompt,” the team said. “We perform validity checks for each generated brick to ensure it does not collide with existing bricks. If the structure is unstable, we roll back to a stable state by removing all unstable bricks, and resume generation from that point.”
Researchers said BrickGPT will allow users to account for structural instability while saving time drafting assembly instructions. Next steps for the project include scaling the model, expanding the type of objects that can be generated, and improving diversity to create more complex structures.
