Give an AI agent a model idea. Guide it, let it build it, and get an LDraw LEGO© model.

What you get when the building process finishes:

• Its source code, in LDraw language.
• Different views: 3D viewer, 3D player, VR interactive (Meta Quest 3), images...
• Blender editable glTF file, in .glb format, metainfo as Blender's Custom Properties.
• Chat history and agent thinking process.
• and more... 👌

Take a look at the video:

Run ldraw-nova as a web app, with Docker. You need Git and Docker.

This web app will run dockerized, and to build the Docker image, two sibling repos are required:

1. Clone both repos side by side, at the same tag, so they work together:

git clone --branch v0.6.0 https://github.com/anteloc/ldraw-nova.git
git clone --branch v0.6.0 https://github.com/anteloc/ldraw-nova-docker.git

2. Build the Docker image. The first build takes a while and needs about 5 GB of disk space:

cd ldraw-nova-docker
docker compose build

3. Start the app:

4. Open it in your browser:

• https://localhost:8443: needed for VR on Meta Quest 3. The certificate is self-signed, so accept the browser's warning the first time.
• http://localhost:8765: plain HTTP, no certificate warnings. Use it if the self-signed certificate gets in the way. VR won't work over it.

Other devices on your network can reach the app by your computer's IP instead of localhost, e.g. https://192.168.1.20:8443 from a Quest 3. The app has no login, so only run it on networks you trust.

To stop it:

Well, to summarize: I did this in order to get agentic LLMs capable of designing buildable, physical things!

Finding LDraw, an assembly language (pun intended! 😜) that would be at the same time simple, low level, and executable in order to produce 3D CAD models, gave me the idea of experimenting with both ChatGPT and Claude in order to try and make them code in LDraw, same as they do with other programming languages.

To my surprise, even though this language is heavily focused on math (parts rotations, positioning...), which LLMs are usually bad at, agents did pretty well instead on initial tests, and subsequent projects also yielded good results, but never enough in order to consider generated models to be correct:

These three attempts, and quite some other experimentation, led me to the following conclusions:

💡 Conclusion 1: there is a minimum resistance path to geometry math for agents, i.e.:

• Giving the agents tooling to generate LDraw sources would sidestep (evil!) geometry math
• ... because they do way better at generating python code that produces math
• ... than on producing math themselves!

💡 Conclusion 2:

• Agents tend to do better when learning from python code that produces models
• ... than from models themselves (LDraw's evil geometry, again...)

Then, the only thing left 🤔 was to create a python-based tooling with the required primitives, verbs, constructive vocabulary... so agents would learn by example and do similar things on their own.

Which proved to be really hard to get right, even if vibe coding it... until GPT-6 Astra and Claude Opus 5.5 arrived... and vibe-coded it right! 🚀🚀🚀

ldraw-nova provides the tools, examples and instructions an agent needs to design models with real LDraw parts.

The process is as follows:

1. The agent takes a prompt.
2. Reads instructions.md and related documents to LDraw language and LEGO© models building.
3. Plans how to build the model: required parts, submodels to be created, aesthetics...
4. Iteratively:
   1. Renders images from the model/submodel(s)
   2. Inspects them, adjusts positioning, aesthetics... and back to rendering
5. ... until it considers the model finished and ready to deliver!

Provided tooling helps the agent in:

• Finding suitable parts.
• Also, example models and submodels to start with.
• Collision and gaps detection for placing parts correctly.
• Headless rendering for inspecting current results.
• and more...

The agent doesn't actually start with placing parts, except for things like e.g. prototyping and learning by altering pre-existing example models.

The way it produces models is more like:

• Collects the required information, from experimental results, docs and planning.
• Builds one or more plans, that fully describe the model and submodels, including its geometry, like e.g. atlas-crane.plan.json
• And with that plan, it creates one or more generator scripts like e.g. generate.py
• ... that when executed, produce LDraw source file(s), a very specialized 3D CAD language.
• ... like e.g. atlas-crane.mpd

To summarize, this is like:

• an agent creating a generator
• ... that produces a 3D model
• ... in an assembly language named LDraw 🤯

A compiler of sorts, so to say 🤓

flowchart TD
    agent([agent]) -- produces --> plan[plan.json]
    plan -- interpretation --> gen[generator.py]
    gen -- execution --> model[model.mpd]

These are some of the guides and references given to the agent in order to make it a builder: