Rendered at 16:47:33 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
YuechenLi 15 minutes ago [-]
@baigy
Huh. Interesting. I was just doing the final clean-up for something convergent to this research that I had been working on for the past few months. I think I arrived at your thesis (code first semantics from a different direction in CAD, so I think it'd be interesting for us to compare notes.
Have you formalized this into a compiler infrastructure yet? I think Python on its own would be too slow to build complex parts, especially since for triangle mesh, accuracy inversely correlates to performance.
Vision is generally not the most reliable form of checks for LLMs, even on GPT 5.6 Sol, so a recommendation I would have is to instead emit JSON or CSV of the color/topology data for the LLM to inspect directly, and this is the instance where ray query for topology checking will greatly improve accuracy in general. SDFs are a bit more complicated right now, I have a full implementation designed for 3D analysis
My own experimental compiler generated mesh suffers from the spiderweb effect: it's very polygon efficient but not very friendly towards UV unwrapping in general, and I'm struggling to find the correct approach for that. If you have any suggestions, I'd love if you can point me towards the correct approach.
Definitely very interesting though.
baigy 2 hours ago [-]
I co-authored this paper. It's a new technique to generate 3D graphics as source code instead of a point cloud.
Under the hood, it generates 3D objects with separate, sophisticated internal assembly, producing an editable "kit of parts" (instead of monolithic blobs).
E.g. imagine you generated a 3D washing machine via this approach. It's not merely going to be just "geometry" that looks like a washing machine. We actually know that there is a `Door`, `Drum`, `Control_panel` etc. Which things belong to which assemblies. What moves and where its pivot is. And eventually what those components are supposed to do.
Most current 3D GenAI cannot do this since it generates "monolithic blobs" that look good, but are unusable in downstream workflows (e.g. game engines). I.e. if you generate a 3D bicycle using traditional approaches, it's basically a blob. When you need the wheels to turn, a human (or another AI) must spend time cutting the blob into parts, naming them, placing pivots and rigging joints. I.e. you need post-generation segmentation workflows of some sort.
The paper breaks down the whole technique, and there's a showcase (+ github repo) you can play around with: https://nova3d.xyz/
RobotToaster 15 minutes ago [-]
> there's a showcase (+ github repo) you can play around with: https://nova3d.xyz/
Before anyone else bothers giving them your Google account, there's zero free generations, something they conveniently don't disclose until after funneling you to sign up.
Sorry I got more traction on Blender and haven't integrated BYOK on the app.
P.s. the max I can do is BYOK. I can't hand out free gens. I'm unfunded and paying out of my own pocket. It is what it is.
avaer 30 minutes ago [-]
Have you explored optimizing the assets to be game-ready? This kind of decomposition works if you have a single object on screen, and it's super artist + programmer friendly. But the generated assets have ~50 mesh parts, which means importing just a couple of these into a scene and you've blown your entire draw call budget for a shippable game. It's the brick wall every gamedev realizes after trying to make a scene out of easy-to-work-with primitives. You just can't hit a playable frame rate like this unless your entire game consists of just a few objects.
Have you experimented with atlasing, mesh fusion, baking animations, standardizing PSO's to a scene budget, etc? Because if this can't be automated, I've found it really limits the utility of such freeform generation techniques in practice.
baigy 3 minutes ago [-]
[flagged]
NimraNoor 11 minutes ago [-]
[flagged]
zactato 17 minutes ago [-]
How does this compare to parametric modeling tools like Fusion/Solidworks/ProE ? Is it more about the integration with game specific tools?
baigy 6 minutes ago [-]
[dead]
7 minutes ago [-]
WilcoKruijer 45 minutes ago [-]
This looks super interesting. I'm trying out the hosted app using "bring your own key", I've added an OpenAI key but it doesn't seem to let me generate a 3d model. It's still saying I need credits. Is this expected?
With the app, I haven't added BYOK yet. I guess I should have before I posted on HN!
baigy 38 minutes ago [-]
Hang on right there!
RobotToaster 23 minutes ago [-]
Cool. Do the individual parts still use point clouds? Or are they meshes or CSG?
baigy 2 minutes ago [-]
The parts are originally defined in code, not stored as point clouds. That code builds the geometry using primitives, curves, custom mesh operations and sometimes CSG/booleans. When executed in Blender, the final exported GLB contains meshes.
How far are we from speaking a GameCube-era game into existence as a pastime?
baigy 1 hours ago [-]
GameCube-era game (with full synchronous multiplayer play): 1-2 quarters. Early "self-generating" Metaverse: ~ 3-6 quarters. The Matrix: ~5-7 years
avaer 40 minutes ago [-]
I am doubtful about the timeline.
Not because AI can't do it; it totally can. LLMs have been able to run the full artist + code pipeline at least since the beginning of the year. I've built several physics-synced network simulation stacks without reading a single line of code. Agents playtest my games overnight and I wake up to a list of technical issues fixed, and FPS boosted. If you know how to ask the shaders will look great.
The problem is that making a game actually worth playing (something that Nintendo would allow to be released) isn't something that was ever possible to do as a pasttime, AI or not. You have to be in front of the computer all day guiding it. Worse, AI does not have any notion of experiencing or evaluating fun, so you can't automate this. That would be a killer research problem to tackle, though!
If we're talking about making something that passes a sniff test, you could make a metaverse right now. It just wouldn't beat the bottom of the Steam barrel in terms of what players prefer.
WASDx 22 minutes ago [-]
> Agents playtest my games overnight
I'm curious if you want to elaborate. What kind of games? Turn based? Do you just feed it repeated screenshots?
avaer 9 minutes ago [-]
> Agents playtest my games overnight
I'm curious if you want to elaborate. What kind of games? Turn based? Do you just feed it repeated screenshots?
3D ARPG with dozens of systems, think Genshin or Fortnite. But it's all typescript running in the browser, so native browser introspection/debuggability came for free.
Fable has a cromulent time building its own tests, tools and pipelines. But there's no magic, it literally uses the gamepad and plays the game itself, taking screenshots, profiling, and debugging as it goes. The game ticks are fully controllable so it can frame-advance at its own pace.
Sometimes I take screenshots if it's some very complex multi-step repro. But 90% of the time it drives the engine itself.
baigy 33 minutes ago [-]
I agree. AI does not solve "product market fit". It mostly solves engineering. Currently AI is great as a tool, but not really a co-creator with taste.
Have you formalized this into a compiler infrastructure yet? I think Python on its own would be too slow to build complex parts, especially since for triangle mesh, accuracy inversely correlates to performance.
Vision is generally not the most reliable form of checks for LLMs, even on GPT 5.6 Sol, so a recommendation I would have is to instead emit JSON or CSV of the color/topology data for the LLM to inspect directly, and this is the instance where ray query for topology checking will greatly improve accuracy in general. SDFs are a bit more complicated right now, I have a full implementation designed for 3D analysis
My own experimental compiler generated mesh suffers from the spiderweb effect: it's very polygon efficient but not very friendly towards UV unwrapping in general, and I'm struggling to find the correct approach for that. If you have any suggestions, I'd love if you can point me towards the correct approach.
Definitely very interesting though.
Under the hood, it generates 3D objects with separate, sophisticated internal assembly, producing an editable "kit of parts" (instead of monolithic blobs).
E.g. imagine you generated a 3D washing machine via this approach. It's not merely going to be just "geometry" that looks like a washing machine. We actually know that there is a `Door`, `Drum`, `Control_panel` etc. Which things belong to which assemblies. What moves and where its pivot is. And eventually what those components are supposed to do.
Most current 3D GenAI cannot do this since it generates "monolithic blobs" that look good, but are unusable in downstream workflows (e.g. game engines). I.e. if you generate a 3D bicycle using traditional approaches, it's basically a blob. When you need the wheels to turn, a human (or another AI) must spend time cutting the blob into parts, naming them, placing pivots and rigging joints. I.e. you need post-generation segmentation workflows of some sort.
The paper breaks down the whole technique, and there's a showcase (+ github repo) you can play around with: https://nova3d.xyz/
Before anyone else bothers giving them your Google account, there's zero free generations, something they conveniently don't disclose until after funneling you to sign up.
Sorry I got more traction on Blender and haven't integrated BYOK on the app.
P.s. the max I can do is BYOK. I can't hand out free gens. I'm unfunded and paying out of my own pocket. It is what it is.
Have you experimented with atlasing, mesh fusion, baking animations, standardizing PSO's to a scene budget, etc? Because if this can't be automated, I've found it really limits the utility of such freeform generation techniques in practice.
With the app, I haven't added BYOK yet. I guess I should have before I posted on HN!
Not because AI can't do it; it totally can. LLMs have been able to run the full artist + code pipeline at least since the beginning of the year. I've built several physics-synced network simulation stacks without reading a single line of code. Agents playtest my games overnight and I wake up to a list of technical issues fixed, and FPS boosted. If you know how to ask the shaders will look great.
The problem is that making a game actually worth playing (something that Nintendo would allow to be released) isn't something that was ever possible to do as a pasttime, AI or not. You have to be in front of the computer all day guiding it. Worse, AI does not have any notion of experiencing or evaluating fun, so you can't automate this. That would be a killer research problem to tackle, though!
If we're talking about making something that passes a sniff test, you could make a metaverse right now. It just wouldn't beat the bottom of the Steam barrel in terms of what players prefer.
I'm curious if you want to elaborate. What kind of games? Turn based? Do you just feed it repeated screenshots?
3D ARPG with dozens of systems, think Genshin or Fortnite. But it's all typescript running in the browser, so native browser introspection/debuggability came for free.
Fable has a cromulent time building its own tests, tools and pipelines. But there's no magic, it literally uses the gamepad and plays the game itself, taking screenshots, profiling, and debugging as it goes. The game ticks are fully controllable so it can frame-advance at its own pace.
Sometimes I take screenshots if it's some very complex multi-step repro. But 90% of the time it drives the engine itself.