🔍 Read the full analysis: How To Adapt AI Models For 3D Generation on ThorstenMeyerAI.com
Get business pricing on monitors, keyboards and dev gear
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
Hugging Face published scripts and containers that let developers fine-tune TRELLIS.2 on custom 3D asset collections, locally or through Hugging Face Jobs. Its example targets shoe shape and sole geometry using LoRA adapters and 188 footwear objects, but the source reports no completed evaluation or measured improvement.
Hugging Face has published scripts and containers for adapting its TRELLIS.2 3D-generation model to custom asset collections, offering developers a way to train category-specific versions locally or through Hugging Face Jobs. The accompanying example applies LoRA adapters to a dataset of 188 shoes to test whether fine-tuning can improve shoe geometry, including soles; the supplied account does not report a final result.
The example uses footwear from Google Scanned Objects (GSO) and focuses training on parts of TRELLIS.2 that shape an object’s structure and geometry. The authors said their initial tests produced shoes that generally looked right but had inaccurate soles. The training plan is intended to address that category-specific issue, but the described goal is an experiment, not a demonstrated fix.
The workflow uses low-rank adaptation (LoRA), a method that trains small adapter matrices while leaving the model’s original weights frozen. The post says this reduces the number of parameters being trained and the memory needed for gradients and optimizer state; the scripts use the PEFT library. Developers can run preprocessing and training with a local GPU and Docker commands, or use containerized workloads on Hugging Face Jobs.
To try the workflow, users upload a mesh dataset to the Hugging Face Hub and adapt the scripts using a small number of arguments. The post says a Hugging Face token is required, as is access to DINOv3, a gated model used to encode input images. The supplied material does not give hardware requirements, estimated costs, or a measured training time.
Fine-tuning TRELLIS.2 for custom 3D assets
Hugging Face released scripts and containers to adapt TRELLIS.2 to specific object categories. The footwear example outlines an experiment, but reports no completed results.
What the workflow targets
The example addresses shoes that look plausible but have inaccurate soles.
Coarse shape
LoRA adapters target sparse structure to adapt the model to footwear.
Detailed shape
The shape stage generates geometry; the example aims to improve shoe details such as soles.
Stage left unchanged
The dataset lacks separate material maps, and the authors say base colors and textures are satisfactory.
How TRELLIS.2 builds a mesh
The described pipeline turns an input image into a textured mesh.
BiRefNet removes the background, then DINOv3-L encodes the image.
Sparse structure guides the model’s detailed geometry generation.
A separate texture stage generates materials before the outputs are combined into a GLB mesh.
Ways to run the experiment
Teams with suitable assets can choose a local or cloud workflow.
- Use the provided Docker commands with your own GPU.
- Run preprocessing and training scripts on your machine.
- Run containerized workloads on cloud hardware.
- Upload the mesh dataset to the Hugging Face Hub.
What teams need
Access and evaluation matter alongside the training setup.
- A Hugging Face token and access to gated DINOv3.
- A comparison of pretrained and fine-tuned outputs, including held-out footwear.
- Clear geometry criteria and reported training settings, hardware, and costs.
A Route to Category-Specific 3D Models
The release addresses a practical limitation of general-purpose 3D generation: a model may create an object that looks plausible overall while missing a feature that matters to a particular use. Reliable geometry can be important in settings such as robotics, simulation, and media production, where a visually convincing result may not meet a task’s requirements.
Fine-tuning offers teams with suitable assets a way to test whether a broad model can be adapted to a narrow category without retraining all its parameters. The packaging of preprocessing and training for both local and cloud runs may also help developers reproduce experiments across different environments. Those are potential benefits of the workflow, not results established by the shoe example.
The example makes another practical choice visible: developers can target model components relevant to their problem. The authors adapt the stages associated with coarse structure and detailed geometry, while leaving texture generation unchanged. They say the footwear data includes diffuse color maps but not the separate material maps needed to train the texture stage, and that the base model’s colors and textures were already satisfactory for their task. The source does not compare this choice with alternatives or quantify any savings.
Popular right now
Bose QuietComfort Wireless Noise-Cancelling Over-Ear Headphones, Twilight Blue
Ninja PB051 Single-Serve Coffee Maker for Pods and Grounds
iRobot Roomba 105 Combo Robot Vacuum & Mop with AutoEmpty Dock
As an affiliate, we earn on qualifying purchases.
How TRELLIS.2 Produces a Mesh
The post describes TRELLIS.2 as a system that turns an input image into a textured 3D mesh through several stages. It first removes the image background with BiRefNet and encodes the image with DINOv3-L. A sparse-structure flow model predicts a coarse shape, which guides a shape flow model that produces more detailed geometry. A separate texture flow model generates surface materials; the outputs are combined into a mesh that can be exported as a GLB file.
The example places LoRA adapters in the sparse-structure and shape flow diffusion transformers, targeting the parts of the pipeline most relevant to shoe form. The authors leave the texture flow stage untouched. This is the stated rationale for the training plan, rather than evidence that it is the best configuration.
The post presents TRELLIS and TRELLIS.2 as using a unified latent representation intended to support different 3D outputs through decoders, rather than requiring a distinct generation model for every representation. That background helps explain the appeal of adapting a model’s geometry pipeline to a specific category, but the supplied material centers on the fine-tuning tools and shoe experiment.
“Initial tests produced shoes that looked right but had inaccurate soles.”
— The authors of the fine-tuning example
The Shoe Experiment Has No Results Yet
The supplied material does not say whether the fine-tuning run was completed or whether it improved sole accuracy. It includes no before-and-after examples, quantitative benchmark, or independent evaluation. It also does not specify the test set, scoring criteria, or how the authors judged whether a sole was accurate.
Practical requirements remain unclear as well. The account does not report the hardware configuration, compute cost, training duration, or amount of data preparation needed. Nor does it show whether a model trained on the 188-object subset would generalize to other footwear or images that differ from the training data. The authors’ initial observation is not presented as a controlled comparison.
Run the Workflow and Measure Results
Developers can use the published repository to prepare a mesh dataset on the Hugging Face Hub, adapt the scripts, and run preprocessing and training either locally or through Hugging Face Jobs. They will need a Hugging Face token and access to gated DINOv3. The supplied account points to a walkthrough, but does not provide a reported outcome for the shoe training run.
The next useful evidence would be a completed comparison showing how the adapted model performs against the original TRELLIS.2 model on the same inputs. Details on evaluation methods, compute requirements, and performance on footwear outside the training subset would help developers judge whether the workflow is practical for their own projects. Until those results are reported, the release is a usable training setup and a proposed experiment—not proof of improved 3D generation.
Key Questions
What did Hugging Face publish?
Hugging Face published scripts and containers for fine-tuning TRELLIS.2 on custom 3D assets, with options to run locally or through Hugging Face Jobs.
What is the shoe example trying to improve?
It tests whether fine-tuning can improve shoe structure and geometry, including soles that the authors said were inaccurate in initial tests. The supplied account does not report whether it worked.
What training method does the example use?
The workflow uses LoRA adapters through the PEFT library. It trains small adapter matrices while keeping the model’s original weights frozen.
What do developers need to run the workflow?
They need a custom mesh dataset uploaded to the Hugging Face Hub, a Hugging Face token, and access to the gated DINOv3 model. Local runs also require suitable GPU hardware; the source does not specify hardware needs or costs.
Has the fine-tuned model been shown to perform better?
No measured improvement is reported in the supplied material. It provides the training plan but no completed comparison, benchmark, or independent evaluation.
Primary source: Hugging Face · via ThorstenMeyerAI.com
