Autonomous Spatial Apparel Placement via Multimodal Vision Pipelines
Integrating Google Gemini Vision and 3D offline configurators to autonomously generate weekly store-floor layout configurations for Tata Trent Westside.
Developed for Tata Trent Westside (National Fashion & Retail Chain).
Excel parsing (Ascot/Wardrobe) by display location (table, hanger, shelf)
Bounding-box cropping & Lanczos proportional centering on 1024x1024
Diffusion & vision prompt preserving fabric texture & folding symmetry
Offline C# configurator rendering physical store fixture clearances
The Problem: Scaling Physical Merchandising Across National Outlets
Store managers across hundreds of national retail outlets previously spent hours interpreting static PDF lookbooks and manually arranging floor displays. Furthermore, store Wi-Fi was notoriously inconsistent, requiring an offline-first desktop runtime capable of synchronizing when network availability was restored.
Multimodal Spatial Vision Architecture
The architecture bridges cloud AI visual analysis with an offline desktop spatial simulator:
- ›Autonomous SKU ingestion pipeline parsing Ascot and Wardrobe seasonal catalogs directly into structured display matrices.
- ›Automated background removal (rembg) and alpha-channel bounding box calculation with Lanczos proportional canvas re-centering (1024x1024).
- ›Google Gemini 2.0 multimodal prompting enforcing folding symmetry, fabric texture preservation, and pure white e-commerce rendering.
- ›Offline-capable Windows desktop 3D store configurator in C# calculating 3D fixture bounding-box clearances and visual merchandising density.
import os
from google import genai
from PIL import Image
client = genai.Client()
def generate_folded_apparel_prompt(sku: str, color: str, location: str) -> str:
"""
Generates deterministic multimodal prompt for retail planogram rendering
"""
return f"""Generate a high-resolution, realistic product image of a folded {color} garment
(SKU: {sku}) for display at {location}. The final image should meet the following criteria:
1. Folding Style: Neatly folded as in retail packaging; maintain symmetry and alignment in folds.
2. Realism & Texture: Natural fabric texture (cotton, linen, denim); soft diffused daylight only.
3. Framing: Centered 1024x1024 px square output with clean spacing.
4. Background: Pure white or transparent background suitable for planogram ingestion.
5. Exclusions: No human models, mannequins, props, dramatic lighting, or stylized edits.
6. Color Fidelity: Exact {color} fidelity without tinting or filters."""from PIL import Image
import numpy as np
def align_garment_to_canvas(image: Image.Image, target_size=(1024, 1024)) -> Image.Image:
"""
Calculates alpha bounding box and centers garment on target transparent canvas
"""
if image.mode != 'RGBA':
image = image.convert('RGBA')
alpha = image.split()[-1]
bbox = alpha.getbbox()
if bbox is None:
return Image.new('RGBA', target_size, (0, 0, 0, 0))
cropped = image.crop(bbox)
scale_factor = min(target_size[0] / cropped.width, target_size[1] / cropped.height) * 0.9
new_size = (int(cropped.width * scale_factor), int(cropped.height * scale_factor))
scaled = cropped.resize(new_size, Image.Resampling.LANCZOS)
result = Image.new('RGBA', target_size, (0, 0, 0, 0))
x = (target_size[0] - new_size[0]) // 2
y = (target_size[1] - new_size[1]) // 2
result.paste(scaled, (x, y), scaled)
return resultKey Takeaways & Architectural Insights
- 01Multimodal vision models are extraordinarily capable at qualitative spatial reasoning when paired with rigid physical geometry constraints.
- 02Combining automated alpha masking (rembg) with Lanczos re-scaling produces production-ready 3D textures from raw warehouse photos.
- 03Offline desktop software remains indispensable for physical retail environments with unreliable mall connectivity.
Build Spatial AI Pipelines
Consult with Aabhas on multimodal vision, retail automation, and offline-first 3D systems.