Spatial AI & Vision·Nov 2024·8 min read

Autonomous Spatial Apparel Placement via Multimodal Vision Pipelines

Integrating Google Gemini Vision and 3D offline configurators to autonomously generate weekly store-floor layout configurations for Tata Trent Westside.

Architect:Aabhas Kumar Jha (AI & Full-Stack Systems Engineer)

Developed for Tata Trent Westside (National Fashion & Retail Chain).

SYS-ARCH // 05 · TATA TRENT WESTSIDEMultimodal Gemini 2.0 Vision Pipeline & 3D Retail Fixture Planogram
LIVE TELEMETRY|60 FPS VECTOR ENGINE
++++
TATA TRENT WESTSIDE · SPATIAL AI & 3D STORE PLANOGRAMCAD RUNTIME: 3D OFFLINESTAGE 1: GEMINI VISIONSKU: ASCOT LINENrembg: ALPHA MASK CUTOUTLANCZOS 1024x1024 CANVFOLD SYMMETRY: VERIFIED ✓100% COTTONCOLOR FIDELITY: OKSTAGE 2: 3D STORE FIXTURE PLANOGRAMASCOT (NAVY)WARDROBE (SLATE)RESERVE INVENTORY CLEARANCE: 18cmSTORE #104FIXTURE G-02CAPACITY: 94%
Fig. ARCH //Nationwide store visual merchandising automation: rembg alpha masking and Lanczos 1024x1024 proportional canvas centering via Gemini 2.0 Vision, feeding an offline Windows C# 3D retail fixture planogram with real-time clearance and capacity validation.
Deployment Scope
National Fleet
Tata Trent Westside retail stores
Visual Pipeline
Multimodal Gemini
Spatial apparel placement & validation
Offline Runtime
Windows C# 3D
Offline-capable desktop configurator
Automation
Weekly SKU Cadence
Direct floor plan generation to managers
Autonomous Store Merchandising Pipeline
STAGE 01
Catalog Ingestion

Excel parsing (Ascot/Wardrobe) by display location (table, hanger, shelf)

STAGE 02
rembg Alpha Masking

Bounding-box cropping & Lanczos proportional centering on 1024x1024

STAGE 03
Gemini Vision AI

Diffusion & vision prompt preserving fabric texture & folding symmetry

STAGE 04
3D Planogram Gen

Offline C# configurator rendering physical store fixture clearances

The Problem: Scaling Physical Merchandising Across National Outlets

Store managers across hundreds of national retail outlets previously spent hours interpreting static PDF lookbooks and manually arranging floor displays. Furthermore, store Wi-Fi was notoriously inconsistent, requiring an offline-first desktop runtime capable of synchronizing when network availability was restored.

Multimodal Spatial Vision Architecture

The architecture bridges cloud AI visual analysis with an offline desktop spatial simulator:

  • ›Autonomous SKU ingestion pipeline parsing Ascot and Wardrobe seasonal catalogs directly into structured display matrices.
  • ›Automated background removal (rembg) and alpha-channel bounding box calculation with Lanczos proportional canvas re-centering (1024x1024).
  • ›Google Gemini 2.0 multimodal prompting enforcing folding symmetry, fabric texture preservation, and pure white e-commerce rendering.
  • ›Offline-capable Windows desktop 3D store configurator in C# calculating 3D fixture bounding-box clearances and visual merchandising density.
Production Multimodal Prompt Engineeringfolded_sku_generator.py
import os
from google import genai
from PIL import Image

client = genai.Client()

def generate_folded_apparel_prompt(sku: str, color: str, location: str) -> str:
    """
    Generates deterministic multimodal prompt for retail planogram rendering
    """
    return f"""Generate a high-resolution, realistic product image of a folded {color} garment 
(SKU: {sku}) for display at {location}. The final image should meet the following criteria:

1. Folding Style: Neatly folded as in retail packaging; maintain symmetry and alignment in folds.
2. Realism & Texture: Natural fabric texture (cotton, linen, denim); soft diffused daylight only.
3. Framing: Centered 1024x1024 px square output with clean spacing.
4. Background: Pure white or transparent background suitable for planogram ingestion.
5. Exclusions: No human models, mannequins, props, dramatic lighting, or stylized edits.
6. Color Fidelity: Exact {color} fidelity without tinting or filters."""
Alpha Masking & Lanczos Centering Engineprocess_images.py
from PIL import Image
import numpy as np

def align_garment_to_canvas(image: Image.Image, target_size=(1024, 1024)) -> Image.Image:
    """
    Calculates alpha bounding box and centers garment on target transparent canvas
    """
    if image.mode != 'RGBA':
        image = image.convert('RGBA')
    
    alpha = image.split()[-1]
    bbox = alpha.getbbox()
    if bbox is None:
        return Image.new('RGBA', target_size, (0, 0, 0, 0))
    
    cropped = image.crop(bbox)
    scale_factor = min(target_size[0] / cropped.width, target_size[1] / cropped.height) * 0.9
    new_size = (int(cropped.width * scale_factor), int(cropped.height * scale_factor))
    scaled = cropped.resize(new_size, Image.Resampling.LANCZOS)
    
    result = Image.new('RGBA', target_size, (0, 0, 0, 0))
    x = (target_size[0] - new_size[0]) // 2
    y = (target_size[1] - new_size[1]) // 2
    result.paste(scaled, (x, y), scaled)
    return result

Key Takeaways & Architectural Insights

  • 01Multimodal vision models are extraordinarily capable at qualitative spatial reasoning when paired with rigid physical geometry constraints.
  • 02Combining automated alpha masking (rembg) with Lanczos re-scaling produces production-ready 3D textures from raw warehouse photos.
  • 03Offline desktop software remains indispensable for physical retail environments with unreliable mall connectivity.

Build Spatial AI Pipelines

Consult with Aabhas on multimodal vision, retail automation, and offline-first 3D systems.