Stylumia NXT Hackathon 2024

Theme: AI Problem Statement - Universal Fashion Ontology & Feature Extraction System

The fashion industry's complexity demands a sophisticated approach to feature extraction that goes beyond traditional methods. The core challenge lies in developing a universal system capable of understanding and categorizing the intricate details of any fashion item, regardless of its type or origin. This challenge is multifaceted and can be broken down into several key components:

1. Ontological Complexity ( Must Have): Fashion is not just a collection of items; it's a complex language with its grammar and vocabulary. The challenge is to create a comprehensive ontology that can accurately represent the vast spectrum of fashion features across all categories. This ontology must be flexible enough to evolve with fashion trends yet structured enough to maintain consistency
   
   Note - Your final feature extraction output will be as good as the ontology that you create. This Is one of the most crucial steps
   

   Here are some resources on what an Ontology might look like :- https://www.palantir.com/docs/foundry/ontology/overview/

2. Multi-Modal Understanding (Must Have): Fashion items are described through various modalities - visual (images, videos), textual (descriptions, reviews), and sometimes even tactile The system must seamlessly integrate these diverse data types to form a cohesive understanding of each item.

3. Contextual Interpretation (Good To Have): Fashion features often derive meaning from their A "cold shoulder" in a dress means something entirely different from a "cold shoulder" in a sweater. The system must be able to interpret features within their specific context across different fashion categories.

4. Scalability Precision (Must Have): There's a fundamental tension between the need for a system that can handle the vast scale of the global fashion industry and the requirement for precise, nuanced feature extraction. Balancing these competing demands is a core challenge.

5. Temporal Dynamics (Good To Have): Fashion is inherently temporal, with styles and trends evolving rapidly. The system must not only understand current fashion but also be adaptable enough to recognize and categorize emerging trends and features.
   
   Think how your system would capture newer keywords as part of the taxonomy. Ex- half-sleeve as a keyword has always existed , how would your system capture say when a newer sleeve-length comes up or say the same keyword called out differently comes up and everyone’s talking about the same.

6. Bridging Expertise Gaps (Good To Have): While the goal is automation, the system must effectively incorporate human expertise, especially for complex or novel features. The challenge is to create a symbiotic relationship between AI and human fashion experts.
   
   The fundamental challenge is to create a system that can "think" about fashion the way a seasoned fashion expert would, but at a scale and speed that only technology can achieve. This system should not just identify features but understand their significance, their relationships to other features, and their place within the broader context of fashion as a whole.
   
   Such a system would revolutionize numerous aspects of the fashion industry, from design and production to marketing and retail, by providing a universal "language" for describing and analyzing fashion items. The potential applications range from hyper-personalized shopping experiences to more efficient and sustainable design and production processes.
   
   In essence, we're not just solving a classification problem; we're aiming to create a system that truly understands the language of fashion in all its complexity and nuance.

Objective

Develop an AI-driven, universal feature extraction system that can identify and categorize key features of any fashion product across all categories. The system should leverage both visual and textual data, utilizing an agentic workflow and incorporating continuous learning mechanisms.

Key Goals

• Build a Comprehensive and Adaptive Fashion Ontology (Must Have) :
  • Create a detailed, hierarchical representation of fashion features spanning all
  • Implement mechanisms for the ontology to autonomously expand and adapt to new trends and items.
  • Resources -
•  Develop Advanced AI-Driven Feature Extraction Techniques (Must Have)
  • Utilize cutting-edge technologies in computer vision, natural language processing, and multi-modal AI to analyze images and text descriptions
  • Implement AI agents capable of understanding, interpreting, and learning complex fashion attributes across any category.
•  Ensure High Performance and Scalability (Must Have):
  • Optimize the system for speed to allow real-time or near-real-time
  • Maintain high accuracy in feature identification while scaling to handle increasing volumes and varieties of fashion items.
• Achieve Universal Applicability (Good To Have):
  • Design the system to be flexible and applicable to any fashion product category, from apparel to accessories, without reliance on category-specific hard-coded rules.
• Implement Agentic Workflow (Good To Have):
  • Develop AI agents capable of autonomously navigating the feature extraction process, from data ingestion to output generation.
  • Create a human-in-the-loop system where AI agents can request human expertise for complex or novel cases, learning from these interactions.
• Incorporate Continuous Learning and Feedback Loop (Must Have):
  • Design a system that learns from its successes and failures, continuously improving its feature extraction capabilities.
  • Implement mechanisms to incorporate user feedback, expert annotations, and market trends into the learning process.
• Enable Efficient Backfilling (Good To Have):
  • Develop capabilities for the system to efficiently process and annotate existing image
  • Ensure consistency between newly processed items and backfilled

Constraints

• Computational Efficiency: Balancing the depth of analysis with the need for real-time or near-real-time processing.
• Data Quality and Availability: Handling incomplete, inconsistent, or low-quality data inputs across various fashion categories.

Data - You’ll have access to 100K products across multiple categories ( Ex - Dresses , Sneakers , Shirts , Earrings etc) and Departments ( Men's , Women’s , Home & Hardware , Jewellery)

Data Format - CSV

Data Fields -

• category_Id- Unique identifier for product category ( Dresses , Shirts etc) department_Id- Unique identifier for product department (Men’s , Women’s , Boys etc)
• channel_Id- Unique identifier for product channel/Brand-Geography(H&M US ,Myntra IND)
• product_Id- Unique identifier for the product
• description - Detailed product description including materials, sizes, and features meta_info - Additional product metadata
• sku- Stock Keeping Unit identifier brand-Brand name
• feature_image- URL for the main product image product_name-Full product name/title feature_image_s3- S3 storage URL for product image feature_list-List of product features
• category_name- Product category Name retailer_name- Name of the retailer (Meesho IN)

Data Link - Link

The below are the key outputs that are must

1. Ontology

2. Feature Extraction

3. Demo showcasing ( Agentic workflow , Feedback/Learning engine loop)