DTM E26. Using Generative AI for Fashion Apparel Cataloging – Dr. Amritendu, NeuroPixel.ai
From the Deep Tech Musings Podcast - Get actionable and tactical insights to take your Deep Tech startup from 0 to 1 [Idea to Traction]
Listen now on - Spotify, Apple, Google
Dr. Amritendu is an AI researcher and the co-founder of NeuroPixel.ai, an Indian startup that currently aims to build the future of fashion cataloging using generative AI. In this episode, Dr. Amritendu walks us through his research experiences, his fight with cancer, going through the Entrepreneur First program, his startup story, the tech enabling the generative AI solution, and his future ambitions. More importantly, he calls upon the need to bridge the gap between research & industry and solve hard problems.
Listen to the episode here,
or on the below platforms
Where to find us:
Amritendu Mukherjee – Amritendu Mukherjee (LinkedIn)
Neuropixel.ai - Neuropixel.ai (LinkedIn)
Pronojit Saha, DTM Podcast - pronojitsaha (LinkedIn), @pronojits (Twitter)
Show Notes & Summary:
(2:22) Amritendu’s inspiring personal journey battling cancer
Amritendu completed his PhD in machine learning, deep neural networks, statistical learning theory, and probability theory, with a focus on computer vision and image processing.
During his PhD, Amritendu was diagnosed with stage four blood cancer, and underwent three years of comprehensive treatment, including chemotherapy and biopsies.
Despite the physical strain, he continued with his research and successfully completed the thesis while undergoing treatment.
His experience with cancer taught him valuable life lessons and shaped his perspective, and he has now founded his own startup.
(7:40) Amritendu’s dive into the world of research and finding life’s purpose
He reflects on the importance of independence and freedom in research, especially after recovering from his treatment.
He shares a personal moment of realization while swimming alone in a pool, emphasizing the importance of appreciating life and pursuing what truly matters.
He highlights the influence of Randy Pausch's last lecture and the idea of working on things that are important to oneself.
He stresses on the significance of bridging the gap between theoretical research and practical applications for the benefit of humanity, citing their interest in cancer biology and the importance of immunotherapy.
He mentions the need to understand the limitations of research but also the importance of pushing those limitations to make progress and improve the current situation.
A reference is made to Naval Ravikant's definition of happiness as achieving something greater than oneself and not worrying about others' opinions.
He muses on the transient nature of existence and the importance of realizing it.
(14:05) Amritendu’s experience with the Entrepreneurs First startup program
Amritendu joined Entrepreneurs First in 2020, attracted by their focus on deep tech and lab-to-market fit.
Entrepreneurs First has notable sponsors from companies like Demis Hassabis from Google DeepMind and Reid Hoffman from LinkedIn.
Prior to joining, he was completing his PhD and had a growing desire to apply theoretical concepts to build something that could improve processes.
Entrepreneurs First is a six-month program where top talents are identified and their entrepreneurship skills are developed.
Amritendu met his co-founder, Arvind Nayer, at Entrepreneurs First and they jointly began exploring the areas where they have the "right to win" and build their startup.
(17:47) A problem they identified that can be a game changer
Amritendu comes from an AI/ML/CV background, while Arvind has experience in the fashion e-commerce industry.
They identified a pressing problem in the cataloging process of fashion e-commerce websites, which is currently manual-intensive and not easily adaptable to remote work during the COVID-19 pandemic.
Their goal is to automate a significant portion of the cataloging process using AI, ML, and computer vision technologies.
Fashion retailers currently rely on physical photo shoots for cataloging, which has limitations and lead times of around 15 days.
Personalization is a crucial aspect in the fashion industry, and the founders aim to address this by showing different models with different sizes of apparel without the need for physical photo shoots. This can significantly improve customer decision-making and conversion rates.
(25:20) Finding the solution to this problem using generative AI
The ideal solution would be to show any model wearing any apparel with a click of a button, but this is a complex problem with infinite possibilities.
The current solution involves shooting the apparel once and reproducing it as many times as needed.
Fashion brands and online platforms are looking for deep tech solution providers to tackle this problem.
The technical process involves segmentation, warping, fusion, and missing part generation using conditional GAN and traditional image processing algorithms.
Understanding the deformation based on shape, pose, and volumetric information is crucial, but it is still an unsolved problem and requires further research.
(33:25) Hardest technical challenges that Amritendu and his team are facing currently
Refinement of segmentation is still a challenge, especially for intricate details.
Achieving perfect segmentation is crucial for fashion e-commerce platforms, as even tiny errors are noticeable.
Estimating 3D parameters from 2D images is also a challenging research area.
The training data volume requirement varies, but to start with, around 5,000 images per apparel type are estimated, with a focus on generalization.
Transfer learning is expected to help gradually reduce the number of required training images for each apparel type.
(37:59) Lab vs real-world performance metrics
Performance metrics like structural similarity index are used in academic settings to evaluate algorithm performance, but real-world deployment presents different challenges.
Implementing research in practical applications brings its own nuances and complexities, which is exciting for engineers.
Assessing the algorithm's capability to produce outputs indistinguishable from the actual images is crucial for customer satisfaction, and customer testing is conducted for this purpose.
(43:56) What can we expect in the future from NeuroPixel in generative AI?
Neuropixel.ai aims to be at the forefront of AI, machine learning, and computer vision research, focusing on solving unsolved problems across various verticals, including e-commerce, synthetic media generation, satellite imagery analysis, and traffic solutions.
While they have a broad vision, their initial focus is on fashion e-commerce, particularly in developing AI-powered cataloging and virtual try-on solutions.
One specific problem they are interested in is generating completely synthetic AI models for visual marketing, allowing personalized ads with changes in background, attire, or the person featured.
The idea of changing attire in a similar way as changing backgrounds is intriguing and aligns with their goal of exploring innovative and impactful ideas.
(47:00) Where does Amritendu see AI/ML progressing in the next few years and what are some personalities he looks up to?
He agrees with the concept of data-centric AI promoted by Andrew Ng and emphasizes the need for AI to be more explainable, understanding its learning process and important parameters.
He is anticipating significant work in the field of immersive experiences, particularly in the AR/VR space, and highlights the importance of enjoying the journey and having fun in what you do.
As a researcher, he finds inspiration in figures such as Professor Andrew Ng and Professor Joshua Bengio for their contributions to education and deep learning, respectively.
He also mentions Professor Michael Black for his inspiring work in computer vision, and personally admire personalities like Sir Edmund Hillary and Rahul Dravid for their achievements and values as human beings.
Ultimately, he emphasizes on the importance of striving to be better human beings and finding happiness in personal growth.
If you enjoyed this episode, please leave us a rating on Spotify, Apple, or wherever you listen to podcasts. It helps us reach more people who are interested in deep tech & grow the community.
Also, don’t forget to subscribe to Deep Tech Musings Podcast on pronojits.substack.com so you never miss an episode.
Thank you for listening! See you next time!