Bailey Thigpen

Home

Projects

About

Contact

Bailey Thigpen

AI / ML CATEGORIZER

Categorizing products went from days to minutes with AI

Human-in-the-loop (HITL) product labeling helps the machine learning model categorize products quickly and accurately in a game-like user experience.

Artificial Intelligence

User Testing

Dark Mode

AI / ML CATEGORIZER

Categorizing products went from days to minutes with AI

Human-in-the-loop (HITL) product labeling helps the machine learning model categorize products quickly and accurately in a game-like user experience.

Artificial Intelligence

User Testing

Dark Mode

AI / ML CATEGORIZER

Categorizing products went from days to minutes with AI

Human-in-the-loop (HITL) product labeling helps the machine learning model categorize products quickly and accurately in a game-like user experience.

Artificial Intelligence

User Testing

Dark Mode

Problem Statement

eCommerce sellers spend many hours manually identifying and labeling the appropriate marketplace category for each of their products.

Problem Statement

eCommerce sellers spend many hours manually identifying and labeling the appropriate marketplace category for each of their products.

Overview

eCommerce sellers need to accurately label their products' categories to list them on a marketplace. Currently, this process requires users to spend many hours reviewing marketplace categories, determining which of their products correspond to which categories, and manually labeling each product's category in their product data -- And this is just the first step in the overall data mapping process! With AI, we developed a process that automates product categorization, reducing the time required from hours or days to just minutes.

Role

Senior Product Designer

Timeline

January - March 2024

Team

Product Manager, 4 Data Scientists, 3 Front-End Engineers, 2 Back-End Engineers

Overview

eCommerce sellers need to accurately label their products' categories to list them on a marketplace. Currently, this process requires users to spend many hours reviewing marketplace categories, determining which of their products correspond to which categories, and manually labeling each product's category in their product data -- And this is just the first step in the overall data mapping process! With AI, we developed a process that automates product categorization, reducing the time required from hours or days to just minutes.

Role

Senior Product Designer

Timeline

January - March 2024

Team

Product Manager, 4 Data Scientists, 3 Front-End Engineers, 2 Back-End Engineers

Overview

eCommerce sellers need to accurately label their products' categories to list them on a marketplace. Currently, this process requires users to spend many hours reviewing marketplace categories, determining which of their products correspond to which categories, and manually labeling each product's category in their product data -- And this is just the first step in the overall data mapping process! With AI, we developed a process that automates product categorization, reducing the time required from hours or days to just minutes.

Role

Senior Product Designer

Timeline

January - March 2024

Team

Product Manager, 4 Data Scientists, 3 Front-End Engineers, 2 Back-End Engineers

Discover

Discover

What is "category mapping"?

Category mapping is the process of identifying and labeling the appropriate marketplace category for each of a seller's products. To list their products on a marketplace, sellers must assign a specific data point to each product indicating its category.

What is "category mapping"?

Category mapping is the process of identifying and labeling the appropriate marketplace category for each of a seller's products. To list their products on a marketplace, sellers must assign a specific data point to each product indicating its category.

What is "category mapping"?

Category mapping is the process of identifying and labeling the appropriate marketplace category for each of a seller's products. To list their products on a marketplace, sellers must assign a specific data point to each product indicating its category.

What is the process today?

I conducted 6 user interviews to better understand the current process by having users walk me through specific examples of how they currently approach category mapping. Here's what I learned:

What is the process today?

I conducted 6 user interviews to better understand the current process by having users walk me through specific examples of how they currently approach category mapping. Here's what I learned:

What is the process today?

I conducted 6 user interviews to better understand the current process by having users walk me through specific examples of how they currently approach category mapping. Here's what I learned:

*repeat for EVERY marketplace!*

What are the pain points?

Users communicated a lot of frustrations around the current category mapping process:

What are the pain points?

Users communicated a lot of frustrations around the current category mapping process:

What are the pain points?

Users communicated a lot of frustrations around the current category mapping process:

Define

Define

How could AI help?

Collaborating with our data science team, we discovered that implementing a machine learning model for category mapping could significantly reduce our users' time and effort. Initially, the model would require human input through a Human-in-the-Loop (HITL) experience. Here's how it works:

How could AI help?

Collaborating with our data science team, we discovered that implementing a machine learning model for category mapping could significantly reduce our users' time and effort. Initially, the model would require human input through a Human-in-the-Loop (HITL) experience. Here's how it works:

How could AI help?

Collaborating with our data science team, we discovered that implementing a machine learning model for category mapping could significantly reduce our users' time and effort. Initially, the model would require human input through a Human-in-the-Loop (HITL) experience. Here's how it works:

The AI model will analyze the input (the user's product data), categorize products according to the marketplace's requirements, and assign a confidence level to each category mapping.

If the model is not confident in a product's category mapping, it will surface that product and the user will manually select the correct category for the product. The model will learn from this input and apply the knowledge to other similar products.

Defining product requirements

The ML model had specific requirements and outputs, making it essential to collaborate closely with the data science team to design an experience that would ensure success for both the model and the user.

Defining product requirements

The ML model had specific requirements and outputs, making it essential to collaborate closely with the data science team to design an experience that would ensure success for both the model and the user.

Defining product requirements

The ML model had specific requirements and outputs, making it essential to collaborate closely with the data science team to design an experience that would ensure success for both the model and the user.

  • Users will annotate a sample set of products (between 20 and 100), depending on the ML model's confidence

  • Users must have enough information on the product to be able to identify its category (image, title, description, others?)

  • Users will be presented with 3 category predictions per product

  • Categories will be at the lowest subcategory level

  • Users can select an "I don't know" option to skip the product

  • Users must select a category for each product to finish the process

  • Users should have some sort of progress indicator

  • After users have reviewed the sample set of products, they can review and edit the final output

  • The experience should be fun and delightful, perhaps including gamification elements

Defining design goals

After all the discussions I had with various teams, I concluded that for the design to be successful it must accomplish the following:

Defining design goals

After all the discussions I had with various teams, I concluded that for the design to be successful it must accomplish the following:

Defining design goals

After all the discussions I had with various teams, I concluded that for the design to be successful it must accomplish the following:

Speed

Users can review products as quickly and easily as possible.

Accuracy

Users have all of the product info they need to select the correct category.

Confidence

Users should gain confidence in our product and in our AI / ML capabilities.

Easy Final Review

Users can easily review + edit the final output of categories for ALL of their products.

Delight

Users should enjoy this experience - it should feel easy and fun!

Ideate

Ideate

Wireframing the human-in-the-loop (HITL) experience

I began brainstorming various ways to create an experience that allows users to quickly label the category for a sample set of products. Here’s a glimpse into the ideation process:

Wireframing the human-in-the-loop (HITL) experience

I began brainstorming various ways to create an experience that allows users to quickly label the category for a sample set of products. Here’s a glimpse into the ideation process:

Wireframing the human-in-the-loop (HITL) experience

I began brainstorming various ways to create an experience that allows users to quickly label the category for a sample set of products. Here’s a glimpse into the ideation process:

Narrowing down the options

All of the wireframed options above essentially boiled down to 2 main approaches: users either select the category for products one by one, or in bulk via a list of all products to categorize.

Narrowing down the options

All of the wireframed options above essentially boiled down to 2 main approaches: users either select the category for products one by one, or in bulk via a list of all products to categorize.

Narrowing down the options

All of the wireframed options above essentially boiled down to 2 main approaches: users either select the category for products one by one, or in bulk via a list of all products to categorize.

CONCEPT A

Users select the category for products one by one.

Pros

  • The product card is easily digestible

  • Plenty of room to provide as much info on the SKU as needed

  • CTA is front and center

Cons

  • How does the user know when they're done? Progress bar helps but is vague

  • Will user be able to review lots of products quickly if it's just one at a time?

CONCEPT B

Users have a list of all products to categorize.

Pros

  • Users can filter products to focus on particular areas

  • Bulk action option so users can categorize multiple products at once

  • This table pattern is common in our app, it would be familiar to users

Cons

  • Not much direction for the user

  • Smaller product image size

Ultimately, the team wasn't sure which option would meet our goal of users labeling products as fast as possible. There's an easy way to figure that out… user testing! 🙌

User Testing

User Testing

Developing the research plan

Working quickly to meet our tight deadline, we created a research strategy involving 6 total users.

Developing the research plan

Working quickly to meet our tight deadline, we created a research strategy involving 6 total users.

Developing the research plan

Working quickly to meet our tight deadline, we created a research strategy involving 6 total users.

Participants

6

Total users of varying experience levels

Users for 5+ years

Users for 2-3 years

Users for less than 1 year

A/B Testing Strategy

  • In a Figma prototype, users will annotate a sample set of 20 products from their own catalog.

  • Users will complete this exercise on both Concept A (one at a time) and Concept B (list view).

  • We will measure how long it takes each user to annotate the sample set with each concept.

Goal

Determine which user experience produces the fastest annotation result.

Testing concept A vs concept B

Users interacted with each prototype by labeling the correct category for 20 of their products. We measured the time it took each user to complete labeling for each concept.

Testing concept A vs concept B

Users interacted with each prototype by labeling the correct category for 20 of their products. We measured the time it took each user to complete labeling for each concept.

Testing concept A vs concept B

Users interacted with each prototype by labeling the correct category for 20 of their products. We measured the time it took each user to complete labeling for each concept.

Concept A

Testing Notes

Users were able to start annotating right away - the cta was clear

Users thought this was actually fun - like a video game!

Users weren't sure what to expect when this process was finished

Concept B

Testing Notes

Users found filters and bulk options helpful - they're used to reviewing/editing product data that way

Users weren’t sure how to interact with the page right away

Users spent a lot of time up front trying to strategize how they could filter the products into helpful groups

Users found images to be the most helpful when categorizing, but in a table the images are small

And the winner is...

Ultimately, concept A resulted in products being labeled the fastest, with an average of 4 seconds per product compared to option B at 22 seconds per product.

And the winner is...

Ultimately, concept A resulted in products being labeled the fastest, with an average of 4 seconds per product compared to option B at 22 seconds per product.

And the winner is...

Ultimately, concept A resulted in products being labeled the fastest, with an average of 4 seconds per product compared to option B at 22 seconds per product.

🎉 Concept A 🎉

Users were able to label the product's category in 4 seconds on average!

Since the primary action was prominently featured, users were able to focus on their goal and categorize products quickly and easily. Concept A streamlined this process, making it feel engaging and efficient. In contrast, Concept B, which included filtering and bulk action options (a common pattern in our application) caused users to spend excessive time upfront strategizing how to group their products. This approach also resulted in more mistakes when categorizing multiple products simultaneously.

One of our design goals was to make the process as enjoyable as possible, so we were thrilled when a user compared Concept A to playing a video game!

Design

Design

Fleshing out the design

With the general direction finalized, I created high-fidelity designs while also exploring some improvements to the content and layout.

Fleshing out the design

With the general direction finalized, I created high-fidelity designs while also exploring some improvements to the content and layout.

Fleshing out the design

With the general direction finalized, I created high-fidelity designs while also exploring some improvements to the content and layout.

  1. Added all product images since users stated that was most helpful

  2. Added a callout for which marketplace the user is selecting categories for

  3. Shortened the action statement

  4. Added a quick link for the user to view all of the marketplace's categories if necessary

  5. Added the ability for users to go backwards in case they made a mistake

  6. Restructured the product data layout to make it look kind of like a product listing, and added the price data point

  7. Stacked the category options so they're easier to compare

  8. Added option for the user to enter a category if none of our predictions are correct

  9. An extra progress callout so the user knows where they are in the process

A new constraint was discovered

During the design phase, we discovered that the progress bar, and therefore the overall process of labeling products, couldn't work the way we imagined it would.

A new constraint was discovered

During the design phase, we discovered that the progress bar, and therefore the overall process of labeling products, couldn't work the way we imagined it would.

A new constraint was discovered

During the design phase, we discovered that the progress bar, and therefore the overall process of labeling products, couldn't work the way we imagined it would.

We initially expected users to label products until the progress bar reached 100%, indicating that the machine learning model had gathered sufficient data to categorize the rest of the user’s product catalog. This approach would require real-time communication between the model and the interface to update the progress bar, but our engineers indicated that this isn't feasible at this stage.

This introduces several challenges, both technically and in terms of user experience. We need to address questions such as: When will the user know they have completed the labeling process? How do we effectively communicate progress to them?

Designing a solution

The solution to this new constraint led us to redefine the overall process and user flow.

Designing a solution

The solution to this new constraint led us to redefine the overall process and user flow.

Designing a solution

The solution to this new constraint led us to redefine the overall process and user flow.

Instead of the model learning in real time, users will label a pre-determined number of products based on a percentage of the products in the user's catalog. The machine learning model will do more work upfront (instead of in real time) to select products from diverse categories and that it's particularly uncertain about.

After completing the labeling of this set, the user will receive a confidence score indicating how confident the model is in labeling the remaining products in their catalog. The user can then choose whether to let the model proceed with labeling the rest of their products or to label additional products to improve the confidence score.

Design Changes

  • Instead of showing a percentage for progress, show the number of products to review. This keeps the gamification element that users liked in testing.

  • New screen - once the user is finished labeling a set of products, show a confidence score with options to either move forward or label more products

  • More loading states will be required while the model is generating the product set and the confidence score

Result

Result

Final designs

Here are the final designs for the end-to-end human-in-the-loop (HITL) user experience, including how users review the results of the machine learning model's categorization of their entire product catalog.

Final designs

Here are the final designs for the end-to-end human-in-the-loop (HITL) user experience, including how users review the results of the machine learning model's categorization of their entire product catalog.

Final designs

Here are the final designs for the end-to-end human-in-the-loop (HITL) user experience, including how users review the results of the machine learning model's categorization of their entire product catalog.

Prototypes and interaction design

To go above and beyonce on the user experience, I spent a lot of time refining the animations, transitions, and timing of each interaction.

Prototypes and interaction design

To go above and beyonce on the user experience, I spent a lot of time refining the animations, transitions, and timing of each interaction.

Prototypes and interaction design

To go above and beyonce on the user experience, I spent a lot of time refining the animations, transitions, and timing of each interaction.

The Introduction

We added a screen to introduce the user to the concept of AI mapping and the overall process they're about to get into.

Users can enter this flow from a couple areas within the app, so we added a marketplace dropdown so the user can ensure they're mapping categories for the correct channel.

Human-in-the-Loop (HITL)

This is where users review each product and select the correct category to train the AI model. The product image is front and center, and there's more product data if the user needs it.

It took many iterations to get the transition timing right, and ensure the interactions were clear and smooth between each product.

Confidence Score

After the user labels their last product, there's a celebratory animation while the AI confidence score loads.

The user can decide to either move forward and allow the AI model to categorize the rest of their products, or to repeat the labeling process to increase the confidence before moving forward.

Review AI Results

Here, the user reviews how all of their products were categorized by the AI model. There's a top-level category panel on the left, and the subcategory sections on the right.

Users can select multiple products via the checkbox in the product cards to move them to a different category in bulk.

Once the user is happy with the product categories, they select Finalize in the top right to finish the process.

The impact

Not only did this project significantly reduce the amount of time it takes for a user to categorize their products, it also won my company an award!

The impact

Not only did this project significantly reduce the amount of time it takes for a user to categorize their products, it also won my company an award!

The impact

Not only did this project significantly reduce the amount of time it takes for a user to categorize their products, it also won my company an award!

Greatly reduced user time and effort

BEFORE

Hours - Days

Category mapping manually using spreadsheets was a nightmare and took forever.

AFTER

Minutes!

With the AI "magic mapper" as users were calling it, they could categorize all of their products for a marketplace is a matter of minutes!

An AI innovation award!

The mvp version of this product has already won my company an award by Hermes for AI innovation!

Thanks for scrolling!

I'm always open to discussing new opportunities to connect and create.

Contact Me

thigpen.j.bailey@gmail.com

+1 (919) 818-0813

LinkedIn

Thanks for scrolling!

I'm always open to discussing new opportunities to connect and create.

Contact Me

thigpen.j.bailey@gmail.com

+1 (919) 818-0813

LinkedIn