Dick's Sporting Goods (DSG)

The My Account team did not have consistent methods to get user feedback about the page or ScoreCard Gold. In the first week, I organized 2x/week DSG store visits, where the full team learned how to run customer interviews. I also worked with the PMs to create a "Share Your Thoughts" footer link to passively gather Qualtrics feedback.

In the engagement's first 2 weeks, we interviewed 41 customers, 11 store employees, 5 "best customers," and received 350 Qualtrics responses. We asked broad questions around DSG and loyalty programs.

Dick's shoppers liked the brand's quality and range. But we found they were apathetic towards the ScoreCard. They had no sense of why or when they were receiving rewards. They often equated them with promotional coupons. Nobody we talked to had ever used a Pro Service.

Maybe users weren't familiar with the term "Pro Services" or used "coupon" to mean "reward." In the second week, we listed out all Pro Services and asked about interest. Just 15% had used a pro service in the last year, all at the initial point of purchase. We asked customers what coupons they used; 75% of those who spent over $300 were using basic %-off or brand coupons. Critically, they didn't remember receiving or using $10 rewards at any point.

2 weeks of research proved regular and high-spending shoppers had low appetite for Pro Services. As an informed product team, we pushed back on the Pro Services Gold reimbursement plan Marketing wanted. We instead prioritized a better online progress and delivery system for ScoreCard and Gold because of the confusion and passivity we heard in interviews. There was also business value in getting users to track their progress to Gold ($500): customers who reached Gold were more likely to return and make larger purchases at higher margins.

100% Committed

The My Account team was reluctant to do regular user research. The team was unsure if in-store insights would translate to ecommerce. Our client PM worried about the productivity hit and the optics of being absent once a week.

I pitched regular user research as our weekly grounding point. If Wednesday is our store visit day, then we can plan our entire week around it and reduce cognitive load. The PMs know user research + synthesis (Wednesday + Thursday) creates new stories and priorities, which they can work on Friday and Monday. Designers and PMs prepare for research on Tuesday and the cycle repeats. We are not absent, but doing our expected Wednesday store research.

July onward, we ran 14 tests in 12 weeks (two twice-in-a-week) sessions with 145 users. Our main hypothesis: if we can explain ScoreCard points progress to customers, they will be willing to shop more often and hit their 300/500 point targets.

I had to adapt standard interview into a fast and approachable experience to suit our in-store users. I started handing users our cell phone (instead of asking them to visit a link) to reduce setup friction. Users were distracted by an active notetaker with a clipboard, so we simply observed the interviews and used a template to record what they did immediately after the test.

Sometimes our design ideas fell flat. It was clear within the first two interviews when, for example, users didn't care about a specific perk or missed too-subtle notifications. Unlike previous engagements with specific users, we could simply iterate on the design in-store and find new DSG shoppers to test with.

We sometimes tested our same in-store research plan on UserTesting.com with ecommerce customers as a control. We found the feedback was the same, just more expensive to recruit.

After the Pivotal engagement, the client designers have continued the cadence, averaging 8 user interviews a week.

Enabling Partners

When we said "don't do Pro Services and focus on ScoreCard communication," both marketing and ecommerce stakeholders were concerned. We were talking to too few users, not making data-driven decisions, just doing what our users said: creating friction in a two-year project.