Mixed Method: Creator Rewards Program Beta Launch
Background
Team: Creator and Content monetization
role: lead researcher
Timeline: 6 weeks
product: Creator rewards program (CRP) Additional Rewards
Previously, the Creator Rewards Program consisted only of Standard Rewards, which were calculated based on the video views and Revenue Per Mille (RPM). The Creator and Content team launched a new monetization strategy that not only rewarded creators from Standard Rewards, but also included Additional Rewards, which rewarded high quality content (e.g., 1080p+, niche, etc). This study aimed to assess the new CRP monetization strategy with a quantitative and qualitative approach.
Methodology: large scale survey + interviews
I chose to start with a quantitative large scale survey first to identify broad patterns and quantify satisfaction/perceptions. From there, I conducted in depth interviews to dig deeper into the “why”, leveraging quantitative insights to structure interviews. Probing into the motivations, pain points, and contextual nuances provided more color to complement survey insights.
Objective
The Creator and Content team beta-launched a new monetization model called “Additional Rewards.” Previously, creator monetization relied solely on “Standard Rewards,” which paid creators based on video views. Additional Rewards represented a strategic shift—rewarding not only high engagement (from Standard Rewards), but also high quality content (from Additional Rewards). I led a two-phase mixed method research study to assess creator satisfaction, understand perceptions of the design changes, and identify opportunities to refine and strengthen the monetization strategy overall.
research questions
How satisfied are creators in the beta launch with the new CRP Additional Rewards monetization strategy?
How aware are creators in the beta launch with the Additional Rewards feature? How can we improve the new CRP Additional Rewards monetization strategy?
How does the new CRP design compare to the previous design?
Methodology
Total Sample Size Break Down (Top Tier: ~500k+ followers; Middle Tier: ~50k-500k followers; Low Tier: ~10-50k followers)
1. large scale Survey
participants
N=About 4,000 responses across US, UK, KR, JP, MX, and BR regions.
I launched the survey to all creators who participated in the beta launch (a total of ~80,000). We had about a 5% response rate with support from operations (e.g., account managers, partners, etc).
*Due to the smaller sample sizes from Korea, Japan, and Mexico, findings from these regions were treated as directional signals rather than definitive insights. While they highlight potential trends and areas of opportunity, they should be validated through additional research.
*All numbers are approximations and are changed due to confidentiality.
procedure
Respondent List Preparation: The product team provided a list of all the creator UIDs who participated in the beta launch
Launch: I launched with an internal survey tool called ByteSurvey which sends the survey via an in-app notification directly on the TikTok app.
Data Collection: The survey was in field for about 3-5 days
Analysis: After I aligned with the product team on the total sample size as well as a regional breakdown, I downloaded the data set and analyzed the data via R. One limitation of the internal tool was that it did not allow us to distinguish responses from top-tier, mid-tier, and lower-tier creators.
application: Internal survey tool called Bytesurvey
I launched via an internal tool called ByteSurvey, which closely mimics Qualtrics. The internal tool allows creators to take the survey from their TikTok in-app notification.
method justification: confidently capture precise perceptions and satisfaction
By capturing insights from a larger sample size, the team can reliably identify users’ perceptions and satisfaction of the new Additional Rewards feature. However, there are a few limitations with a survey. For example, it was not possible to differentiate survey responses by creator tier (e.g., Top, Middle, Low) due to constraints with the internal survey tool ByteSurvey. While I considered including a demographic question asking creators to disclose their general follower count, this was taken out by the legal team during the scoping phase because it was considered sensitive.
2. In-depth interviews
participants
I recruited participants from survey responses and reached out to participants who were interested in a participating in a follow-up interview.
Must be from the US (To maximize efficiency within the project’s timeline, I focused exclusively on U.S. participants, drawing from the largest respondent pool (n=1,500))
N=15 total US creators
5 were Top Tier Creators (~500k+ followers)
5 were Middle Tier Creators (~50k-500k followers)
5 were Low Tier Creators (~10k-50k followers)
Procedure
Introduction (5min): Provide instructions and built rapport
Overall Satisfaction of CRP (15min): Understand their general satisfaction with the Creator Rewards Program overall
Additional Rewards Awareness/Perceptions (20min): Understand their awareness and perceptions of the new Additional Rewards
New Design Feedback (15min): I asked creators to walk through how they normally look at their their Creator Rewards to better understand perceptions and usability of the new designs
Wrap-up Questions (5min): Follow-up questions were asked to allow participant to reflect on their overall experience and provide any other feedback
application: Zoom calls
I recruited participants by reaching out via email, which they provided within the survey if they were interested in participating in a follow-up interview. I used Calendly to schedule a time, and then I interviewed participants via Zoom.
method justification: nuanced contextual understanding of perception and satisfaction
Given the survey limitations, the in-depth interviews allowed the team to fully understand the context of a creators experience, which can be vastly different depending on what tier they are, whether they create part time or full time, and the type of content they post. Survey results from phase 1 were leveraged to shape the discussion guide, allowing me to probe more deeply into the pain points highlighted by the survey.
Key Findings
1. Low satisfaction (20% top2) with additional rewards was likely driven by limited benefits experienced by mid-low tier creators, suggesting the feature should prioritize top tier creators as their targeted audience.
Survey insights show that majority (73%) of creators were neutral about Additional Rewards, likely because many (65%) reported little to no changes in their overall revenue since the launch of Additional Rewards (see appendix).
Interview insights validated this trend. While mid-low tier creators generally saw little to no changes in revenue, most top tier creators reported increased earnings since the launch of Additional Rewards. Top creators were already focused on producing high-quality content which aligned with the Additional Rewards structure.
Overall, more creators were satisfied with the entire Creator Rewards Program (60% top2) compared to the new Additional Rewards feature (20% top2). Interviews revealed that while most creators were grateful that the CRP allowed them to make money while also pursuing a career they felt passionate about, many did not experience tangible benefits from Additional Rewards. This is likely because mid-low-tier creators, who represented over 80% of the beta launch sample size (n=3,275), generally reported no meaningful impact, which aligns with the 73% who felt neutral about the feature.
This suggests that the team should prioritize top tier creators as the target audience when launching Additional Rewards, rather than mid-low tier creators who face a steeper ramp-up before realizing any meaningful benefits.
2. Only 5% noticed the additional reward detail pages due to unintuitive entry points, suggesting that better navigation can improve education and boost usage.
Survey results showed that only 20% of creators had heard of Additional Rewards, 12% were familiar with the feature, and just 3% felt like they understood how it worked. This may be because only 5% of creators noticed the detail pages, which contained key information to explain Additional Rewards purpose and eligibility criteria.
Interview insights brought more clarity and direction for the product design team. Many creators reported they had never seen the Additional Rewards detail pages because the entry point felt hidden and unintuitive. Navigation to the detail pages takes 3-4 additional clicks from the CRP home screens (as shown in image below), and most expected a long list of dry descriptions. When directed to the detail pages, many were pleasantly surprised at how interactive and engaging it was, but often struggled to return after exiting. Many preferred the information be surfaced higher in the navigation, rather than buried behind so many clicks.
Improving navigation to the detail pages will help creators better understand how to align their content with the Additional Rewards criteria. This is especially important for top tier creators and potential top tier creators who often only need minor adjustments to their content creation, before they see increased revenue from rewards.
3. 86% of creators (top2 box) preferred the new CRP designs because of the clearer graphs, suggesting the team should move forward with updated designs.
Survey results showed that most creators (86% top2 box) preferred the new CRP designs compared to the old designs (8% top2 box). Most creators (65-67%) reported the new design was clearer, more useful, and more visually appealing compared to the old design (28-32%).
Interview insights provided more detail, showing that the new bar graph was the key driver for its improved clarity and usefulness. The old design showed rewards as a line graph (left image in red), which was hard to interpret and usually forced creators to click into each day for details. In contrast, the new design was shown as a bar graph (right image in blue), which made it easier to understand their rewards at a glance.
By simplifying the rewards graph, the new design enhances clarity and the overall Creator Rewards Program experience.
Old design (left image) compared to the new design (right image)
Key Recommendations
🎬 Focus on current and potential top tier creators: Top tier creators who are already producing high quality content are the demographic who will benefit the most from Additional Rewards. The team should prioritize increasing awareness and education among this group. Middle tier creators with potential to become top tier creators should also be considered a key audience for engagement. By increasing education of Additional Rewards, creators are more likely to produce high quality content, which can increase earnings and overall satisfaction.
ℹ️ Make information on Additional Rewards easier to access: For the full launch of Additional Rewards, consider surfacing information in the detail pages to be more front and center so creators can easily refer back to the eligibility criteria as they are ramping up to the new changes. This will make the Additional Rewards information intuitive and easy to find.
📊 Launch updated designs in full public launch: Move forward with launching new designs because the graphs are clear, easy to use, and visually appealing compared to the old designs.
Impact
impacted creator monetization strategy with shift to top tier creators
The product team refocused Additional Rewards to prioritize current and potential top-tier creators. This change was shared across important product roadmaps and strategy documents.
The Creator team met to realign their monetization strategy, ensuring opportunities were available for all creator tiers, now that Additional Rewards was focused primarily on top creators.
impacted product roadmap to increase awareness for top-mid tier creators
Product team launched in-app and email notifications to top and middle tier creators during the full launch of Additional Rewards.
Design team surfaced more information upfront to catch creators’ attention, increase education, and motivate them to click further in to explore the Additional Rewards detail pages.
moved forward with new designs for public launch of additional rewards
The team moved forward with the new designs after positive feedback on the clarity and usefulness of the graphs
