Customer Research

Customer Research

Guide for gathering and synthesizing real customer intelligence — from online communities, review sites, video comments, and social platforms — using the Hyper MCP scraper toolkit.

The goal is always the same: surface what customers actually say (in their own words), not what you assume they say.

Out of scope — defer to other skills

Requirements

• Hyper MCP installed. https://app.hyperfx.ai/mcp
• Apify scrapers toolkit enabled at https://app.hyperfx.ai/integrations — provides Reddit, Twitter, YouTube, TikTok, and Instagram scrapers.

Not all scrapers need to be active for every run — enable the ones relevant to your ICP (Reddit and one review site is the minimum). If a scraper tool is missing from the tool list, skip that source and continue with the others.

Tool surface

Critical rules

1. Always capture verbatim language. Don't paraphrase customer quotes — the exact words are what gets used in copy and messaging. Extract and preserve them. 2. Scrape before summarizing. Don't rely on your training data to describe what customers say about a product. Actually fetch the sources. 3. Label confidence on every insight. High = 3+ independent sources, unprompted. Medium = 2 sources or prompted only. Low = single source. Never present a Low-confidence finding as a conclusion. 4. Mind the bias of each source. Reddit skews technical and skeptical. Review sites skew toward power users and people with strong opinions. Support tickets skew toward problems. Factor this in before generalizing. 5. Don't invent persona details. If you don't have data for a persona field, leave it blank rather than filling it in with assumptions. 6. `youtube_transcript` is slow (~15–30s). It spins up an isolated sandbox. Only use it for videos where the language in the spoken content (not comments) is what matters.

---

Two modes

Most research combines both modes. Establish which applies before starting.

Mode 1 — Analyze existing assets

The user provides raw material: interview transcripts, survey responses, NPS verbatims, support tickets, win/loss notes. No tool calls needed — the job is extraction and synthesis.

Read references/synthesis-templates.md for the extraction framework, persona template, and VOC quote bank format. Then produce the requested deliverable.

Mode 2 — Go find research online

The user needs intel from online communities, review sites, and social platforms. This is where MCP tools do the heavy lifting.

See references/source-playbooks.md for per-source tool call examples and signal extraction tips.

---

Mode 2 workflow

Bias toward action. If the user's message includes a product name (or URL) and a recognizable goal (research competitors, build a persona, understand churn, find VOC language), skip the questions, state your plan in one sentence, and start Step 1. Only ask when something essential is genuinely missing — product identity or target segment, for example. Don't ask all five questions before doing anything.

Step 1 — Pick sources based on ICP type

Before calling anything, decide which sources are worth hitting for this specific audience:

Minimum viable run: Reddit + one review site. Add supplementary sources only when the minimum doesn't produce enough signal, or when the ICP table above calls for them.

For platform-by-platform tool call examples, read references/source-playbooks.md.

Step 2 — Run targeted scrapes

Pull from at least 2 sources. Single-source findings are low confidence by definition.

Reddit — the highest-signal source for most ICPs:

scrape_reddit(
    searches=["[product category] frustrations", "[competitor name] problems"],
    sort="top",
    time="year",
    max_items=50,
    skip_comments=False,
    search_posts=True,
    search_comments=True
)

For specific subreddits, pair with start_urls:

scrape_reddit(
    start_urls=["https://www.reddit.com/r/marketing/"],
    searches=["CRM"],
    sort="top",
    time="year",
    max_items=30
)

YouTube comments — rich qualitative data:

# Step 1: find the relevant videos
youtube_top_videos(query="[product category] honest review", max_results=5, sort_by="views")

# Step 2: mine comments from the top results
youtube_comments_search(
    start_urls=["https://www.youtube.com/watch?v=VIDEO_ID_1", "https://www.youtube.com/watch?v=VIDEO_ID_2"],
    max_comments=100,
    comments_sort_by="0"   # "0" = top comments, "1" = newest
)

X/Twitter — complaints, frustrations, and niche conversations:

search_tweets(
    search_terms='"[product name]" frustrating OR broken OR switched OR canceled',
    max_items=50,
    min_faves=5
)

Review sites (G2, Capterra, Trustpilot):

# G2 reviews for a specific product
web_scrape_page(
    url="https://www.g2.com/products/[product-slug]/reviews",
    ai_query="Extract the top complaints and pain points from customer reviews. Include verbatim quotes.",
    use_proxy=True
)

TikTok — consumer conversations and trending frustrations:

scrape_tiktok_videos(
    search_queries=["[product category] problems", "[competitor name] review"],
    results_per_page=30
)

Google discovery — find threads and communities you haven't thought of:

search_google_results(
    query='site:reddit.com "[product category]" "I switched" OR "I quit" OR "stopped using"',
    num_results=20
)

Step 3 — Extract signal from raw data

For each source, extract into this structure:

Step 4 — Synthesize across sources

After pulling from 3+ sources, synthesize into the research report format in references/synthesis-templates.md. The report includes:

• Top themes ranked by frequency × intensity
• VOC quote bank organized by theme
• Confidence labels on every finding
• Source bias notes

Step 5 — Build personas (optional)

Only build personas if you have ≥5 independent data points from a consistent segment. If not, say so and describe what additional research is needed first.

Persona template is in references/synthesis-templates.md.

---

Questions to ask before starting

Only ask what's genuinely missing. If the product and goal are clear, go. If not, lead with these — one or two at a time, not all at once:

1. What's the product? (if not obvious from context — a URL works) 2. What's the goal? Improve messaging? Build personas? Understand churn? Find product gaps? 3. Who is the target segment? (all customers, a specific tier, churned users, prospects who didn't convert) 4. What do you already have? (transcripts, surveys, tickets, nothing) 5. What deliverable do you need? (synthesis report, quote bank, persona, competitive language comparison)

---

Deliverables

Ask which one(s) the user needs before generating:

Source Playbooks

Per-source tool call patterns, search strategies, and signal extraction tips for customer research.

---

Reddit

Reddit is typically the highest-signal source for most B2B and B2C ICPs. People write long, unfiltered explanations of their problems, alternatives they considered, and why they switched away from products.

What to look for:

• "I've been using X for Y months and..." → reveals trigger, tenure, and accumulated friction
• "We just switched from X to Y because..." → reveals switching triggers and decision criteria
• "Does anyone else struggle with..." → reveals shared pain points
• Threads with 50+ upvotes where people are explaining their situation in detail

Search strategies:

# Broad pain/frustration sweep
scrape_reddit(
    searches=["[product category] frustrating", "[product category] problems", "[product name] vs"],
    sort="top",
    time="year",
    max_items=50,
    skip_comments=False,
    search_posts=True,
    search_comments=True
)

# Competitor switching conversations
scrape_reddit(
    searches=["switched from [competitor]", "moved away from [competitor]", "[competitor] alternative"],
    sort="top",
    time="year",
    max_items=30
)

# Subreddit-specific (when you know where your ICP hangs out)
scrape_reddit(
    start_urls=[
        "https://www.reddit.com/r/marketing/",
        "https://www.reddit.com/r/entrepreneur/"
    ],
    searches=["[product category]"],
    sort="top",
    time="year",
    max_items=40
)

Bias note: Reddit skews technical, skeptical, and vocal. Mainstream buyers who are satisfied don't usually post. Weight complaints appropriately.

Subreddits by ICP:

---

YouTube Comments

YouTube comments are rich with authentic customer language — especially on tutorial videos, comparison videos, and "honest review" videos. People ask questions, share frustrations, and describe their context in the comments.

What to look for:

• Questions that start with "How do I..." → reveals gaps in the product or onboarding
• Comments comparing alternatives: "I tried X and Y, ended up with Z because..."
• Frustrated reactions to features: "why can't it just..."
• "This changed everything for me" type comments → reveals the aha moment

Step 1 — Find relevant videos:

# Find high-view comparison and review videos
youtube_top_videos(
    query="[product name] review 2026",
    max_results=10,
    sort_by="views"
)

# Find tutorial videos (comments reveal confusion and gaps)
youtube_top_videos(
    query="[product category] tutorial for beginners",
    max_results=10,
    sort_by="views"
)

# Find competitor videos
youtube_top_videos(
    query="[competitor name] vs [product name]",
    max_results=5,
    sort_by="views"
)

Step 2 — Mine comments from the top 3–5 results:

youtube_comments_search(
    start_urls=[
        "https://www.youtube.com/watch?v=VIDEO_ID_1",
        "https://www.youtube.com/watch?v=VIDEO_ID_2",
        "https://www.youtube.com/watch?v=VIDEO_ID_3"
    ],
    max_comments=150,
    comments_sort_by="0"    # "0" = top (highest voted), "1" = newest
)

Optional — mine the transcript for language:

# Only when the spoken content (not comments) matters — e.g., a customer story video
youtube_transcript(video_id_or_url="https://www.youtube.com/watch?v=VIDEO_ID")

Note: youtube_transcript takes 15–30s. Use sparingly.

---

X / Twitter

Twitter is best for complaints (they're short and sharp), product comparisons, and finding people mid-decision. The advanced search syntax in search_terms is powerful.

What to look for:

• Complaints with engagement (faves ≥ 5 filters out noise)
• "Just switched from..." and "Can't believe X doesn't..." patterns
• Threads where people are asking for recommendations → reveals decision criteria

Search strategies:

# Frustrated customers
search_tweets(
    search_terms='"[product name]" (frustrating OR broken OR terrible OR "doesn\'t work" OR canceled)',
    max_items=50,
    min_faves=5
)

# Switching conversations
search_tweets(
    search_terms='"switched from [product name]" OR "moved from [product name]" OR "[product name] alternative"',
    max_items=40,
    min_faves=3
)

# Request threads — people mid-decision
search_tweets(
    search_terms='"looking for" "[product category]" (recommend OR suggestions OR alternatives)',
    max_items=30,
    min_replies=2
)

# Competitor comparisons
search_tweets(
    search_terms='"[competitor] vs [product]" OR "[product] vs [competitor]"',
    max_items=30,
    min_faves=3
)

Bias note: Twitter skews toward people with opinions strong enough to post publicly. Use to find emotional language and extreme positions — validate frequency against Reddit and review sites.

---

Review Sites (G2, Capterra, Trustpilot)

Review sites are goldmines for structured pain/benefit language. 1–3 star reviews reveal why customers churn. 4 star reviews often contain the most nuanced insight ("love it but wish it could..."). 5 star reviews reveal the aha moment in customers' own words.

G2:

# Product reviews
web_scrape_page(
    url="https://www.g2.com/products/[product-slug]/reviews",
    ai_query="Extract verbatim customer quotes about: (1) biggest pain points, (2) what they wish the product did differently, (3) what convinced them to buy. Include the star rating context.",
    use_proxy=True
)

# Competitor reviews (what do their customers complain about?)
web_scrape_page(
    url="https://www.g2.com/products/[competitor-slug]/reviews?filters%5Bnps_score%5D%5B%5D=3&filters%5Bnps_score%5D%5B%5D=2&filters%5Bnps_score%5D%5B%5D=1",
    ai_query="What are the most common complaints about this product? Extract verbatim quotes.",
    use_proxy=True
)

Capterra:

web_scrape_page(
    url="https://www.capterra.com/p/[id]/[product-name]/",
    ai_query="Extract the top pros and cons in customers' own words. Pull verbatim quotes from negative reviews.",
    use_proxy=True
)

Trustpilot (B2C products):

web_scrape_page(
    url="https://www.trustpilot.com/review/[domain.com]",
    ai_query="Extract the most common complaint themes from low-rated reviews. Include verbatim quotes.",
    use_proxy=True
)

App store reviews (mobile products):

# iOS
web_scrape_page(
    url="https://apps.apple.com/us/app/[app-name]/id[app-id]",
    ai_query="Extract the most common pain points from 1-3 star reviews. Include exact customer quotes.",
    use_proxy=True
)

Tip: If web_scrape_page returns JavaScript-blocked content, try firecrawl_scrape_url instead:

firecrawl_scrape_url(
    url="https://www.g2.com/products/[product-slug]/reviews",
    only_main_content=True
)

---

TikTok

Best for B2C and consumer-facing products. The comment sections on product review and comparison videos contain short but high-density emotional language.

# Search for product/category conversations
scrape_tiktok_videos(
    search_queries=["[product category] honest review", "[product name] worth it"],
    results_per_page=30
)

# Hashtag mining (when you know the community hashtag)
scrape_tiktok_videos(
    hashtags=["[producthashtag]", "[categoryhashtag]"],
    results_per_page=30
)

Note: scrape_tiktok_videos returns video metadata and engagement stats, not the comments themselves. Use the video titles and captions as research signal. For comment text, note the video URLs and scrape comments via the Hyper browser tools if needed.

Bias note: TikTok skews younger and consumer-oriented. Strong for CPG, lifestyle, and consumer SaaS. Less useful for enterprise B2B.

---

Google — Finding discussion threads

Google is useful for discovering discussion sources you haven't thought of, not for reading the content itself.

# Find Reddit threads about a specific pain
search_google_results(
    query='site:reddit.com "[product category]" "I switched" OR "I quit" OR "stopped using"',
    num_results=20
)

# Find competitor complaints across the web
search_google_results(
    query='"[competitor name]" problems OR complaints OR "doesn\'t work" -site:[competitor.com]',
    num_results=20,
    max_age_days=365
)

# Find community discussions (forums, Slack archives, Hacker News)
search_google_results(
    query='"[product category]" ("tell me" OR "recommend" OR "alternatives") site:news.ycombinator.com',
    num_results=10
)

# Find review roundups
search_google_results(
    query='"best [product category]" OR "[product category] alternatives" 2026',
    num_results=15,
    max_age_days=180
)

Then use firecrawl_scrape_url or web_scrape_page to read the most relevant URLs.

---

Instagram

Best for pulling content from brand or community accounts — useful when researching how a competitor presents themselves and what language they use in captions.

# Competitor brand posts
scrape_instagram_posts(
    usernames=["competitor_handle"],
    results_limit=30,
    data_detail_level="detailedData"
)

# Community / niche accounts
scrape_instagram_posts(
    usernames=["niche_community_account"],
    results_limit=20
)

Note: scrape_instagram_posts returns post captions, engagement, and metadata — not comment text. It's most useful for researching competitor messaging and content angles, not direct customer voice.

---

What to do when a source fails

Some scraper tools are Apify-backed and occasionally return fetch failed or timeout:

1. Retry once after a short pause. 2. If it fails again, try an alternative tool (e.g., firecrawl_scrape_url instead of web_scrape_page). 3. If still failing, note the source as unavailable and continue with remaining sources. Don't block the entire research on one failed scrape. 4. Never invent data to fill a gap — mark it as "source not available."

Synthesis Templates

Output formats for customer research deliverables.

---

Extraction framework (for existing assets — Mode 1)

For each asset (transcript, survey, ticket batch, NPS verbatims), extract across five dimensions:

1. Jobs to Be Done

• Functional job: the task they're trying to complete
• Emotional job: how they want to feel while doing it
• Social job: how they want to be perceived

2. Pain Points

• Prioritize pains mentioned unprompted and with emotional language
• "Our spreadsheets are a disaster" > "we have process challenges"

3. Trigger Events

• What changed that made them start looking for a solution?
• Common triggers: team growth, new hire, missed deadline, embarrassing incident, competitor move, board pressure

4. Desired Outcomes

• What does success look like in their words?
• How do they measure it? What does it do for their reputation internally?

5. Alternatives Considered

• Competitor, DIY, do nothing, hire someone, build internally
• What almost made them choose an alternative instead?

---

Confidence labeling

Label every insight before presenting it:

Never present a Low-confidence finding as a conclusion. Present it as a hypothesis.

---

Research synthesis report

Use this format after gathering from 3+ sources.

## Research Synthesis: [Product / Category] — [Date]

**Sources used:** [list: Reddit r/X (N posts), G2 reviews (N), YouTube comments (N videos), X/Twitter (N tweets), etc.]
**Segment studied:** [who this research covers]
**Confidence ceiling:** [max confidence level possible given sample size]

---

### Theme 1: [Name — make it descriptive, e.g. "Setup takes too long and nobody documents it"]

**Frequency:** Appeared in X of Y sources
**Intensity:** High / Medium / Low (based on emotional language, unprompted mentions)
**Confidence:** High / Medium / Low

**Representative quotes:**
- "[exact verbatim quote]" — [Source: Reddit r/marketing, 2026-04-15, 234 upvotes]
- "[exact verbatim quote]" — [Source: G2 review, 4 stars, 2026-02]
- "[exact verbatim quote]" — [Source: YouTube comment on "X vs Y" video]

**What this means:**
[1–2 sentences on implication for messaging / product / positioning]

---

### Theme 2: [Name]
[same structure]

---

### Competitive language (what customers say about alternatives)

| Competitor / Alternative | What customers say | Source |
| --- | --- | --- |
| [Name] | "[verbatim]" | [Source] |

---

### Source bias notes

[Explain any skews: "Reddit responses are heavier on technical users, may not represent mainstream buyer. Review site sample skews toward English-speaking US market."]

---

### Research gaps

What we still don't know and what would fill it:
- [Gap 1] → would validate with: [method]
- [Gap 2] → would validate with: [method]

---

VOC quote bank

Use this when the deliverable is customer language for copy, messaging, or positioning projects.

## VOC Quote Bank: [Product / Category] — [Date]

### Pain language

| Quote | Source | Sentiment | Confidence |
| --- | --- | --- | --- |
| "[exact quote]" | Reddit r/X, 2026-04 | Frustrated | High |
| "[exact quote]" | G2 review, 3 stars | Negative | Medium |

### Outcome language (what success sounds like)

| Quote | Source | Confidence |
| --- | --- | --- |
| "[exact quote]" | Interview transcript | High |
| "[exact quote]" | YouTube comment | Medium |

### Trigger language (what made them start looking)

| Quote | Source | Confidence |
| --- | --- | --- |
| "[exact quote]" | Twitter, 47 likes | Medium |

### Alternative language (how they describe competitors / workarounds)

| Quote | Competitor referenced | Source |
| --- | --- | --- |
| "[exact quote]" | [Name] | Reddit |

### Objection language (hesitations and fears)

| Quote | Source | Confidence |
| --- | --- | --- |
| "[exact quote]" | G2 review, 4 stars | Medium |

---

Persona template

Only build a persona once you have ≥5 independent data points from a consistent segment. If you don't, say what additional research is needed.

## [Persona Name] — [Role / Title Range]

**Profile**
- Title range: [e.g., "Marketing Manager to VP of Marketing"]
- Company size: [e.g., "50–500 employees, Series A–C"]
- Industry: [if narrow]
- Reports to: [who]
- Team size: [if relevant]

**Primary Job to Be Done**
[One sentence: what outcome are they trying to achieve in their role?]
Source: [cite the data point(s)]

**Trigger Events**
What causes them to start looking for a solution?
- [Trigger 1] — Source: [cite]
- [Trigger 2] — Source: [cite]

**Top Pains** (in their words)
1. "[verbatim or close paraphrase]" — Source: [cite]
2. "[verbatim or close paraphrase]" — Source: [cite]
3. "[verbatim or close paraphrase]" — Source: [cite]

**Desired Outcomes**
- [What success looks like to them — source: cite]
- [How they measure it — source: cite]
- [How it makes them look internally — source: cite if available]

**Alternatives They Consider**
- [Competitor / DIY / do nothing] — Source: [cite]

**Objections and Fears**
- [What makes them hesitate] — Source: [cite]

**Key Vocabulary**
Exact phrases they use (sourced from research — do not invent):
- "[phrase]" — [source]
- "[phrase]" — [source]

**How to Reach Them**
- Platforms: [where they spend time — cite if from research, note if assumed]
- Content they consume: [formats, topics]

**Data confidence:** [High / Medium / Low — and how many data points this persona is built from]
**Gaps:** [What fields are unfilled because we don't have data yet]

Persona anti-patterns:

• Don't name them cutely ("Marketing Mary") unless your team finds it helpful
• Don't average across different segments — build separate personas
• Don't invent details — blank is better than fabricated
• Revisit quarterly — personas decay as your market evolves

---

Jobs-to-be-done map

Use when the deliverable is a structured JTBD framework for product or positioning work.

## Jobs-to-be-Done Map: [Product / Segment] — [Date]

### Job 1: [Functional job name]

**Functional job:** [What they're trying to do]
**Emotional job:** [How they want to feel while doing it]
**Social job:** [How they want to be perceived]

**Current solution:** [How they do it today]
**Friction with current solution:** [What's broken or inadequate]
**Desired outcome:** [What success looks like]

**Trigger:** [What causes them to seek a better solution]

**Representative quotes:**
- "[quote]" — [source]

---

### Job 2: [Name]
[same structure]

---

Research gap analysis

Use when the user has partial data and needs to know what's missing before acting.

## Research Gap Analysis: [Product] — [Date]

### What we know (and confidence level)

| Finding | Confidence | Sources |
| --- | --- | --- |
| [Finding] | High | Reddit (N), G2 (N) |
| [Finding] | Medium | Single interview |

### What we don't know

| Gap | Why it matters | How to fill it |
| --- | --- | --- |
| Why do enterprise buyers churn at month 6? | Can't fix without knowing root cause | 5 exit interviews; mine support tickets for months 5–7 |
| What does success look like for the marketing segment? | Current personas conflate marketing and ops buyers | 3–5 interviews with marketing-specific buyers |
| How do customers describe us vs. [competitor]? | Can't sharpen positioning without knowing current perception | G2 comparison reviews; competitor-switching Reddit threads |

### Recommended next steps (prioritized)

1. [Highest-value gap to fill, method, estimated effort]
2. [Second priority]
3. [Third priority]