Case Study · Social Media Data Science · Market Intelligence

Market & Content Intelligence on YouTube

How the AI Agents & Automation niche shifted in a year — in format, engagement, comment integrity and vocabulary — measured on real YouTube data with age-normalized metrics and corrected statistics.

Topic: AI Agents & Automation
Windows: 2025 YTD vs 2026 YTD
Sample: 375 videos · 5,459 comments
Collected: Oct 2, 2026

Open Interactive Dashboard Methodology

Live sample · YouTube Data API v3375 videos · 5,459 organic commentscollected Oct 2, 2026

Executive Highlights

What a VP of Marketing needs to know

Three shifts, each with its effect size and its uncertainty attached.

Algorithmic format shift

Long-form rose from 44% to 67% of the videos YouTube surfaces for the topic; Shorts went from 33% to 12%

Short

32.6%

12.2%

Mid-length

23.5%

20.7%

Long-form

43.9%

67.0%

2025 YTD
2026 YTD

Median video length moved from 5.5 min to 13.4 min. Within 2026 YTD, longer videos also earn more reach relative to videos of the same age (Spearman ρ duration × relative velocity = 0.30, vs 0.04 in 2025 YTD).

Long-form share q < 0.001 · Shorts share q < 0.001 · median length q < 0.001 (Benjamini-Hochberg, 24 tests)

Reach and interaction decoupled

Engagement ratio +15% YoY, driven by Shorts (+70%) while long-form moved −11%

Short

1.9%

3.2%

Mid-length

2.1%

3.1%

Long-form

2.9%

2.6%

2025 YTD
2026 YTD

The link between length and engagement reversed: Spearman ρ duration × engagement went from +0.36 to −0.22. Comment rate moved +23% (q = 0.014). Long-form now wins reach; short videos win interaction per view.

Overall 95% CI [−1%, +32%] · q = 0.033 · Shorts n = 23 this year, q = 0.002 · long-form q = 0.526

Comment-section integrity

32% of 2025 YTD videos carried coordinated brand-promotion comments, vs 4% in 2026 YTD

31.6%

3.7%

2025 YTD
2026 YTD

199 comments promoting 5 products were removed before scoring sentiment. On the remaining organic comments, the negative share went from 8.6% to 9.0%: no significant change.

Videos with promo comments: two-proportion z-test q < 0.001 · organic negative share q = 0.859. Older videos had more time to collect spam.

Core Insights

What changed year over year

What YouTube surfaces for the topic got longer, interaction moved toward shorter formats, comment sections got cleaner, and the conversation moved from “n8n” to “claude”.

Short (<1 min)

33% → 12%

of surfaced videos · q < 0.001

Engagement 1.86% → 3.16% (+70%)

n = 60 / 23 · q = 0.002

Mid-length (1–10 min)

24% → 21%

of surfaced videos · q = 0.688

Engagement 2.14% → 3.09% (+44%)

n = 44 / 37 · q = 0.212

Long-form (>10 min)

44% → 67%

of surfaced videos · q < 0.001

Engagement 2.91% → 2.60% (−11%)

n = 82 / 110 · q = 0.526

Share of surfaced videos by subtopic

Two-proportion z-test, Benjamini-Hochberg corrected

Subtopic	2025 YTD	2026 YTD	Δ pp	Test
Build & Tutorials	44.9%	55.9%	+10.9	q = 0.082
Risks & Future of Work	3.7%	4.3%	+0.5	q = 0.903
Tools & Launches	11.2%	10.6%	−0.6	q = 0.903
Business & Monetization	14.4%	13.8%	−0.6	q = 0.903
Explainers & Concepts	15.0%	8.5%	−6.5	q = 0.113

No subtopic shift survives correction for multiple tests: the topic mix held steady.

Vocabulary shift

Log-odds ratio with an informative Dirichlet prior (z-score)

In titles

Rising

claude z +3.0
business z +2.5
claude code z +2.2
actually z +2.0
first ai z +2.0
youtube z +1.9

Fading

n8n z −3.5
aiagent z −1.7
agentic z −1.7
agentic ai z −1.7
automation agency z −1.7
ultimate z −1.5

In viewer comments

Rising

claude z +3.5
security z +2.9
skill z +2.9
claude code z +2.8
top z +2.4
skills z +2.3

Fading

n8n z −3.2
life z −3.1
ai automation z −2.9
easy understand z −2.9
automation z −2.8
easy z −2.8

In viewers' words

The most-liked organic comment with a clear tone, per year. Authors omitted; links, mentions and profanity filtered.

“Imagine standing in a future elevator and the elevator AI and your phone start cracking jokes about you like this.”

2025 YTD · positive tone 12.2KVADER +0.54

“Why do I suddenly see my grandson in the year 2100, lying old in bed, getting PTSD flashbacks when he hears that sound because he fought in the Great AI Robot War”

2025 YTD · negative tone 14.1KVADER −0.67

“This is great. I tell everyone, if I were 30, I would dive deep into AI, but then seeing videos like this I think, why would I not do it at 71?”

2026 YTD · positive tone 1.6KVADER +0.70

“AI will kill us all: by the way, buy my stock....”

2026 YTD · negative tone 874VADER −0.69

What moves with what

Spearman ρ per year. Relative velocity = views/day vs videos of the same year and age band.

Pair	2025 YTD	2026 YTD
Comment sentiment × engagement ratio	+0.21n=163	+0.00n=153
Comment sentiment × relative velocity	+0.03n=164	+0.25n=156
Duration × engagement ratio	+0.36n=186	−0.22n=170
Duration × relative velocity	+0.04n=187	+0.30n=176
Channel subscribers × relative velocity	+0.36n=187	+0.38n=176
Channel subscribers × engagement ratio	+0.30n=186	−0.20n=170
Like rate × comment rate	+0.52n=186	+0.54n=170

Strategic implications

Generated from the computed results, so they change if the data does.

For creators

Build for reach with long-form, for interaction with Shorts

Across both windows the 20–40m band earns the most reach relative to videos of the same age (median 1.82× the typical video), and in 2026 YTD the duration–reach correlation rose to 0.30. Shorts are now a minority of what surfaces but carry a higher engagement ratio (3.16% vs 2.60% for long-form, on only 23 Shorts): use them to start conversations, not to win discovery.

For creators

Follow the vocabulary: claude, business

Title terms gaining weight: “claude”, “business”, “claude code”. Fading: “n8n”, “aiagent”. “claude”, “claude code”, “n8n” move the same way in what viewers write in comments, which makes it a demand signal and not only a creator trend.

For brands

Audit comment sections before reading sentiment

32% of last year's videos in this niche carried coordinated brand-promotion comments. Positive-sounding comment sections can be bought: require organic-engagement reporting (with spam and creator comments removed) before using comment sentiment as a placement KPI.

For brands

Price reach on channel size, judge quality on engagement

Channel size is the most stable predictor of reach in both years (Spearman ρ subscribers × relative velocity = 0.36 → 0.38). Use it to set CPM expectations. The link between channel size and engagement flipped sign (ρ = +0.30 → −0.20): in the current window larger channels get less interaction per view. Compare creators on engagement only within the same subscriber tier.

For brands

Watch Tools & Launches as an early friction signal

Negative organic comments in Tools & Launches went from 9.4% to 15.2% (q = 0.082). Not significant after correcting for multiple tests: monitor it, but don't act on it yet.

Methodology Deep Dive

Normalizing for time before comparing

A video with a year on the platform and one with a month are not competing on the same terms. Every metric below exists to make a fair comparison possible — and the ones that cannot be made fair are not reported across years.

daily_velocity = views / days_since_published # videos < 14 days excluded

engagement_ratio = (likes + comment_count) / views × 100 # views ≥ 100

relative_velocity = velocity / median(velocity | same year, same 30-day age band)

sentiment_index = mean(VADER compound of organic comments) × 100 # ≥ 5 comments

YoY effect = Δ median · bootstrap 95% CI (2,000 resamples) · Mann-Whitney U · Benjamini-Hochberg q

Age confound

With one snapshot, period and video age are almost perfectly confounded: median age is 511 days for last year's videos vs 142 for this year's. Raw views move −56% YoY while views-per-day move +132% — opposite directions, and neither is a valid cross-year comparison. Year-over-year claims rest on ratios (engagement, comment tone); velocity is only compared within a year and age band.

Is engagement age-sensitive?

Tested, not assumed — and yes, this year: Spearman ρ between age and engagement is 0.25 (0.01 last year). Older videos engage more, so this year's younger sample is biased down: the YoY change is a conservative estimate. Restricting this year to videos ≥120 days old (n = 103) moves it from +15% to +25%.

A designed sample, not a census

search.list returns an algorithm-ranked selection. The sample takes 12 videos per month and query (“AI agents”, “AI automation”), so video counts measure the design, not market supply. Findings describe what the platform surfaces for the topic — which is what a viewer or a media buyer actually meets.

Format definitions

Formats follow the brief: Short < 1 min, mid-length 1–10 min, long-form > 10 min. Since October 2024 Shorts can run up to 3 minutes, so some Shorts land in “mid”; the duration chart keeps a separate 1–3 min band (28 → 9 videos) to make that visible.

Comment integrity

Before scoring sentiment the pipeline removed 156 comments by creators or containing links, 14 cross-video duplicates and 199 coordinated promotional comments for 5 products. Candidates come from an automated report (a brand named in comments on many videos that never mention it); the final list is confirmed by a person.

Language and NLP limits

29 non-English videos were excluded; of the remaining comments only English ones are scored (5,459), because VADER is an English lexicon. It handles negation, intensifiers, emoji and “but” contrasts, but not sarcasm. Topics come from a rule-based classifier designed on the sample's own titles.

Retention is not observable

Audience retention and shares exist only in the owner-authenticated YouTube Analytics API. This study measures engagement ratio, (likes + comments) / views, and never labels it retention. 19 videos with hidden like counts are excluded, not imputed as zero. With 24 tests, every p-value is corrected with Benjamini-Hochberg (q).

Data pipeline

// 375 videos · 5,459 comments scored · 4,006 API quota units

Ingestion
API v3 · month × query strata
search → videos → comments
Cleaning
language · spam · astroturfing
likes=None ≠ 0
Features
age-normalized metrics
velocity · engagement · RVI
NLP
sentiment · topics · terms
VADER + log-odds
Inference
non-parametric tests
bootstrap · MWU · BH-FDR
Export
typed JSON contracts
data/ · public/data/

Slice it yourself.

Filter by period, format and subtopic, and watch every chart recompute from the video-level data.

Open Interactive Dashboard