Gathered 2026-09-19. Answers the risk flagged in the pain-point research: a channel engine needs channel intelligence, and platforms have been closing access since 2023. Conclusion: NOT a blocker. It is a purchasing decision.
Why direct Reddit access failed on 2026-09-18
Not a sandbox problem. Reddit deprecated unauthenticated .json endpoints on
2026-05-28; by 2026-05-30 every free unauthenticated path returned 403. Driver
is commercial — Reddit data is now a licensed AI-training asset (Google and
OpenAI deals worth ~$130M in 2024).
Reddit's own tiers now:
- Free tier: 100 queries/min per OAuth client, non-commercial only (personal projects, bots, mod tools, academic). Self-service registration closed in late 2025; since the Responsible Builder Policy (2026-06-05) every client goes through a manual approval queue that is slow, opaque, and can be silently rejected.
- Commercial: requires written approval, ~$12,000/month minimum for ~50M calls, overage ~$0.24 per 1,000 calls.
Tier 1 — free and open (no key, no approval)
- Hacker News — Firebase API, no auth, no strict rate limits. Plus the Algolia HN search API, also free and keyless. Fully open.
- Bluesky / AT Protocol — no paid tier, no per-call fee, no app review, no developer portal. Public data needs no token. The most open social platform.
- Stack Exchange — free API tier.
- Pinterest — free at Trial and Standard tiers.
- The open web — Discourse forums, Substack, newsletters, blogs, docs, changelogs, communities. No gate at all.
Tier 2 — cheap, pay-per-use (dollars, not thousands)
- Reddit via Apify actors — no Reddit account or API key needed. Scrapes by
subreddit, search query, user history or post URL.
Arctic Shift-based ~$0.75/1k rows · OpenClawAI
$2/1k posts · official Apify actor $3.40/1k results ($1.15/1k posts on free tier). - Search APIs — Exa (semantic/neural) ~$5/1k queries · Tavily ~$0.008/credit · Serper (Google SERP, cheap) · Brave Search API (independent 30B-page index).
- Page fetchers — WebPeel ~$0.002/page · Jina Reader (cheap) · Firecrawl ~$0.016/page.
- SparkToro — purpose-built for "where does this audience actually hang out" (subreddits, podcasts, YouTube channels, forums). $50/mo, free tier of 5 searches. Both a validation of the CHANNELS stage and a possible input.
Tier 3 — gated or expensive (design around these)
- X/Twitter — flat-rate plans retired; pay-per-use only since Feb 2026. $0.005/read, $0.01/profile lookup, $0.20 to post with a URL.
- LinkedIn — partnership agreement required for anything beyond surface data. Effectively closed.
- Instagram — Basic Display API sunset 2024-12-04; personal-account access gone. Graph API Advanced Access needs Meta App Review + Business Verification.
- TikTok — Research API restricted to academic institutions.
Legal position (US)
- Scraping public data is generally lawful: hiQ v. LinkedIn (9th Cir.) held public data access is not "unauthorized" under the CFAA; Van Buren narrowed CFAA further. Meta v. Bright Data — Judge Chen granted summary judgment for Bright Data, holding Meta's ToS prohibits only logged-in scraping, not logged-off scraping of public content.
- But: hiQ still lost on breach of contract, because it had accepted LinkedIn's User Agreement. ToS breach is civil, not criminal — yet it is a real basis for suit and for being blocked.
- robots.txt is best practice, not statute. Bypassing a login or a technical block invites CFAA and contract claims. Personal data stays regulated even when public (GDPR/CCPA, EDPB Opinion 28/2024).
Operating rules for firstfoot: prefer official APIs where free; pay a vendor where not; never authenticate in order to scrape; never bypass a technical block; store channel-level facts, not personal data.
What this means
firstfoot does not need bulk social data. Per client it needs a few hundred data points: which 5-10 communities matter, their rules and posting bars, what performs there. At Apify rates that is single-digit dollars per client.
Architectural consequence: mirror the existing src/providers/ pattern for
data. A src/sources/ layer with the same three tiers the model layer already
has — free, paid, manual — so firstfoot runs with zero keys and gets better when
keys are added. Same thesis as "manual is a real executor, not a failure state."
Sources
- https://www.socialcrawl.dev/blog/reddit-data-api-2026
- https://crawlora.net/blog/reddit-json-api-blocked-2026
- https://prowlo.com/blog/reddit-data-api
- https://apify.com/themineworks/reddit-scraper
- https://use-apify.com/docs/best-apify-actors/best-reddit-scrapers
- https://docs.bsky.app/
- https://www.blotato.com/blog/bluesky-api-pricing
- https://thedatacollector.substack.com/p/how-to-scrape-hacker-news-in-2026
- https://www.algolia.com/developers/code-exchange/hacker-news
- https://www.socialcrawl.dev/blog/ultimate-guide-social-media-apis-2026
- https://www.getphyllo.com/post/instagram-api-pricing-explained-iv
- https://webpeel.dev/blog/benchmarks
- https://brave.com/learn/best-search-api-2026/
- https://blog.apify.com/hiq-v-linkedin/
- https://use-apify.com/docs/what-is-apify/is-apify-legal
- https://www.gwi.com/blog/audience-research-tools