Reddit Subreddit Scraper
Pull recent posts from any subreddit (sorted new or top, paged by time) as clean JSON or CSV, without hitting Reddit's own listing cap on a single call.
What you're collecting
Monitoring or researching a subreddit means pulling a stream of posts, not one at a time, and usually more than a single page's worth. Reddit's own listings cap at roughly 1,000 items, so anything beyond that needs a way to page further back by time rather than relying on one call.
How UGC Scraper handles it
POST /v1/subreddit takes a subreddit name (or URL), a sort of new or top, and a limit of up to 100 posts per call. An after/before Unix-timestamp cursor lets you keep paging by time window, so you can build up a history well past a single listing's cap by making repeated calls.
cURL
curl -X POST https://api.ugcscraper.com/v1/subreddit \
-H "Authorization: Bearer rps_live_your_key" -H "Content-Type: application/json" \
-d '{"target":"webscraping","sort":"new","limit":100}'Python
import requests
r = requests.post(
"https://api.ugcscraper.com/v1/subreddit",
headers={"Authorization": "Bearer rps_live_your_key"},
json={"target": "webscraping", "sort": "new", "limit": 100},
)
listing = r.json()
for post in listing["items"]:
print(post["title"], post["score"])Node.js
const res = await fetch("https://api.ugcscraper.com/v1/subreddit", {
method: "POST",
headers: {
Authorization: "Bearer rps_live_your_key",
"Content-Type": "application/json",
},
body: JSON.stringify({ target: "webscraping", sort: "new", limit: 100 }),
});
const listing = await res.json();
for (const post of listing.items) {
console.log(post.title, post.score);
}Output structure
{
"kind": "subreddit",
"target": "webscraping",
"count": 25,
"items": [
{
"id": "abc123",
"title": "string",
"author": "string",
"subreddit": "webscraping",
"score": 84,
"num_comments": 12,
"permalink": "https://www.reddit.com/r/webscraping/comments/abc123/"
}
]
}This is a listing of post summaries, not full posts with comments; scrape an individual post's permalink to get its comment thread.
Practical use cases
- Community monitoring: track new posts in a subreddit your product, brand, or competitors get discussed in.
- Trend and topic research: pull top posts over a time range to see what a community engaged with most.
- Dataset building: page through a subreddit's full history by time window for training data or analysis.
Limitations
Scores and comment counts in a listing are a snapshot from when each post was captured, accurate for research and trend analysis but not a live-updating number. If you need a current score for a specific post, scrape that post directly or use /v1/refresh.