Most SEO articles repeat information from other SEO articles. Public datasets offer another option: investigate a question, examine the data, and publish findings that readers cannot get from another generic guide.
You do not need access to a large company’s analytics account to start researching search behavior, website performance, technology adoption, or content trends. Several public datasets provide useful information, although they measure different things.
Some are free to explore directly. Others require SQL, a Google Cloud account, or data processing skills.
1. Search and demand datasets
1. Google Trends datasets on BigQuery
Free access options
Google provides aggregated and normalized search-trend datasets through BigQuery, including rising and popular queries across geographic areas. These can help researchers study demand patterns, emerging topics, and regional differences. The data is not a complete list of every Google search, and normalized interest should not be mistaken for absolute search volume.
Compare interest in topics over time and across locations. This is useful for studying seasonality, comparing product categories, and finding when interest begins to rise. The index is relative, so a score of 100 does not mean 100 searches.
Explore page-view patterns for Wikipedia articles. It can help identify public interest in topics, compare attention over time, and investigate whether a major event created a spike. Wikipedia pageviews are not Google search volumes, so use them as an additional signal rather than a substitute for keyword research.
2. Website performance and technology datasets
CrUX contains real-user experience data from eligible Chrome users. It helps researchers examine Core Web Vitals and compare field performance across origins or supported page-level records. It does not include every website or every visitor, and a lab test cannot be treated as equivalent to field data.
Use SQL to query CrUX data over time, build benchmarks, and investigate differences in real-world performance. This is more suitable for repeatable studies than manually testing a few URLs. BigQuery usage can incur costs beyond applicable free limits.
6. HTTP Archive
HTTP Archive periodically crawls websites and records information about page resources, markup, technologies, and performance. Use it to study trends such as image formats, script usage, page weight, and adoption of web technologies. Its sampling and crawl methodology matter when interpreting results.
httparchive.org
This annual report turns HTTP Archive data into research chapters with expert explanations. It is a good starting point if you want to understand web trends without writing SQL yourself. The 2025 methodology, for example, identifies the July 2025 crawl as the source for many of that edition’s metrics.
8. Chrome UX Report dashboards
Use the dashboard to visualize performance trends for a specific origin without building a full reporting pipeline. It is helpful for website benchmarking and explaining changes to non-technical stakeholders.
3. Large-scale web and content datasets
9. Common Crawl
Common Crawl provides large collections of crawled web pages and related metadata. Researchers can investigate web content, links, language patterns, and changes in the public web. It is technically demanding, and its crawl is not a complete or real-time copy of the internet.
Start here to learn how to access crawl indexes, files, and metadata. It is particularly useful when you need a repeatable method for analyzing a large set of pages rather than manually collecting individual URLs.
11. Google BigQuery public datasets
Browse datasets made available through Google Cloud, including resources relevant to search trends and web research. Check each dataset’s license, update frequency, schema, and usage cost before building a study around it. BigQuery’s public dataset program does not mean every query is unlimited or free.
12. Kaggle datasets
Search for datasets related to search queries, website performance, digital marketing, and web analytics. Quality varies widely. Check who collected the data, when it was collected, how fields are defined, and whether the dataset is synthetic before using it in an article.
13. GitHub
Researchers and developers publish datasets, SQL queries, analysis notebooks, and scripts on GitHub. It is useful for reproducing studies and checking methodology. A repository being public does not automatically make its data accurate, licensed for every use, or maintained.
4. Broader datasets for market and topic research
14. GDELT Project
GDELT monitors and organizes global news and media information. It can help study how topics appear in news coverage, how attention changes around events, and how stories spread across sources. It measures media activity, not direct Google ranking performance.
Use country-level economic, population, and internet-related indicators to add context to international digital-market research. For example, a study comparing search interest between countries may need population or internet-access context to avoid misleading conclusions.
How to turn public data into an SEO study
A useful study starts with a question, not a dataset.
For example, you could investigate whether websites with heavier JavaScript usage also tend to have different performance metrics. You could compare topic interest across countries or analyze how website technologies changed across several HTTP Archive crawls.
Follow this process:
- Write a specific research question. Avoid broad goals such as “study SEO trends.”
- Choose a dataset that actually measures the question. Search interest, web performance, organic clicks, and website traffic are different measures.
- Document the sample. Include dates, geographic scope, URL selection rules, exclusions, and data limitations.
- Analyze the data. Compare groups, look for changes over time, and investigate outliers.
- Separate correlation from causation. If faster websites have better outcomes in your sample, that alone does not prove speed caused the difference.
- Publish your methodology. Give readers enough detail to understand how you reached your conclusions and, where possible, reproduce the analysis.
The biggest opportunity is to publish small, transparent studies with useful findings, rather than inventing broad claims from a limited sample. A study of 200 carefully selected websites with a clear methodology can be more useful than a huge spreadsheet with no explanation.
I am an SEO specialist with more than 7 years of hands-on experience in SEO, organic content marketing, technical SEO, and social media. I have an engineering degree, giving me an unusual combination of technical understanding and digital marketing expertise. Know more about me.
