Introduction
This page will help you get started with our Scraping APIs.
Important service update
Enjoy reliable data by paying only for successful requests.
With these Scraping APIs, you can finally gather scraping results without any Captchas or IP address blocks. That's because instead of setting up IP addresses, here you only need to select your target and send your query to our endpoint https://scraper-api.smartproxy.com/v2/scrape. Therefore, things like proxy management, retries, result localization, and many other usual setbacks will be all taken care of to return 100% correct results.
Simplified path your request takes when targeting Google
You can adjust each search request according to certain parameter values and get the results in either HTML or JSON format, if we are able to parse requested website in JSON.
Performing a search is as simple as making a GET or POST HTTPS request to the Scraping API queries endpoint. To make a query, provide your authentication information (user:pass) and the search query. Some integration options will also require specification about your application/JSON content.
Now you can: Retrieve paid and organic data, Monitor SEO, Monitor prices, and more.
What you can scrape with our Scraping APIs: Google Search – retrieve paid and organic SERP data; Google Shopping; Popular products and listings; Videos; Ads; Images; Related questions and searches; Top stories; Featured snippets; Local pack; Hotel availability; Travel industry; Restaurant data; Amazon Search; Baidu; Wayfair; Yandex; Bing; Anything else.
Screenshot of a couple first results and full response received from Google Search “world” query:
Google Search example
{
"pos": 1,
"url": "https://www.worldbank.org/en/home",
"desc": "With 189 member countries, the World Bank Group is a unique global partnership fighting poverty worldwide through sustainable solutions.World Bank Live · Data · Careers · Where we work",
"title": "World Bank Group - International Development, Poverty ...",
"url_shown": "https://www.worldbank.org› ...",
"pos_overall": 1
},
{
"pos": 2,
"url": "https://www.who.int/",
"desc": "WHO's primary role is to direct international health within the United Nations' system and to lead partners in global health responses.World Health Assembly · WHO Coronavirus (COVID-19... · COVID-19 vaccines",
"title": "WHO | World Health Organization",
"url_shown": "https://www.who.int",
"pos_overall": 2
},
{
"results": [
{
"content": {
"url": "https://www.google.com/search?q=world&lr=lang_en&nfpr=1&hl=en",
"page": 1,
"results": {
"paid": [],
"organic": [
{
"pos": 1,
"url": "https://www.worldbank.org/en/home",
"desc": "With 189 member countries, the World Bank Group is a unique global partnership fighting poverty worldwide through sustainable solutions.World Bank Live · Data · Careers · Where we work",
"title": "World Bank Group - International Development, Poverty ...",
"url_shown": "https://www.worldbank.org› ...",
"pos_overall": 1
},
{
"pos": 2,
"url": "https://www.who.int/",
"desc": "WHO's primary role is to direct international health within the United Nations' system and to lead partners in global health responses.World Health Assembly · WHO Coronavirus (COVID-19... · COVID-19 vaccines",
"title": "WHO | World Health Organization",
"url_shown": "https://www.who.int",
"pos_overall": 2
},
{
"pos": 3,
"url": "https://en.wikipedia.org/wiki/World",
"desc": "In this sense, world history refers to the history of humanity as a whole or world politics is the discipline of political science studying issues that ...",
"title": "World - Wikipedia",
"url_shown": "https://en.wikipedia.org› wiki › World",
"pos_overall": 3
},
{
"pos": 4,
"url": "https://www.weforum.org/",
"desc": "The World Economic Forum is an independent international organization committed to improving the state of the world by engaging business, political, ...",
"title": "The World Economic Forum",
"url_shown": "https://www.weforum.org",
"pos_overall": 4
},
{
"pos": 5,
"url": "https://www.worldometers.info/coronavirus/",
"desc": "Country, OtherTotal; CasesNew; CasesTotal; DeathsNew; DeathsTotal; RecoveredNe...World259,773,111+81,4535,193,091+1,229234,916,758+9...Asia81,625,181+16,9781,205,680+18178,895,360+2...Europe71,598,510+58,2971,393,285+69963,443,906+6...View 237 more rows",
"title": "Coronavirus Cases - Worldometer",
"url_shown": "https://www.worldometers.info› coronavirus",
"pos_overall": 5
},
{
"pos": 6,
"url": "https://www.worldvision.org.uk/",
"desc": "Donate to World Vision today to support vulnerable children in fragile and developing countries. Find out more about how you can help these causes today.",
"title": "World Vision UK: International Children's Charity",
"url_shown": "https://www.worldvision.org.uk",
"pos_overall": 6
},
{
"pos": 7,
"url": "https://ourworldindata.org/covid-vaccinations",
"desc": "The Our World in Data COVID vaccination data. To bring this pandemic to an end, a large share of the world needs to be immune to the virus. The safest way to ...",
"title": "Coronavirus (COVID-19) Vaccinations - Statistics and Research",
"url_shown": "https://ourworldindata.org› covid-vaccinations",
"pos_overall": 7
},
{
"pos": 8,
"url": "https://www.bbc.co.uk/news/world",
"desc": "Get the latest BBC World News: international news, features and analysis from Africa, the Asia-Pacific, Europe, Latin America, the Middle East, South Asia, ...",
"title": "World - BBC News",
"url_shown": "https://www.bbc.co.uk› news › world",
"pos_overall": 8
},
{
"pos": 9,
"url": "https://worldhappiness.report/",
"desc": "The World Happiness Report is a publication of the Sustainable Development Solutions Network, powered by data from the Gallup World Poll and Lloyd's ...",
"title": "Home | The World Happiness Report",
"url_shown": "https://worldhappiness.report",
"pos_overall": 9
}
],
"knowledge": {
"title": "World",
"description": "In its most general sense, the term \"world\" refers to the totality of entities, to the whole of reality or to everything that is. The nature of the world has been conceptualized differently in different fields. Some conceptions see the world as unique while others talk of a \"plurality of worlds\"."
},
"search_information": {
"query": "world",
"showing_results_for": "world",
"total_results_count": 11900000000
},
"total_results_count": 11900000000
},
"last_visible_page": 10,
"parse_status_code": 12000
},
"url": "https://www.google.com/search?q=world&lr=lang_en&nfpr=1&hl=en",
"task_id": "6869593036469325825",
"created_at": "2021-11-25 11:10:10",
"updated_at": "2021-11-25 11:10:18"
}
]
}
Try it out yourself in our API if you have an active SERP subscription!
Don't be a stranger – get in touch with us for more details and code examples.
Updated 3 months ago