Forcing headers
Site Unblocker uses predefined header and cookies combinations to achieve the most efficient success rates for targeted websites.
If you require to send your own headers and/or cookies, specific headers need to be added to your request:
Header | Valid values | Description |
---|---|---|
X-Smartproxy-Force-User-Cookies | 1 | User-provided cookies can sometimes degrade the performance of the scraping target. User-provided cookies are dropped by default, however you can override this behaviour with this header. |
X-Smartproxy-Force-User-Headers | 1 | User-provided headers can sometimes degrade the performance of the scraping target. User-provided headers are dropped by default, however you can override this behaviour with this header. Note that we will always reject certain headers |
If these headers are not present in your request, any additional header and/or cookies will be ignored.
Headers
To send additional headers with your request, useX-SU-Custom-*
prefix before adding the custom header. It can be named as per your requirement, e.g - X-SU-Custom-User-Agent
, X-SU-Custom-My-Header
curl -k -v -x unblock.smartproxy.com:60000 \
-U "USERNAME:PASSWORD" "https://ip.smartproxy.com/" \
-H "X-Smartproxy-Force-User-Headers: 1" \
-H "X-SU-Custom-User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.79 Safari/537.36"
-H "X-SU-Custom-My-Header: Custom header content here" \
import requests
proxies = {
'http': 'http://USERNAME:[email protected]:60000',
'https': 'http://USERNAME:[email protected]:60000'
}
headers= {
'X-SU-Custom-My-Header': 'Custom header content here',
'X-Smartproxy-Force-User-Headers: 1',
'User-Agent':'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.79 Safari/537.36'
}
response = requests.request(
'GET',
'https://ip.smartproxy.com/',
verify=False,
proxies=proxies,
headers=headers
)
print(response.text)
import fetch from 'node-fetch';
import createHttpsProxyAgent from 'https-proxy-agent'
const username = 'YOUR_USERNAME';
const password = 'YOUR_PASSWORD';
const agent = createHttpsProxyAgent(
`http://${username}:${password}@unblock.smartproxy.com:60000`
);
process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = 0;
const headers = {
'X-SU-Custom-My-Header': 'Custom header content here',
'X-Smartproxy-Force-User-Headers: 1',
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.79 Safari/537.36'
}
const response = await fetch('https://ip.smartproxy.com/', {
method: 'get',
headers: headers,
agent: agent
});
console.log(await response.text());
Cookies
To add your custom cookies, you do not require to use the prefix.
curl -k -v -x unblock.smartproxy.com:60000 \
-U "USERNAME:PASSWORD" "https://ip.smartproxy.com/" \
-H "X-Smartproxy-Force-User-Cookies: 1" \
-H "Cookie: WOW=4206969; NICE=8008135"
import requests
proxies = {
'http': 'http://USERNAME:[email protected]:60000',
'https': 'http://USERNAME:[email protected]:60000'
}
headers= {
'Cookie': 'WOW=4206969; NICE=8008135',
'X-Smartproxy-Force-User-Cookies: 1',
}
response = requests.request(
'GET',
'https://ip.smartproxy.com/',
verify=False,
proxies=proxies,
headers=headers
)
print(response.text)
import fetch from 'node-fetch';
import createHttpsProxyAgent from 'https-proxy-agent'
const username = 'YOUR_USERNAME';
const password = 'YOUR_PASSWORD';
const agent = createHttpsProxyAgent(
`http://${username}:${password}@unblock.smartproxy.com:60000`
);
process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = 0;
const headers = {
'Cookie': 'WOW=4206969; NICE=8008135',
'X-Smartproxy-Force-User-Cookies: 1',
}
const response = await fetch('https://ip.smartproxy.com/', {
method: 'get',
headers: headers,
agent: agent
});
console.log(await response.text());
Updated 6 days ago