Node Unblocker for Web Scraping: Setup, Middleware & Proxy Integration (2026)

Node Unblocker is an open-source Node.js proxy middleware that sits between your scraper and target websites — forwarding HTTP requests, rewriting URLs and headers, and masking your real IP. It was originally built for bypassing internet censorship, but its Express-compatible API and streaming architecture make it a useful tool in lightweight Node.js scraping stacks. This guide covers setup, middleware customisation, IP rotation, Puppeteer integration, and where managed residential proxies outperform a self-hosted approach.

What Is Node Unblocker?

Node Unblocker is an open-source npm package (package name: unblocker) that provides proxy middleware for Node.js and Express applications. Originally created for bypassing internet censorship, it functions as a programmable HTTP/HTTPS proxy server — receiving requests from your scraper, fetching the target page on your behalf, rewriting all internal URLs, and streaming the response back.^[2]

The key distinction from a standard HTTP proxy is its REST API model: instead of configuring proxy settings in your scraper's network client, you route requests through a URL pattern like http://your-proxy-server/proxy/https://target.com. This makes it integrable with any HTTP client library — fetch, axios, or a headless browser — without proxy protocol configuration.

Node Unblocker processes data as a stream rather than buffering full responses, which keeps memory usage low and latency minimal even on large pages. It automatically handles URL rewriting in HTML, CSS, and JavaScript so that relative links on the proxied page continue working correctly through the proxy.

How Node Unblocker Works

The architecture is a standard forward proxy with Express middleware hooks layered on top:

1. Your scraper sends a request to http://proxy-server/proxy/{target-url}.
2. Node Unblocker strips the /proxy/ prefix, extracts the target URL, and opens a new outbound HTTP request to that URL using the proxy server's IP.
3. The response streams back through Node Unblocker. Before delivery, it rewrites all URLs in the HTML/CSS/JS to maintain the /proxy/ prefix — so all subsequent resource requests (images, scripts, AJAX) also route through the proxy.
4. Middleware hooks fire on both the request (before forwarding) and the response (before delivering to the scraper), allowing header injection, cookie manipulation, or rate-limit logic.

The target website sees the proxy server's IP address, not your real one. Because Node Unblocker runs on a server you control, you decide what headers to send, what cookies to manipulate, and how to handle redirects.^[3]

Basic Setup: Node Unblocker in 5 Minutes

mkdir scraper-proxy && cd scraper-proxy
npm init -y
npm install express unblocker

// index.js — basic Node Unblocker proxy server
const express  = require('express');
const Unblocker = require('unblocker');

const app = express();

// Mount unblocker middleware — all /proxy/* requests go through it
const unblocker = new Unblocker({ prefix: '/proxy/' });
app.use(unblocker);

const PORT = process.env.PORT || 8080;
app.listen(PORT, () => {
  console.log(`Proxy running on http://localhost:${PORT}`);
  console.log(`Test: http://localhost:${PORT}/proxy/https://httpbin.org/ip`);
});

// Required for WebSocket support in some environments
app.on('upgrade', unblocker.onUpgrade);

node index.js

# In a second terminal — should return the server's IP, not yours
curl http://localhost:8080/proxy/https://httpbin.org/ip

The proxy is now running. Any URL prefixed with /proxy/ will be fetched through the server's IP. If the server is a remote VPS or cloud instance, your real IP is hidden from all target websites.

Using Node Unblocker for Web Scraping

With the proxy running, integrate it into any Node.js scraping workflow using fetch or axios:

// scraper.js — fetch through Node Unblocker proxy
const axios = require('axios');

const PROXY_BASE = 'http://localhost:8080/proxy';
const TARGET     = 'https://books.toscrape.com';

async function scrape(url) {
  const proxiedUrl = `${PROXY_BASE}/${url}`;
  const response   = await axios.get(proxiedUrl);
  return response.data;
}

// Fetch book listing through proxy — target sees proxy IP
scrape(TARGET).then(html => {
  console.log('Page length:', html.length);
  // Pass to Cheerio, JSDOM, or any HTML parser
});

All resource requests embedded in the returned HTML (stylesheets, scripts, images) will also use the proxy URL prefix automatically — Node Unblocker's URL rewriting handles this without any additional configuration.

Middleware: Customising Requests and Responses

Node Unblocker's most powerful feature for scraping is its middleware system. Middleware functions fire on every request (before forwarding to the target) and every response (before delivering to the scraper), giving you full control over headers, cookies, and content.^[4]

const Unblocker = require('unblocker');

const unblocker = new Unblocker({
  prefix: '/proxy/',

  // REQUEST MIDDLEWARE — fires before forwarding to target
  requestMiddleware: [
    function setHeaders(data) {
      // Inject realistic browser headers
      data.clientRequest.headers['user-agent'] =
        'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 Chrome/124.0 Safari/537.36';
      data.clientRequest.headers['accept-language'] = 'en-US,en;q=0.9';
      data.clientRequest.headers['accept'] =
        'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8';
      // Remove headers that reveal proxy usage
      delete data.clientRequest.headers['x-forwarded-for'];
      delete data.clientRequest.headers['via'];
    }
  ],

  // RESPONSE MIDDLEWARE — fires before returning to scraper
  responseMiddleware: [
    function logStatus(data) {
      console.log(
        `[${new Date().toISOString()}]`,
        data.clientResponse.statusCode,
        data.remoteRequest.href
      );
    }
  ]
});

Common middleware use cases for scraping:

• User-Agent rotation — rotate through a list of realistic browser UA strings per request to avoid UA-based fingerprinting.
• Referrer injection — set the Referer header to simulate organic navigation from a search engine.
• Cookie management — persist session cookies across requests to maintain login state.
• Rate limiting — add request throttling logic to avoid triggering rate limit thresholds on target servers.
• X-Forwarded-For stripping — remove headers that reveal the original client IP or proxy usage.

IP Rotation Strategy

A single Node Unblocker instance has one IP — the server it runs on. This IP will be flagged and blocked by any serious target after a moderate number of requests. The standard solution is to deploy multiple Node Unblocker instances across different cloud servers and implement a rotation pool in your scraper:^[3]

// Proxy pool rotation — multiple Node Unblocker instances
const proxyPool = [
  'http://server1.example.com:8080/proxy',
  'http://server2.example.com:8080/proxy',
  'http://server3.example.com:8080/proxy',
];

function getRandomProxy() {
  return proxyPool[Math.floor(Math.random() * proxyPool.length)];
}

async function scrapeWithRotation(targetUrl) {
  const proxy = getRandomProxy();
  const proxiedUrl = `${proxy}/${targetUrl}`;
  try {
    const res = await axios.get(proxiedUrl, { timeout: 15000 });
    return res.data;
  } catch (err) {
    // Remove failed proxy from pool or log for review
    console.error(`Proxy failed: ${proxy}`, err.message);
    throw err;
  }
}

Each Node Unblocker instance deployed on a separate VPS or cloud instance (DigitalOcean droplets from $4/month, Railway from $5/month) gives you a different datacenter IP. This works for targets without aggressive anti-bot systems but reaches its limits quickly on protected sites — datacenter IPs from major cloud providers are flagged by ASN at the IP reputation layer before a single request is processed.

Puppeteer Integration

For JavaScript-heavy targets where Node Unblocker alone is insufficient (SPAs, AJAX-loaded content), combine it with Puppeteer by routing browser traffic through the proxy:

const puppeteer = require('puppeteer');

async function scrapeWithPuppeteer(targetUrl) {
  const browser = await puppeteer.launch({
    // Route all browser traffic through Node Unblocker
    args: ['--proxy-server=http://localhost:8080'],
    headless: 'new'
  });

  const page = await browser.newPage();

  // Set realistic viewport and user-agent
  await page.setViewport({ width: 1366, height: 768 });
  await page.setUserAgent(
    'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 Chrome/124.0 Safari/537.36'
  );

  // Navigate via the proxy prefix URL
  await page.goto(`http://localhost:8080/proxy/${targetUrl}`, {
    waitUntil: 'networkidle2',
    timeout: 30000
  });

  const html = await page.content();
  await browser.close();
  return html;
}

Pairing Node Unblocker with Residential Proxies

The single most effective upgrade to any Node Unblocker setup is placing a commercial residential proxy behind it — routing Node Unblocker's outbound requests through a residential IP pool instead of the server's datacenter IP. This combines Node Unblocker's URL rewriting and middleware capabilities with the trust level of a genuine consumer ISP connection.

Configure the outbound proxy in Node Unblocker using the httpsAgent option with a Nstproxy residential proxy:

const express      = require('express');
const Unblocker    = require('unblocker');
const { HttpsProxyAgent } = require('https-proxy-agent');

const app = express();

// Nstproxy residential proxy credentials
const NSTPROXY_HOST = 'gate.nstproxy.io';
const NSTPROXY_PORT = '8080';
const NSTPROXY_USER = 'your_username';
const NSTPROXY_PASS = 'your_password';

const proxyUrl   = `http://${NSTPROXY_USER}:${NSTPROXY_PASS}@${NSTPROXY_HOST}:${NSTPROXY_PORT}`;
const proxyAgent = new HttpsProxyAgent(proxyUrl);

const unblocker = new Unblocker({
  prefix: '/proxy/',
  // Route all outbound requests through residential proxy
  httpsAgent: proxyAgent,
  httpAgent:  proxyAgent,
  requestMiddleware: [
    function removeProxyHeaders(data) {
      delete data.clientRequest.headers['x-forwarded-for'];
      delete data.clientRequest.headers['via'];
    }
  ]
});

app.use(unblocker);
app.listen(8080, () => console.log('Proxy with residential IPs running on :8080'));

With this setup, every request forwarded by Node Unblocker exits through a clean Nstproxy residential IP — making it appear as genuine household traffic to the target website while retaining all of Node Unblocker's URL rewriting and middleware capabilities.

Node Unblocker vs Managed Proxy Solutions

Comparison based on ScrapingBee's Node Unblocker analysis and IPRoyal's 2026 Node Unblocker tutorial.

Node Unblocker Limitations in 2026

Pros & Cons Snapshot

FAQ

Further Reading