How to Scrape Websites with JavaScript in 2026 | Nstproxy Way
TL;DR
Use native fetch() plus Cheerio when the response already contains the data; a browser is unnecessary overhead for static HTML.
Use Playwright when the required values appear only after JavaScript runs, but wait for a meaningful page condition instead of an arbitrary delay.
Production scrapers need explicit timeouts, status checks, schema validation, bounded retries, rate limits, and stable identifiers—not only selectors that work once.
Nstproxy Crawl fits the point where rendering, retries, site discovery, and output conversion become infrastructure work rather than application logic.
Scrape only public or authorized pages, honor applicable rules and terms, and minimize retained data.
JavaScript web scraping starts with the response, not a browser
JavaScript web scraping is the process of requesting a permitted web resource, extracting the fields you need, and returning a stable record for downstream use. The cheapest reliable method is determined by where the data exists: in the initial HTML, in an embedded JSON payload, behind a documented API, or only in a rendered browser state.
That decision matters more than library popularity. A static page can often be handled with Node.js fetch() and Cheerio. A client-rendered catalog may require Playwright. A broad or repeated workflow may be better served by a managed collection layer such as Nstproxy Crawl, while your JavaScript code keeps ownership of validation and business rules.
The MDN Fetch guide notes an important failure mode: fetch() does not reject merely because the server returns an HTTP error. Your code must check response.ok or the status explicitly before parsing a body. This small detail separates a valid record from a branded 404 page accidentally stored as product data.
The right method is the least complex option that consistently returns complete, valid data.
Page behavior
First choice
Upgrade when
Complete HTML in the response
fetch() + Cheerio
Required fields are absent or markup changes frequently
Structured JSON endpoint
Direct JSON request
The endpoint is undocumented, unstable, or access is not authorized
Content appears after scripts run
Playwright
Browser operations, queues, retries, or artifacts dominate maintenance
Many pages or bounded site discovery
Managed crawl API
You need custom domain validation beyond generic extraction
Cheerio loads and queries HTML without executing page JavaScript. Its official document-loading guide and selector guide make it a good fit for server-rendered pages. Playwright controls a browser page, and its locator documentation recommends user-facing attributes and explicit contracts over brittle CSS paths.
The distinction is practical: do not launch Chromium to parse a title already present in the response, and do not keep adding Cheerio selectors when the HTML is only an empty application shell. For more context, compare scraping and crawling before deciding whether your job is one-page extraction or multi-page discovery.
Stop Maintaining Browser Workers for Every Scrape
Use Nstproxy Crawl for rendered pages, bounded discovery, and structured outputs while your JavaScript handles validation and business logic.
Detailed Tutorial: Build a JavaScript Scraper Step by Step
This tutorial extracts book titles, prices, and canonical URLs from Books to Scrape, a public practice site built for scraping exercises. The workflow is intentionally bounded to one page.
Method 1: Scrape static HTML with fetch and Cheerio
Step 1: Create the project
Use a current Node.js runtime with built-in fetch() and install Cheerio:
The script below checks the HTTP status, validates the content type, parses each product card, normalizes URLs, and rejects an empty result. Those checks make failures visible instead of returning a successful-looking empty array.
The result shape is stable even if presentation text around the cards changes:
{"count":20,"sample":{"title":"A Light in the Attic","priceText":"£51.77","url":"https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html"}}
Treat that output as a contract. A record is accepted only when its title, price format, and absolute URL pass validation. A selector returning twenty nodes is not proof that those nodes are the right twenty products.
Method 2: Render JavaScript with Playwright
Step 1: Confirm that rendering is necessary
Open the network response or disable JavaScript in a test browser. If the target values are already in the HTML, stay with Method 1. If they arrive after an XHR/fetch call, prefer an authorized structured endpoint when one is documented; otherwise render the page.
Step 2: Wait for a semantic condition
Playwright locators resolve against the current DOM and include auto-waiting behavior. A production script should still set a navigation timeout and wait for the specific collection it needs:
Avoid waitForTimeout(5000) as a readiness strategy. It is slow on fast pages and still races on slow pages. Wait for a result container, a known API response, or another condition that means the page is actually ready.
Make a JavaScript scraper production-safe
Production safety comes from bounding work and testing meaning, not from sending requests faster.
Timeout every network boundary. Cover DNS, connection, response, browser navigation, and downstream storage. A hung request must not occupy a worker indefinitely.
Retry only transient failures. Retry selected network errors, 429, and some 5xx responses with bounded exponential backoff and jitter. Do not retry invalid selectors or permanent authorization failures.
Limit concurrency per host. Start conservatively, observe latency and error rates, and respect Retry-After when supplied.
Use stable identifiers. Store the canonical URL or a site-provided product ID and make writes idempotent so a retry does not duplicate records.
Validate content, not only status. Check expected fields, value formats, language, and a minimum/maximum record band. A soft 404 often returns 200.
Record operational context. Log the target host, status, duration, retry count, parser version, and validation outcome—never credentials or private response bodies.
Use Nstproxy Crawl When Rendering, Retries, and Site Discovery Become the Bottleneck
A JavaScript scraper has crossed into infrastructure work when browser workers, retries, extraction cleanup, task state, and artifact storage take more effort than the records you actually need. Nstproxy Crawl addresses that bottleneck as a managed web collection layer for public-page and bounded-site jobs. It returns structured or visual outputs so your Node.js service can stay focused on domain validation, deduplication, enrichment, and storage. Current billing supports on-demand use and subscriptions, with URL processing and proxy traffic accounted for separately; verify the live plan that matches your workload. Nstproxy Crawl can reduce operational overhead, but it does not replace permission checks or business-specific acceptance tests.
Remove browser-fleet maintenance: Use managed rendering when JavaScript execution is necessary instead of operating browser workers yourself.
Prevent uncontrolled site discovery: Set explicit depth, page limits, and inclusion/exclusion rules so a crawl cannot wander into calendars, search pages, login flows, or infinite parameters.
Deliver usable outputs to your application: Request the representation your downstream code needs, then validate the response body's success and task status before accepting it.
Measure cost against accepted records: Review current Crawl billing models and compare cost per validated record, not cost per request alone.
The API credential is a prerequisite for a live Crawl request, so no fabricated managed-service output is shown here. When you integrate it, keep the key in a secret manager or environment variable, never in source control.
Keep JavaScript Web Scraping Authorized and Bounded
Responsible JavaScript web scraping uses public or authorized data for a defined purpose and collects only what the application needs. Read the site's terms, privacy notice, and applicable law; honor contractual and technical limits; and avoid authentication bypass, paywalls, private pages, and regulated personal data without an appropriate legal basis.
The Robots Exclusion Protocol standardizes robots.txt rules for crawlers, while also stating that those rules are not access authorization. Treat robots.txt as one operational signal, not as permission to collect or reuse data. Define retention limits, delete stale raw HTML when it is no longer needed, and keep outreach or profiling workflows out of generic collection pipelines.
Conclusion: build the smallest scraper that survives change
Start with fetch() and Cheerio, promote only genuinely dynamic pages to Playwright, and move recurring rendering and crawl operations to managed infrastructure when their maintenance exceeds your domain logic. Run the static example, add schema assertions for your real authorized target, and measure accepted records before increasing concurrency. If multiple proxy sources later become an operational concern, evaluate Nstproxy Proxy Manager as a separate routing layer.
Yes, JavaScript is a strong scraping choice when your team already uses Node.js or when the target requires browser execution. Native fetch(), Cheerio, Playwright, and mature queue libraries cover workloads from one static page to maintained crawl pipelines.
Q: Should I use Cheerio or Playwright?
Use Cheerio when the initial response contains the required HTML, and use Playwright when JavaScript must run to produce the data. Confirm that difference before accepting the cost and complexity of a browser.
Q: Why does fetch return a page even for a 404?
fetch() resolves with a Response for HTTP error statuses, so your code must check response.ok or response.status. It rejects for selected network-level failures, not every unsuccessful HTTP result.
Q: How do I stop selectors from breaking?
Prefer semantic attributes, stable IDs, structured data, and scoped selectors, then validate the extracted record. Monitoring a schema fingerprint and sample output catches silent drift earlier than a node-count check.
Q: Do proxies make scraping legal?
No, proxies change network routing; they do not grant permission or remove legal, contractual, privacy, or copyright obligations. Use proxies only within a lawful, authorized collection policy.
Q: When should I use Nstproxy Crawl instead of my own scraper?
Use Nstproxy Crawl when browser rendering, bounded discovery, retries, task state, or output conversion have become recurring infrastructure work. Keep your own JavaScript layer for domain-specific validation, identity, storage, and policy enforcement.
Marcus Chen
Aug. 17th 2026
110M+ real IPs with 99.9% access success
Blazing-fast average response ~0.5s for high-concurrency tasks
From only $0.1/GB
Get immediate access to premium residential, datacenter, IPv6 and ISP proxy pools.