Browsing

Service usage

‘s browser service navigates intelligently on your behalf, rendering JavaScript, solving captchas, and retrying failed requests. The service has a RESTful interface that accepts GET requests at https://‍/browser and a Chrome DevTools Protocol (CDP) interface for controlling browser navigation yourself that accepts WebSocket connections at wss:///browser. Webpages can be localized with geotargeting parameters. Those requested via REST are retrieved in real time by default or optionally queued for subsequent retrieval. Up to 3 minutes is allotted per real-time API call to accommodate captcha-solving, multiple retries, and other mitigations. Usage data is published regularly.

If a site you’re targeting remains blocked, even with different browsing params, let us know. We can usually help unblock any site within 48 hours.

Authentication

You can access the service by including your secret API token in an Authorization header:

Authorization: Bearer [API token here]

Here’s an example request that uses the common Curl command:

curl -H "Authorization: Bearer $MASSIVE_TOKEN" \
'https://render.joinmassive.com/browser'\
'?url=https://example.com/'

REST parameters

Besides the geotargeting and scheduling params linked above, required and optional REST params can be added in a standard query string. See our rate card for the prices of premium params. The keys and values supports are as follows:

Key	Required	Premium	Value
`url`	✅	⬜	The URL of up to 2,047 characters of the page to browse; any unsafe characters require URL encoding
`difficulty`	⬜	✅	The difficulty pool to attempt to access the URL from, `low`, `medium`, or `high` (planned); `low` is the default difficulty
`speed`	⬜	✅	The speed to attempt to access the URL at, `light`, `ridiculous`, or `ludicrous` (planned), where `ridiculous` is 30 percent faster on average than `light` speed; `light` is the default speed
`device`	⬜	⬜	The name as returned by the devices resource of the device to emulate browsing on (these names are case insensitive but must include form- or URL-encoded spaces and punctuation marks); device emulation is unused by default
`session`	⬜	⬜	Any unique identifier of up to 255 characters (regardless of character encoding); will make best efforts to route calls in the same session to the same egress node for up to 12 minutes
`captcha`	⬜	⬜	The intended resolution of any detected captcha, `solved`, `ignored`, or `rejected`, where `rejected` results in a `403` response; `solved` is the default captcha resolution
`readiness`	⬜	⬜	The standard ready event to await before snapshotting browsed content, `load` or `domcontentloaded`; `load` is the default ready event
`delay`	⬜	⬜	The number of supplemental seconds to delay before snapshotting browsed content, from `.1` to `10` inclusive; no delay is used by default
`format`	⬜	⬜	The format to output to, `rendered`, `raw`, or `markdown` (see the section below); `rendered` is the default format
`expiration`	⬜	⬜	The age in days of when to consider cached content expired, where `0` disables caching; `1` is the default number of days before expiration
`callback`	⬜	⬜	The encoded HTTP or HTTPS callback URL or Amazon SQS queue URL or ARN of up to 2,047 characters to notify when the (prerequisite) asynchronous content has been retrieved; any SQS queue must grant `sqs:SendMessage` permission to the `arn:aws:iam::180363035301:role/api-instance` AWS principal

CDP interaction

If your use case involves interacting with not just reading pages, you can connect Puppeteer, Playwright, or another CDP-compatible automation framework to the browser service:

#!/usr/bin/env node
import puppeteer from 'puppeteer';

const browser = await puppeteer.connect({
  browserWSEndpoint: 'wss://render.joinmassive.com/browser',
  headers: {
    Authorization: `Bearer ${process.env.MASSIVE_TOKEN}`
  }
});

View Playwright sample

#!/usr/bin/env python3
import os
from playwright.sync_api import sync_playwright

with sync_playwright() as playwright:
    browser = playwright.chromium.connect_over_cdp(
        'wss://render.joinmassive.com/browser',
        headers={
            'Authorization':
                f"Bearer {os.environ['MASSIVE_TOKEN']}"
        }
    )

The connection string takes geotargeting params and a subset of the REST params, namely difficulty, device, and session:

const browser = await puppeteer.connect({
  browserWSEndpoint:
    'wss://render.joinmassive.com/browser' +
    '?difficulty=medium',
  headers: {
    Authorization: `Bearer ${process.env.MASSIVE_TOKEN}`
  }
});

After connecting, most of your framework’s API (see the Puppeteer reference, Playwright reference, et cetera) will be available:

const [page] = await browser.pages();

await page.goto('https://example.com/');
console.log('URL:', page.url());
console.log('Title:', await page.title());

CDP use is metered by time, so you should disconnect from the service when done by calling browser.disconnect() (Puppeteer), browser.close() (Playwright), or the equivalent.

Protocol mixture

You can control costs by mixing REST requests and CDP use within a sticky browser session. E.g., you may want to prepare a page for interaction over REST then interact with the page over CDP:

View REST-to-CDP sample

#!/usr/bin/env node
import puppeteer from 'puppeteer';

const restEndpoint = 'https://render.joinmassive.com/browser';
const cdpEndpoint = 'wss://render.joinmassive.com/browser';
const apiToken = process.env.MASSIVE_TOKEN;
const headers = { Authorization: `Bearer ${apiToken}` };
const sessionId = Date.now();
const orderUrl = 'https://httpbin.org/forms/post';
const sizeSelector = '[name="size"][value="small"]';
const toppingSelector = '[name="topping"][value="cheese"]';
const buttonSelector = 'button';
const requestTimeoutMs = 180_000;
const connectionTimeoutMs = 10_000;
const delayMs = 1_000;
const delay = (ms) => {
  return new Promise((resolve) => {
    setTimeout(resolve, ms);
  });
};
const getCookies = (response) => {
  const cookies = response.headers.getSetCookie?.() ?? [];
  const keyvals = cookies.map((cookie) => {
    return cookie.split(';', 1)[0];
  });

  return keyvals.join('; ');
};

console.log(
  'Making REST request with session',
  sessionId,
  '...'
);

const controller = new AbortController();
let timeoutId = setTimeout(() => {
  controller.abort();
}, requestTimeoutMs);
let response;

try {
  const url =
    restEndpoint +
    '?url=' +
    encodeURIComponent(orderUrl) +
    '&session=' +
    sessionId +
    '&expiration=0';
  response = await fetch(
    url,
    { headers, signal: controller.signal }
  );
} catch (error) {
  const hasTimedOut = error.name == 'AbortError';
  response = new Response(
    hasTimedOut
      ? 'Request timed out'
      : 'Unknown error occurred',
    { status: hasTimedOut ? 504 : 500 }
  );
} finally {
  clearTimeout(timeoutId);
}

if (response.ok) {
  console.log('Order form read successfully');
} else {
  const text = await response.text();

  console.error('Order form read failure:', text.trimEnd());

  process.exit(1);
}

console.log(
  'Connecting via CDP with session',
  sessionId,
  '...'
);

timeoutId = setTimeout(() => {
  console.error('Connection timed out');

  process.exit(1);
}, connectionTimeoutMs);
const url = `${cdpEndpoint}?session=${sessionId}`;
const browser = await puppeteer.connect({
  browserWSEndpoint: url,
  headers: { ...headers, Cookie: getCookies(response) }
});

clearTimeout(timeoutId);

const page = (await browser.pages()).find((page) => {
  return page.url().includes(orderUrl);
});

if (page) {
  console.log('Session handed off successfully');
} else {
  console.error('Session handoff failure');

  process.exit(1);
}

await page.waitForSelector(sizeSelector);
await page.click(sizeSelector);
await delay(delayMs);
await page.waitForSelector(toppingSelector);
await page.click(toppingSelector);
await delay(delayMs);
await page.waitForSelector(buttonSelector);
await Promise.all([
  page.waitForNavigation({ waitUntil: 'domcontentloaded' }),
  page.click(buttonSelector)
]);

try {
  const json = await page.evaluate(() => {
    return document.body.innerText;
  });
  const data = JSON.parse(json);

  if (
    data.form &&
    typeof data.form.size == 'string' &&
    typeof data.form.topping == 'string'
  ) {
    console.log(
      'Ordered',
      data.form.size.toLowerCase(),
      'pizza with',
      data.form.topping.toLowerCase()
    );
  } else {
    console.error('Receipt was malformed');

    process.exit(1);
  }
} catch (error) {
  console.error('Receipt parse failure:', error.message);

  process.exit(1);
}

await browser.disconnect();

To hand off in reverse, from CDP to REST, call your disconnect method first.

Device emulation

The device param lets you browse device-specific content, rather than the default desktop content. For a list of supported smartphone and tablet devices, make a request with your API token and no params to https://‍/browser/devices:

curl -H "Authorization: Bearer $MASSIVE_TOKEN" \
'https://render.joinmassive.com/browser/devices'

The API will return JSON that contains an alphabetized array of device names:

[
  "Blackberry PlayBook",
  "Blackberry PlayBook landscape",
  "BlackBerry Z30",
  "[Remaining device names here]"
]

Then, pass your device name of choice into your browsing request or connection:

curl -H "Authorization: Bearer $MASSIVE_TOKEN" \
'https://render.joinmassive.com/browser'\
'?url=https://crackberry.com/'\
'&device=Blackberry+Playbook'

const browser = await puppeteer.connect({
  browserWSEndpoint:
    'wss://render.joinmassive.com/browser' +
    '?device=Blackberry+Playbook',
  headers: {
    Authorization: `Bearer ${process.env.MASSIVE_TOKEN}`
  }
});

Response format

Read-only pages are returned as rendered HTML, raw (unrendered) HTML, or Markdown optimized for LLM prompts.

Additional examples

Cachebusting

curl -H "Authorization: Bearer $MASSIVE_TOKEN" \
'https://render.joinmassive.com/browser'\
'?url=https://api.weather.gov/alerts'\
'&expiration=0'

End-to-end Playwright

View end-to-end sample

#!/usr/bin/env python3
import os
from playwright.sync_api import sync_playwright

ENDPOINT = 'wss://render.joinmassive.com/browser'
API_TOKEN = os.environ['MASSIVE_TOKEN']
URL = 'https://example.com/'

print(f"Connecting to {ENDPOINT} ...")

with sync_playwright() as playwright:
    browser = playwright.chromium.connect_over_cdp(
        ENDPOINT,
        headers={'Authorization': f"Bearer {API_TOKEN}"}
    )
    context = browser.contexts[0]
    page = context.pages[0]

    print(f"Going to {URL} ...")

    page.goto(URL)
    print(f"URL: {page.url}")
    print(f"Title: {page.title()}")
    browser.close()

Get started

Endpoints

Configuration

API Management

API Reference

Service usage

Authentication

REST parameters

CDP interaction

Protocol mixture

Device emulation

Response format

Additional examples

Cachebusting

End-to-end Playwright

​Service usage

​Authentication

​REST parameters

​CDP interaction

​Protocol mixture

​Device emulation

​Response format

​Additional examples

​Cachebusting

​End-to-end Playwright

Service usage

Authentication

REST parameters

CDP interaction

Protocol mixture

Device emulation

Response format

Additional examples

Cachebusting

End-to-end Playwright