Back to Crawlee

Using puppeteer-extra and playwright-extra

website/versioned_docs/version-3.10/examples/crawler-plugins/index.mdx

3.16.02.7 KB
Original Source

import RunnableCodeBlock from '@site/src/components/RunnableCodeBlock'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import ApiLink from '@site/src/components/ApiLink'; import PuppeteerExtraSource from '!!raw-loader!roa-loader!./puppeteer-extra.ts'; import PlaywrightExtraSource from '!!raw-loader!roa-loader!./playwright-extra.ts';

puppeteer-extra and playwright-extra are community-built libraries that bring in a plugin system to enhance the usage of puppeteer and playwright respectively (bringing in extra functionality, like improving stealth for example by using the Puppeteer Stealth plugin (puppeteer-extra-plugin-stealth)).

:::tip Available plugins

You can see a list of available plugins on the puppeteer-extras plugin list.

For playwright, please see playwright-extras plugin list instead.

:::

In this example, we'll show you how to use the Puppeteer Stealth (puppeteer-extra-plugin-stealth) plugin to help you avoid bot detections when crawling your target website.

<Tabs> <TabItem value="puppeteer" label="Puppeteer & puppeteer-extra" default>

:::info Before you begin

Make sure you've installed the Puppeteer Extra (puppeteer-extra) and Puppeteer Stealth plugin(puppeteer-extra-plugin-stealth) packages via your preferred package manager

bash
npm install puppeteer-extra puppeteer-extra-plugin-stealth

:::

:::tip

To run this example on the Apify Platform, select the apify/actor-node-puppeteer-chrome image for your Dockerfile.

:::

<RunnableCodeBlock className="language-js" title="src/crawler.ts" type="puppeteer"> {PuppeteerExtraSource} </RunnableCodeBlock> </TabItem> <TabItem value="playwright" label="Playwright & playwright-extra">

:::info Before you begin

Make sure you've installed the playwright-extra and puppeteer-extra-plugin-stealth packages via your preferred package manager

bash
npm install playwright-extra puppeteer-extra-plugin-stealth

:::

:::tip

To run this example on the Apify Platform, select the apify/actor-node-puppeteer-chrome image for your Dockerfile.

:::

<RunnableCodeBlock className="language-js" title="src/crawler.ts" type="playwright"> {PlaywrightExtraSource} </RunnableCodeBlock> </TabItem> </Tabs>