Skipping navigations for certain requests

import RunnableCodeBlock from '@site/src/components/RunnableCodeBlock'; import ApiLink from '@site/src/components/ApiLink'; import SkipNavigationSource from '!!raw-loader!roa-loader!./skip-navigation.ts';

While crawling a website, you may encounter certain resources you'd like to save, but don't need the full power of a crawler to do so (like images delivered through a CDN).

By combining the <ApiLink to="core/class/Request#skipNavigation">Request#skipNavigation</ApiLink> option with <ApiLink to="basic-crawler/interface/BasicCrawlingContext#sendRequest">sendRequest</ApiLink>, we can fetch the image from the CDN, and save it to our key-value store without needing to use the full crawler.

:::info

For this example, we are using the <ApiLink to="playwright-crawler/class/PlaywrightCrawler">PlaywrightCrawler</ApiLink> to showcase this, but this is available on all the crawlers we provide.

:::

<RunnableCodeBlock className="language-js"> {SkipNavigationSource} </RunnableCodeBlock>