Link Extractor

curl "https://api.pocketutils.com/v1/link-extract?url=https://example.com" \
  -H "Authorization: Bearer YOUR_API_KEY"

const res = await fetch(
  "https://api.pocketutils.com/v1/link-extract?url=https://example.com",
  { headers: { Authorization: "Bearer YOUR_API_KEY" } },
);
const { data } = await res.json();

{
  "success": true,
  "data": {
    "url": "https://example.com",
    "scannedAt": "2026-07-21T09:14:02.000Z",
    "totalLinks": 64,
    "truncated": false,
    "counts": { "total": 64, "internal": 48, "external": 12, "mailto": 3, "tel": 1 },
    "links": [
      {
        "url": "https://example.com/pricing",
        "originalHref": "/pricing",
        "text": "Pricing",
        "rel": null,
        "type": "internal",
        "scheme": "https"
      },
      {
        "url": "mailto:hello@example.com",
        "originalHref": "mailto:hello@example.com",
        "text": "hello@example.com",
        "rel": null,
        "type": "mailto",
        "scheme": "mailto"
      }
    ]
  }
}

GET

link-extract

curl "https://api.pocketutils.com/v1/link-extract?url=https://example.com" \
  -H "Authorization: Bearer YOUR_API_KEY"

const res = await fetch(
  "https://api.pocketutils.com/v1/link-extract?url=https://example.com",
  { headers: { Authorization: "Bearer YOUR_API_KEY" } },
);
const { data } = await res.json();

{
  "success": true,
  "data": {
    "url": "https://example.com",
    "scannedAt": "2026-07-21T09:14:02.000Z",
    "totalLinks": 64,
    "truncated": false,
    "counts": { "total": 64, "internal": 48, "external": 12, "mailto": 3, "tel": 1 },
    "links": [
      {
        "url": "https://example.com/pricing",
        "originalHref": "/pricing",
        "text": "Pricing",
        "rel": null,
        "type": "internal",
        "scheme": "https"
      },
      {
        "url": "mailto:hello@example.com",
        "originalHref": "mailto:hello@example.com",
        "text": "hello@example.com",
        "rel": null,
        "type": "mailto",
        "scheme": "mailto"
      }
    ]
  }
}

Three endpoints, sharing the Broken Link Checker’s crawl engine but never checking HTTP status — this tool only reports each link’s metadata (anchor text, rel, type) and counts by type. Use it to audit outbound links, find every mailto:/tel: on a site, or feed a link graph.

1 credit per call (single-page extract or crawl start)

Extract from a single page

GET /v1/link-extract?url=... extracts every link on one page and returns immediately.

Query parameters

string

required

The page to extract links from. https:// is assumed if the scheme is omitted.

Response

object

required

Hide properties

string

required

Resolved page URL after any redirects.

string

required

ISO 8601 timestamp.

integer

required

Same as counts.total.

boolean

required

true if the page had more than 5,000 links.

object

required

Show properties

integer

required

All reported unique links, including mailto: and tel:, before the returned-array cap. Excludes bare in-page anchors and other schemes.

integer

required

Links to the same site.

integer

required

Links to a different site.

integer

required

mailto: links.

integer

required

tel: links.

In-page anchors (#...) and other schemes (javascript:, data:, …) are excluded. mailto: and tel: links are included.

array

required

Show item shape

string

required

Resolved absolute URL (or the raw href for non-HTTP schemes).

string

required

Original href after trimming surrounding whitespace.

string

required

Anchor text.

string

The rel attribute, e.g. "nofollow noopener".

string

required

"internal", "external", "mailto", or "tel".

string

required

URL scheme without the colon, e.g. "https", "mailto".

curl "https://api.pocketutils.com/v1/link-extract?url=https://example.com" \
  -H "Authorization: Bearer YOUR_API_KEY"

const res = await fetch(
  "https://api.pocketutils.com/v1/link-extract?url=https://example.com",
  { headers: { Authorization: "Bearer YOUR_API_KEY" } },
);
const { data } = await res.json();

{
  "success": true,
  "data": {
    "url": "https://example.com",
    "scannedAt": "2026-07-21T09:14:02.000Z",
    "totalLinks": 64,
    "truncated": false,
    "counts": { "total": 64, "internal": 48, "external": 12, "mailto": 3, "tel": 1 },
    "links": [
      {
        "url": "https://example.com/pricing",
        "originalHref": "/pricing",
        "text": "Pricing",
        "rel": null,
        "type": "internal",
        "scheme": "https"
      },
      {
        "url": "mailto:hello@example.com",
        "originalHref": "mailto:hello@example.com",
        "text": "hello@example.com",
        "rel": null,
        "type": "mailto",
        "scheme": "mailto"
      }
    ]
  }
}

Extract from an entire site

Use the crawl API when you need links from multiple pages. It returns unique internal, external, mailto:, and tel: links, with the pages where each was found. It does not check whether link destinations work.

Action	Method and path	Success response	Credits
Start extraction	`POST /v1/link-extract/crawl`	`202` and a job ID	1
Get progress or results	`GET /v1/link-extract/crawl/{jobId}`	`200` and a job snapshot	0

The base URL is https://api.pocketutils.com. Replace {jobId} with data.jobId from the start response; do not include the braces.

Authentication and credits

Use Authorization: Bearer YOUR_API_KEY or X-API-Key: YOUR_API_KEY on API requests. API-key callers do not need a Turnstile token. Website callers using a session cookie or the anonymous allowance must provide X-Turnstile-Token when starting a crawl. Polling does not require Turnstile. The start request is billed when it returns 202. Polling costs 0 credits, but every poll counts against your requests-per-minute limit. A later background job failure does not automatically refund an accepted start request. See Authentication and Rate Limits & Credits.

Start request

Send Content-Type: application/json and a JSON object with the following field:

Field	Type	Required	Description
`url`	string	Yes	Entry page on the website. Prefer a complete `https://` or `http://` URL. Whitespace is trimmed; `https://` is added when the scheme is omitted.

The request supports only url. Page limits, timeouts, link caps, custom headers, and crawl depth are not configurable through this endpoint. Use the site root for a general audit. A URL under /docs does not restrict the crawl to that path. A successful start returns 202 Accepted with success: true and data.jobId, a UUID string. Save this ID and poll the matching tool’s status endpoint. The response contains no results yet. Each start creates a new job. Repeating the POST creates another job and can incur another charge; reuse the existing job ID to check progress.

Start example

curl -i -X POST "https://api.pocketutils.com/v1/link-extract/crawl" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com"}'

202 Accepted

{
  "success": true,
  "data": {
    "jobId": "a1b2c3d4-5678-4e9f-9a0b-1c2d3e4f5a6b"
  }
}

Get progress and results

GET /v1/link-extract/crawl/{jobId} takes one required path parameter:

Parameter	Type	Description
`jobId`	string	The extraction job ID returned by `POST /v1/link-extract/crawl`. IDs from Broken Link Checker jobs are not interchangeable.

No request body or query parameters are required. The completed result is returned by this same endpoint; there is no separate result download or pagination endpoint.

curl "https://api.pocketutils.com/v1/link-extract/crawl/a1b2c3d4-5678-4e9f-9a0b-1c2d3e4f5a6b" \
  -H "Authorization: Bearer YOUR_API_KEY"

Job response fields

Field	Type	Description
`success`	boolean	`true` when the job lookup succeeds.
`data.jobId`	string	Requested job ID.
`data.status`	string	`queued`, `sitemap`, `crawling`, `done`, or `error`.
`data.progress`	object	Latest progress counters, described below.
`data.result`	object, optional	Present when `status` is `done`.
`data.error`	string, optional	Failure message when `status` is `error`.

Phases

Phase	Meaning	Client action
`queued`	Job created; preparing discovery.	Keep polling.
`sitemap`	Sitemap discovery phase.	Keep polling.
`crawling`	Fetching internal pages and collecting links.	Keep polling.
`done`	Extraction finished; result available.	Save `data.result` and stop polling.
`error`	Background extraction failed.	Read `data.error` and stop polling.

There is no checking phase in Link Extractor.

Progress fields

Field under `data.progress`	Type	Description
`phase`	string	Same phase as `data.status`.
`message`	string	Human-readable progress message; do not use it as a machine status.
`sitemapFound`	boolean	Whether sitemap discovery found a fetchable sitemap candidate; this does not guarantee eligible page URLs were found.
`sitemapUrlCount`	integer	Eligible internal sitemap URLs retained, up to 400.
`pagesScanned`	integer	Pages fetched and recorded so far, including pages without usable HTML.
`pagesDiscovered`	integer	Unique page URLs known so far, including queued pages. This can grow during the crawl.
`pagesCap`	integer	Fixed page cap: 250.
`linksFound`	integer	Unique links discovered before extractor filtering; may include anchors and other schemes excluded from the final result.

These are counters, not a stable percentage or estimated completion time.

200 OK — running

{
  "success": true,
  "data": {
    "jobId": "a1b2c3d4-5678-4e9f-9a0b-1c2d3e4f5a6b",
    "status": "crawling",
    "progress": {
      "phase": "crawling",
      "message": "Crawling internal pages…",
      "sitemapFound": true,
      "sitemapUrlCount": 58,
      "pagesScanned": 22,
      "pagesDiscovered": 58,
      "pagesCap": 250,
      "linksFound": 340
    }
  }
}

Completed result fields

Field under `data.result`	Type	Description
`url`	string	Normalized entry URL.
`scannedAt`	string	ISO 8601 UTC timestamp when the result was assembled.
`totalLinks`	integer	All unique reported links discovered across scanned pages; equals `counts.total`.
`truncated`	boolean	`true` if more than 3,000 reported links were discovered and the returned `links` array was capped.
`counts`	object	Counts across all reported unique links, including links omitted from the capped array.
`links`	array	Up to 3,000 reported links, sorted by type then URL within the returned subset.

counts contains integer fields total, internal, external, mailto, and tel. The four type counts sum to total. Bare in-page anchors and other schemes are excluded; mailto: and tel: are included even though they are not HTTP links.

Link fields

Field under `data.result.links[]`	Type	Description
`url`	string	Resolved HTTP(S) destination, or the original `mailto:` / `tel:` href.
`originalHref`	string	Original href after trimming surrounding whitespace; may be relative.
`text`	string	Whitespace-normalized anchor text, capped at 120 characters. Falls back to `aria-label`, `title`, image alt text, then `(no anchor text)`.
`rel`	string or null	Trimmed `rel` attribute, or `null` when absent or empty.
`type`	string	`internal`, `external`, `mailto`, or `tel`.
`scheme`	string	Scheme without the colon, such as `https`, `mailto`, or `tel`.
`sources`	string array	Up to 25 scanned source page URLs containing this link.
`sourceCount`	integer	Total distinct scanned source pages containing this link; may exceed `sources.length`.

The crawl result contains links and counts only. It does not include a pages array, page totals, HTTP status codes, or a health score. Read page counters from data.progress. Use Broken Link Checker to check destinations.

200 OK — completed example

{
  "success": true,
  "data": {
    "jobId": "a1b2c3d4-5678-4e9f-9a0b-1c2d3e4f5a6b",
    "status": "done",
    "progress": {
      "phase": "done",
      "message": "Extraction complete.",
      "sitemapFound": false,
      "sitemapUrlCount": 0,
      "pagesScanned": 2,
      "pagesDiscovered": 2,
      "pagesCap": 250,
      "linksFound": 2
    },
    "result": {
      "url": "https://example.com/",
      "scannedAt": "2026-10-12T09:14:02.000Z",
      "totalLinks": 2,
      "truncated": false,
      "counts": {
        "total": 2,
        "internal": 1,
        "external": 0,
        "mailto": 1,
        "tel": 0
      },
      "links": [
        {
          "url": "https://example.com/pricing",
          "originalHref": "/pricing",
          "text": "Pricing",
          "rel": null,
          "type": "internal",
          "scheme": "https",
          "sources": ["https://example.com/", "https://example.com/pricing"],
          "sourceCount": 2
        },
        {
          "url": "mailto:hello@example.com",
          "originalHref": "mailto:hello@example.com",
          "text": "Email us",
          "rel": null,
          "type": "mailto",
          "scheme": "mailto",
          "sources": ["https://example.com/"],
          "sourceCount": 1
        }
      ]
    }
  }
}

How the site is crawled

The crawler looks for sitemap locations in robots.txt and at /sitemap.xml. It expands one level of sitemap indexes, then scans the entry page and follows discovered internal page links breadth-first. A missing sitemap does not prevent crawling. Internal links are classified by hostname, ignoring a leading www.. Other subdomains count as external. External destinations are collected but are not traversed to discover more pages. PDFs, images, archives, and other recognized asset extensions are not traversed as HTML pages. The crawler reads <a href="..."> elements from the fetched HTML. It does not execute page JavaScript, so links added only after client-side rendering may be absent. It does not forward your PocketUtils API key or browser session to the target website. Pages are deduplicated after removing URL fragments and trailing path slashes. Link records are deduplicated by scheme and resolved URL; different query strings or HTTP link fragments can remain separate records. Repeated occurrences on one page do not increase sourceCount. Link metadata represents one discovered occurrence, rather than every variation of its text or rel attribute.

Crawl and result limits

Setting	Value	Meaning
Pages	250	Maximum internal page fetches, including the entry page.
Page crawl budget	6 minutes	Budget for following queued pages, after sitemap discovery and the entry fetch. In-flight requests may finish afterward.
Sitemap page URLs	400	Maximum retained eligible sitemap URLs. This does not increase the page cap.
Sitemap candidates / child maps	15 each	Up to 15 sitemap candidates and up to 15 child maps; indexes are expanded one level.
Concurrent page fetches	6	Number of pages fetched at once.
Page request timeout	15 seconds	Per page fetch.
Sitemap request timeout	10 seconds	Per sitemap or robots request.
Source page sample	25 per link	`sourceCount` includes all discovered source pages even if `sources` is capped.

These limits are fixed by the service. The six-minute page budget is not a promise that the whole job finishes within six minutes. The returned link array is capped at 3,000. totalLinks and counts still cover all reported links discovered within the scanned pages. truncated: false does not prove the entire website was scanned: it describes only the link array cap. Link Extractor does not expose a separate page-truncation flag. Compare the page counters and treat results as an inventory of what this bounded crawl discovered.

Retention and polling behavior

Job state is held in memory. Jobs become eligible for removal 30 minutes after their last progress, result, or error update; cleanup runs every five minutes. Reading a job does not refresh retention. A server restart can also lose a job. Store the completed result in your application if you need it later. Poll about every two seconds for a single job, and use a longer interval when polling several jobs or sharing a small request allowance. Phases can change between polls, so you do not need to observe every phase. Stop on done or error. See Async Jobs & Polling for complete JavaScript and Python examples. A known job returns 200 OK even when data.status is error. The envelope’s success: true means the status lookup succeeded; inspect data.status to determine whether the crawl succeeded. result and error are omitted until applicable, rather than sent as null. Request fresh status snapshots and avoid caching them in your client. The current status handlers do not explicitly set Cache-Control: no-store.

Errors

HTTP status	Applies to	Meaning and action
`400`	Crawl start	Missing, non-string, or invalid `url`. Correct the request before retrying.
`401`	Start or poll	Supplied API key is invalid or revoked. Check your credentials.
`402`	Crawl start	Insufficient credits for the 1-credit start request.
`403`	Crawl start	Website caller omitted a valid Turnstile token. API-key callers bypass this check.
`404`	Status lookup	Extraction job is unknown, expired, lost after restart, or belongs to the other tool.
`429`	Start or poll	Request rate exceeded; slow down. Anonymous start requests can also exhaust the free credit allowance.

Crawl validation and missing-job errors use a string error. Platform authentication, credit, and rate-limit errors use an error object with code and message. Handle both shapes; see Errors.

400 Bad Request — missing URL

{
  "success": false,
  "error": "URL is required and must be a string"
}

404 Not Found

{
  "success": false,
  "error": "Extraction job not found or expired."
}

If the background crawl cannot obtain usable HTML from the entry page or encounters a fatal error, status polling can return a job failure with HTTP 200:

200 OK — failed job

{
  "success": true,
  "data": {
    "jobId": "a1b2c3d4-5678-4e9f-9a0b-1c2d3e4f5a6b",
    "status": "error",
    "progress": {
      "phase": "error",
      "message": "The URL did not return an HTML page to crawl.",
      "sitemapFound": false,
      "sitemapUrlCount": 0,
      "pagesScanned": 0,
      "pagesDiscovered": 0,
      "pagesCap": 250,
      "linksFound": 0
    },
    "error": "The URL did not return an HTML page to crawl."
  }
}

Stop polling a failed job. Correct the target URL or resolve the target site’s access problem before starting a new crawl.

Getting Started

API Reference

Extract from a single page

Query parameters

Response

Extract from an entire site

Authentication and credits

Start request

Start example

Get progress and results

Job response fields

Phases

Progress fields

Completed result fields

Link fields

How the site is crawled

Crawl and result limits

Retention and polling behavior

Errors