Spider · Capability

Spider Cloud API — Crawling

Spider Cloud API — Crawling. Recursively crawl entire websites and collect every page as clean markdown, JSON, HTML, text, or other supported output formats. Self-contained Naftiko capability covering the crawling business surface.

Spider Cloud API — Crawling is a Naftiko capability published by Spider, one of 5 capabilities the APIs.io network indexes for this provider. It bundles 1 operation across the POST method rooted at /v1/crawl.

The capability includes 1 read-only operation. Lead operation: Recursively crawl an entire website and collect every page. Can be deployed as a REST endpoint, MCP tool, or Agent Skill via Naftiko.

Tagged areas include Spider Cloud and Crawling.

Run with Naftiko Spider CloudCrawling

What You Can Do

POST
Crawl — Recursively crawl an entire website and collect every page.
/v1/crawl

MCP Tools

crawl-website

Recursively crawl an entire website and collect every page.

read-only

Capability Spec

spider-cloud-crawling.yaml Raw ↑ 
naftiko: 1.0.0-alpha2
info:
  label: Spider Cloud API — Crawling
  description: 'Spider Cloud API — Crawling. Recursively crawl entire websites and collect every page as
    clean markdown, JSON, HTML, text, or other supported output formats. Self-contained Naftiko capability
    covering the crawling business surface.'
  tags:
    - Spider Cloud
    - Crawling
  created: '2026-05-25'
  modified: '2026-05-25'
binds:
  - namespace: env
    keys:
      SPIDER_CLOUD_API_KEY: SPIDER_CLOUD_API_KEY
capability:
  consumes:
    - type: http
      namespace: spider-cloud-crawling
      baseUri: https://api.spider.cloud
      description: Spider Cloud API — Crawling business capability. Self-contained, no shared references.
      resources:
        - name: crawl
          path: /crawl
          operations:
            - name: crawl
              method: POST
              description: Recursively crawl an entire website and collect every page.
              outputRawFormat: json
              outputParameters:
                - name: result
                  type: object
                  value: $.
              inputParameters:
                - name: body
                  in: body
                  type: object
                  description: Request body (JSON).
                  required: true
      authentication:
        type: bearer
        token: '{{env.SPIDER_CLOUD_API_KEY}}'
  exposes:
    - type: rest
      namespace: spider-cloud-crawling-rest
      port: 8080
      description: REST adapter for Spider Cloud API — Crawling. One Spectral-compliant resource per consumed
        operation, prefixed with /v1.
      resources:
        - path: /v1/crawl
          name: crawl
          description: REST surface for crawl.
          operations:
            - method: POST
              name: crawl
              description: Recursively crawl an entire website and collect every page.
              call: spider-cloud-crawling.crawl
              with:
                body: rest.body
              outputParameters:
                - type: object
                  mapping: $.
    - type: mcp
      namespace: spider-cloud-crawling-mcp
      port: 9090
      transport: http
      description: MCP adapter for Spider Cloud API — Crawling. One tool per consumed operation, routed inline
        through this capability's consumes block.
      tools:
        - name: crawl-website
          description: Recursively crawl an entire website and collect every page.
          hints:
            readOnly: true
            destructive: false
            idempotent: false
          call: spider-cloud-crawling.crawl
          with:
            body: tools.body
          outputParameters:
            - type: object
              mapping: $.