crawl_page

分类: 数据与AI | 上传者: octoparse | 下载: 0 | 版本: v1.0（最新）

Crawls a web page and returns its HTML content. Use this skill when you need to fetch the raw HTML of a URL for further processing, analysis, or extraction. Supports custom headers and timeout configuration.

更新日志: Source: GitHub https://github.com/octoparse/agent-skills

目录结构

当前层级: skills/crawl_page/

📄 example.json 52 B
📄 README.md 600 B
📄 skill.json 911 B
📄 SKILL.md 1.0 KB

SKILL.md

---
name: crawl_page
description: "Crawls a web page and returns its HTML content. Use this skill when you need to fetch the raw HTML of a URL for further processing, analysis, or extraction. Supports custom headers and timeout configuration."
license: MIT
compatibility: Requires HTTP endpoint. Compatible with agent frameworks.
metadata:
  invoke_type: "http"
  endpoint: "https://mcp.octoparse.com"
---

# crawl_page

## Overview

Crawls a web page and returns its HTML content. Use this skill when you need to fetch the raw HTML of a URL for further processing, analysis, or extraction. Supports custom headers and timeout configuration.

## Parameters

- **url** (`string`) (required): The URL of the web page to crawl. Must be a valid HTTP or HTTPS URL.
- **timeout** (`integer`): Request timeout in seconds. Default is 30.
- **headers** (`object`): Optional custom HTTP headers to send with the request (e.g., User-Agent).

## Invoke

- **Type**: http
- **Endpoint**: https://mcp.octoparse.com

## Example

See [example.json](example.json) for sample input.

登录后下载/点赞/收藏 ❤ 2 | ★ 0

crawl_page

目录结构

SKILL.md

举报内容

提示