comfyui-skill-openclaw

分类: 工具与效率 | 上传者: HuangYuChuh | 下载: 0 | 版本: v1.0（最新）

利用 ComfyUI 强大的基于节点的工作流程功能生成图像。支持从不同实例动态加载多个预配置的生成工作流程及其相应的参数映射，从ComfyUI或本地JSON文件批量导入保存的工作流程，将自然语言转换为参数，驱动本地或远程ComfyUI服务，跟踪参数和结果的执行历史，最终将图像返回到目标客户端。 **在以下情况下使用此技能：** (1) 用户请求“生成图像”、“绘制图片”或“执行 ComfyUI 工作流程”。 (2) 用户对图像生成有特定的风格、特征或场景要求。 (3) 用户要求您导入、注册、同步或配置已保存的 ComfyUI 工作流程以供以后重用。 --- # ComfyUI Agent SKILL ## 核心执行规范作为配备 ComfyUI 技能的 OpenClaw Agent，您的目标是将用户的对话请求转换为严格的结构化参数，并将它们交给底层 Python 脚本以跨多服务器环境执行工作流程。 ### UI 管理快捷方式如果用户要求您打开、启动或调出此技能的本地 Web UI，请运行：``bash python3 ./ui/open_ui.py``

更新日志: Source: GitHub https://github.com/HuangYuChuh/ComfyUI_Skills_OpenClaw

目录结构

当前层级: tree/main/

📁 .github/
- 📁 workflows/
  - 📄 ci.yml 659 B
  - 📄 sync-frontend.yml 3.6 KB
📁 asset/
- 📄 banner-ui-dashboard-20260322.png 504.5 KB
📁 data/
- 📄 .gitkeep 0 B
📁 docs/
- 📁 _data/
  - 📄 faq.yml 2.4 KB
- 📁 _includes/
  - 📄 jsonld.html 1.6 KB
- 📁 _layouts/
  - 📄 default.html 2.0 KB
- 📁 assets/
  - 📄 logo.png 898.4 KB
  - 📄 site.css 10.3 KB
- 📄 _config.yml 745 B
- 📄 architecture.md 3.4 KB
- 📄 comfyui-native-routes.md 9.0 KB
- 📄 comfyui-native-routes.zh.md 2.0 KB
- 📄 faq.md 1.1 KB
- 📄 Gemfile 140 B
- 📄 Gemfile.lock 11.3 KB
- 📄 getting-started.md 4.2 KB
- 📄 index.md 1.2 KB
- 📄 llms-full.txt 2.2 KB
- 📄 llms.txt 1.1 KB
- 📄 PROJECT_DISCOVERY_CHECKLIST.md 3.0 KB
- 📄 robots.txt 342 B
- 📄 use-cases.md 3.3 KB
📁 outputs/
- 📄 .gitkeep 0 B
📁 references/
- 📄 workflow-import.md 2.6 KB
📁 scripts/
- 📁 shared/
  - 📄 __init__.py 38 B
  - 📄 config.py 2.3 KB
  - 📄 execution_history.py 4.4 KB
  - 📄 frontend_update.py 7.5 KB
  - 📄 health.py 1.4 KB
  - 📄 json_utils.py 495 B
  - 📄 runtime_config.py 4.9 KB
  - 📄 transfer_bundle.py 30.6 KB
  - 📄 updater.py 8.5 KB
- 📄 clean_generated_ui.sh 1.7 KB
- 📄 comfyui_client.py 38.9 KB
- 📄 registry.py 5.0 KB
- 📄 serve_pages.sh 407 B
- 📄 server_manager.py 5.0 KB
- 📄 transfer_manager.py 4.2 KB
- 📄 update_frontend.sh 1.9 KB
📁 tests/
- 📁 ui/
  - 📄 test_dependency_checker.py 12.8 KB
  - 📄 test_subgraph_compat.py 5.4 KB
- 📄 test_comfyui_client.py 3.2 KB
📁 ui/
- 📁 static/
  - 📁 assets/
    
    📄 index-CljwB4pu.css 58.5 KB
    
    📄 index-l7qhsdKz.js 398.1 KB
    
    📄 versionUpgrade-DZqoNgJw.js 378 B
  - 📄 index.html 4.8 KB
  - 📄 logo.png 898.4 KB
  - 📄 version.json 84 B
- 📄 app.py 24.4 KB
- 📄 comfyui_userdata.py 4.8 KB
- 📄 dependency_checker.py 22.5 KB
- 📄 dependency_installer.py 17.1 KB
- 📄 dependency_registry.py 8.7 KB
- 📄 models.py 6.6 KB
- 📄 open_ui.py 3.8 KB
- 📄 run_ui.bat 524 B
- 📄 run_ui.command 85 B
- 📄 run_ui.sh 3.6 KB
- 📄 services.py 26.9 KB
- 📄 settings.py 1.3 KB
- 📄 workflow_format.py 16.2 KB
- 📄 workflow_import.py 12.8 KB
📄 .gitignore 564 B
📄 CHANGELOG.md 598 B
📄 config.example.json 269 B
📄 LICENSE 1.0 KB
📄 README.md 17.9 KB
📄 README.zh.md 17.0 KB
📄 requirements.txt 66 B
📄 SKILL.md 12.1 KB

SKILL.md

---
name: comfyui-skill-openclaw
description: |
  Generate images utilizing ComfyUI's powerful node-based workflow capabilities. Supports dynamically loading multiple pre-configured generation workflows from different instances and their corresponding parameter mappings, importing saved workflows in bulk from ComfyUI or local JSON files, converting natural language into parameters, driving local or remote ComfyUI services, tracking execution history with parameters and results, and ultimately returning the images to the target client.
  
  **Use this Skill when:**
  (1) The user requests to "generate an image", "draw a picture", or "execute a ComfyUI workflow".
  (2) The user has specific stylistic, character, or scene requirements for image generation.
  (3) The user asks you to import, register, sync, or configure saved ComfyUI workflows for later reuse.
---

# ComfyUI Agent SKILL

## Core Execution Specification

As an OpenClaw Agent equipped with the ComfyUI skill, your objective is to translate the user's conversational requests into strict, structured parameters and hand them over to the underlying Python scripts to execute workflows across multi-server environments.

### UI Management Shortcut

If the user asks you to open, launch, or bring up the local Web UI for this skill, run:

```bash
python3 ./ui/open_ui.py
```

This command will:
- reuse the UI if it is already running
- start it in the background if it is not running
- try to open the browser to the local dashboard automatically

### Native ComfyUI API Surface

This skill is primarily a workflow execution client for a local or remote ComfyUI server.

The core native ComfyUI routes relevant to this skill are:

- `POST /prompt` to submit a workflow run
- `GET /history/{prompt_id}` to poll for completion
- `GET /view` to download generated images

Other native ComfyUI routes such as `/ws`, `/queue`, `/interrupt`, `/upload/image`, `/object_info`, and `/system_stats` exist upstream but are not required for the basic execution path implemented here.

For the route-level reference and the distinction between native ComfyUI routes and this repository's own manager API, see [`docs/comfyui-native-routes.md`](./docs/comfyui-native-routes.md).

The local manager API also exposes higher-level workflow execution and history routes:

- `POST /api/servers/{server_id}/workflow/{workflow_id}/run`
- `GET /api/servers/{server_id}/workflow/{workflow_id}/history`
- `GET /api/servers/{server_id}/workflow/{workflow_id}/history/{run_id}`

### Server Health Check

Before running a workflow, check whether the target ComfyUI server is online.

You can query the manager API endpoint:

```http
GET /api/servers/{server_id}/status
```

This returns JSON with `"status": "online"` or `"status": "offline"`.

**Recommended agent flow:** Before Step 3 (Trigger Image Generation), run a server status check. If offline, ask the user to start ComfyUI and retry once it is online.

### Step 0: Workflow Onboarding and Import (Optional)

Use the manager UI/API when the user wants to register workflows into this skill instead of running them immediately.

- For bulk import from ComfyUI `/userdata`, local files, manager API routes, and import result semantics, read [`references/workflow-import.md`](./references/workflow-import.md).
- Prefer the bulk import flow when the user wants to sync many saved workflows at once.
- Use single-workflow configuration only when the user gives one workflow and wants a targeted setup.

#### Single-workflow auto-configuration

If the user provides you with one new ComfyUI workflow JSON (API format) and asks you to "configure it" or "add it":
1. Check the existing server configurations or default to `local`.
2. Save the provided JSON file to `./data/<server_id>/<new_workflow_id>/workflow.json`.
3. Analyze the JSON structure (look for `inputs` inside node definitions, e.g., `KSampler`'s `seed`, `CLIPTextEncode`'s `text` for positive/negative prompts, `EmptyLatentImage` for width/height).
4. Automatically generate a schema mapping file and save it to `./data/<server_id>/<new_workflow_id>/schema.json`. The schema format must follow:
   ```json
   {
     "workflow_id": "<new_workflow_id>",
     "server_id": "<server_id>",
     "description": "Auto-configured by OpenClaw",
     "enabled": true,
     "parameters": {
       "prompt": { "node_id": "3", "field": "text", "required": true, "type": "string", "description": "Positive prompt" }
       // Add other sensible parameters that the user might want to tweak
     }
   }
   ```
5. Tell the user that the new workflow on the specific server is successfully configured and ready to be used.

### Step 1: Query Available Workflows (Registry)

Before attempting to generate any image, you must **first query the registry** to understand which workflows are currently supported and enabled:
```bash
python ./scripts/registry.py list --agent
```

**Return Format Parsing**:
You will receive a JSON containing all available workflows. Notice they are uniquely identified by the combination of `server_id` and `workflow_id` (or path format `<server_id>/<workflow_id>`):
- For parameters with `required: true`, if the user hasn't provided them, you must **ask the user to provide them**.
- For parameters with `required: false`, you can infer and generate them yourself based on the user's description (e.g., translating and optimizing the user's scene), or simply use empty values/random numbers (e.g., `seed` = random number).
- Never expose underlying node information to the user (do not mention Node IDs); only ask about business parameter names (e.g., prompt, style).
- If multiple workflows match the user prompt across different servers, you may list them acting as candidates, OR simply pick the most relevant one and execute it directly to provide the best user experience.

### Step 2: Parameter Assembly and Interaction

Once you have identified the workflow to use and collected/generated all necessary parameters, you need to assemble them into a compact JSON string.
For example, if the schema exposes `prompt` and `seed`, you need to construct:
`{"prompt": "A beautiful landscape, high quality, masterpiece", "seed": 40128491}`

*If critical parameters are missing, politely ask the user using `notify_user`. For example: "To generate the image you need, would you like a specific person or animal? Do you have an expected visual style?"*

### Step 2.5: Pre-flight Dependency Check (Automatic)

Before executing a workflow, **always** run a dependency check to verify that all required custom nodes and models are available on the ComfyUI server:

```bash
python ./scripts/comfyui_client.py check-deps --workflow <server_id>/<workflow_id>
```

**Return format:**
```json
{
  "status": "success",
  "report": {
    "is_ready": true/false,
    "missing_nodes": [
      {"class_type": "SAMDetectorCombined", "package_name": "ComfyUI Impact Pack", "source_repo": "https://github.com/...", "can_auto_install": true}
    ],
    "missing_models": [
      {"filename": "model.safetensors", "folder": "checkpoints", "loader_node": "CheckpointLoaderSimple"}
    ],
    "summary": "缺少 1 个自定义节点包；缺少 1 个模型文件"
  }
}
```

**Agent behavior:**
1. If `is_ready` is `true` → proceed to Step 3 directly.
2. If `is_ready` is `false` → present the dependency report to the user in a clear, formatted message:
   - List missing custom nodes with package names and whether they can be auto-installed.
   - List missing models with filenames and which folder they belong to.
   - Ask the user: "Do you want me to install the missing custom nodes?"
3. If the user agrees to install, run:
   ```bash
   python ./scripts/comfyui_client.py install-deps --workflow <server_id>/<workflow_id> --repos '["https://github.com/repo1", "https://github.com/repo2"]'
   ```
   This returns installation results for each package. Report the results to the user.
   - If `needs_restart` is `true`, inform the user that ComfyUI needs to restart for changes to take effect.
   - After restart, re-run `check-deps` to confirm everything is resolved, then proceed to Step 3.
4. For missing models: inform the user that models must be downloaded manually, and tell them which folder to place the files in (e.g., "Please download `model.safetensors` and place it in the `checkpoints` folder").

### Step 3: Trigger the Image Generation Task

Once the complete parameters are collected and the dependency check passes, execute the workflow client in a command-line environment (ensure your current working directory is the project root, or navigate to it first).

Pass the full identifier as `<server_id>/<workflow_id>`.

> **Note**: Outer curly braces must be wrapped in single quotes to prevent bash from incorrectly parsing JSON double quotes.

There are two execution modes. Choose based on your environment:

- **Interactive** (chat, messaging, or any context where the user is waiting): use `submit` + `status` so you can send progress updates between polls.
- **Non-interactive** (scripted pipelines, CI, or terminal-only): use the blocking one-shot mode for simplicity.

#### Interactive mode: `submit` + `status`

**Step 3a — Submit the job:**
```bash
python ./scripts/comfyui_client.py submit --workflow <server_id>/<workflow_id> --args '{"key1": "value1", "key2": 123}'
```
Returns immediately: `{"status": "submitted", "run_id": "...", "prompt_id": "..."}`.

After receiving the response, tell the user that image generation has started.

**Step 3b — Poll for progress:**
```bash
python ./scripts/comfyui_client.py status --workflow <server_id>/<workflow_id> --run-id <run_id>
```
Returns immediately with the current state:
- `{"status": "queued", "queue_position": 2, "queue_total": 5}` — waiting in line
- `{"status": "running", "elapsed_ms": 12000}` — ComfyUI is actively generating
- `{"status": "success", "images": ["/path/to/image.png"]}` — done, images downloaded
- `{"status": "error", "error": "..."}` — generation failed

**Polling pattern — this is critical for real-time feedback:**

Each `status` call must be a **separate tool invocation** (i.e., a separate bash command). Do NOT write a shell loop or combine multiple status checks into one command. The correct pattern is:

1. Run `status` as a standalone bash command.
2. Read the returned JSON.
3. If `"queued"` or `"running"`: **send a text message to the user** with the current progress (e.g., "Queued at position 2…", "Generating, 12 seconds elapsed…"), then run `status` again as another standalone bash command.
4. If `"success"`: proceed to Step 4.
5. If `"error"`: report the error.

This ensures each progress update is delivered to the user immediately, rather than being batched at the end.

#### Non-interactive mode: one-shot blocking

```bash
python ./scripts/comfyui_client.py --workflow <server_id>/<workflow_id> --args '{"key1": "value1", "key2": 123}'
```
Blocks until ComfyUI finishes, then returns the full result.

**Result format (both modes)**:
- On success: JSON with `run_id`, `prompt_id`, and an `images` list of absolute local file paths.
- On error: JSON with `run_id` (if created) and `error` message.

The manager stores execution history per workflow, including raw args, resolved args, prompt ID, result files, status, timing, and error summary. History records live under `data/<server_id>/<workflow_id>/history/`.

### Step 4: Send the Image to the User

Once you obtain the absolute local path to the generated image, use your native capabilities to present the file to the user (e.g., in an OpenClaw environment, returning the path allows the client to intercept it and convert it into rich text or an image preview).

## Common Troubleshooting & Notices
1. **ComfyUI Offline**: If the script returns "Error connecting to ComfyUI", run a server status check and ask the user to start the ComfyUI service for that server URL before retrying.
2. **Schema Not Found**: If you directly called a workflow the user mentioned verbally, but the script reports a missing Schema, perform Step 1 `registry.py` and tell the user they need to first go to the Web UI panel to upload and configure the mapping for that workflow on the desired server.
3. **Parameter Format Error**: Ensure that the JSON passed via `--args` is a valid JSON string wrapped in single quotes.

登录后下载/点赞/收藏 ❤ 145 | ★ 0

comfyui-skill-openclaw

目录结构

SKILL.md

举报内容

提示