comfyui-skill-openclaw

分类: 开发与编程 | 上传者: HuangYuChuh | 下载: 0 | 版本: v1.0（最新）

Generate images utilizing ComfyUI's powerful node-based workflow capabilities. Supports dynamically loading multiple pre-configured generation workflows from different instances and their corresponding parameter mappings, converting natural language into parameters, driving local or remote ComfyUI services, and ultimately returning the images to the target client. **Use this Skill when:** (1) The user requests to "generate an image", "draw a picture", or "execute a ComfyUI workflow". (2) The user has specific stylistic, character, or scene requirements for image generation. --- # ComfyUI Agent SKILL ## Core Execution Specification As an OpenClaw Agent equipped with the ComfyUI skill, your objective is to translate the user's conversational requests into strict, structured parameters and hand them over to the underlying Python scripts to execute workflows across multi-server environments. ### Step 0: AI-Native Workflow Auto-Configuration (Optional) If the user provides you with a new ComfyUI workflow JSON (API format) and asks you to "configure it" or "add it": 1. Check the existing server configurations or default to `local`. 2. Save the provided JSON file to `./data/<server_id>/workflows/<new_workflow_id>.json`. 3. Analyze the JSON structure (look for `inputs` inside node definitions, e.g., `KSampler`'s `seed`, `CLIPTextEncode`'s `text` for positive/negative prompts, `EmptyLatentImage` for width/height). 4. Automatically generate a schema mapping file and save it to `./data/<server_id>/schemas/<new_workflow_id>.json`. The schema format must follow: ```json { "workflow_id": "<new_workflow_id>", "server_id": "<server_id>", "description": "Auto-configured by OpenClaw", "enabled": true, "parameters": { "prompt": { "node_id": "3", "field": "text", "required": true, "type": "string", "description": "Positive prompt" } // Add other sensible parameters that the user might want to tweak } } ``` 5. Tell the user that the new workflow on the specific server is successfully co

更新日志: Source: GitHub https://github.com/HuangYuChuh/ComfyUI_Skills_OpenClaw

目录结构

当前层级: 根目录

📁 asset/
- 📄 banner-ui-20250309.jpg 708.2 KB
- 📄 logo.png 898.4 KB
📁 data/
- 📄 .gitkeep 0 B
📁 docs/
- 📄 llms-full.txt 2.2 KB
- 📄 llms.txt 1.1 KB
- 📄 PROJECT_DISCOVERY_CHECKLIST.md 2.8 KB
📁 outputs/
- 📄 .gitkeep 0 B
📁 scripts/
- 📁 shared/
  - 📄 __init__.py 38 B
  - 📄 config.py 1.1 KB
  - 📄 json_utils.py 495 B
  - 📄 runtime_config.py 2.7 KB
- 📄 comfyui_client.py 7.1 KB
- 📄 registry.py 4.9 KB
- 📄 server_manager.py 5.0 KB
📁 ui/
- 📁 static/
  - 📁 js/
    
    📄 api.js 1.8 KB
    
    📄 app.js 37.8 KB
    
    📄 custom-select.js 8.7 KB
    
    📄 dom.js 3.0 KB
    
    📄 i18n.js 25.9 KB
    
    📄 mapping-editor-view.js 9.1 KB
    
    📄 pixel-blast-bg.js 6.0 KB
    
    📄 state.js 2.9 KB
    
    📄 toast.js 2.5 KB
    
    📄 ui-utils.js 685 B
    
    📄 workflow-list-view.js 3.8 KB
    
    📄 workflow-mapper.js 5.9 KB
  - 📁 vendor/
    
    📄 jquery-3.7.1.min.js 85.5 KB
  - 📄 index.html 15.7 KB
  - 📄 logo.png 898.4 KB
  - 📄 styles.css 33.0 KB
- 📄 app.py 6.8 KB
- 📄 json_store.py 522 B
- 📄 models.py 4.0 KB
- 📄 run_ui.bat 410 B
- 📄 run_ui.command 85 B
- 📄 run_ui.sh 475 B
- 📄 services.py 10.6 KB
- 📄 settings.py 1.3 KB
📄 .gitignore 582 B
📄 config.example.json 244 B
📄 LICENSE 1.0 KB
📄 README.md 7.5 KB
📄 README.zh.md 7.1 KB
📄 requirements.txt 66 B
📄 SKILL.md 5.7 KB

SKILL.md

---
name: comfyui-skill-openclaw
description: |
  Generate images utilizing ComfyUI's powerful node-based workflow capabilities. Supports dynamically loading multiple pre-configured generation workflows from different instances and their corresponding parameter mappings, converting natural language into parameters, driving local or remote ComfyUI services, and ultimately returning the images to the target client.
  
  **Use this Skill when:**
  (1) The user requests to "generate an image", "draw a picture", or "execute a ComfyUI workflow".
  (2) The user has specific stylistic, character, or scene requirements for image generation.
---

# ComfyUI Agent SKILL

## Core Execution Specification

As an OpenClaw Agent equipped with the ComfyUI skill, your objective is to translate the user's conversational requests into strict, structured parameters and hand them over to the underlying Python scripts to execute workflows across multi-server environments.

### Step 0: AI-Native Workflow Auto-Configuration (Optional)

If the user provides you with a new ComfyUI workflow JSON (API format) and asks you to "configure it" or "add it":
1. Check the existing server configurations or default to `local`.
2. Save the provided JSON file to `./data/<server_id>/workflows/<new_workflow_id>.json`.
3. Analyze the JSON structure (look for `inputs` inside node definitions, e.g., `KSampler`'s `seed`, `CLIPTextEncode`'s `text` for positive/negative prompts, `EmptyLatentImage` for width/height).
4. Automatically generate a schema mapping file and save it to `./data/<server_id>/schemas/<new_workflow_id>.json`. The schema format must follow:
   ```json
   {
     "workflow_id": "<new_workflow_id>",
     "server_id": "<server_id>",
     "description": "Auto-configured by OpenClaw",
     "enabled": true,
     "parameters": {
       "prompt": { "node_id": "3", "field": "text", "required": true, "type": "string", "description": "Positive prompt" }
       // Add other sensible parameters that the user might want to tweak
     }
   }
   ```
5. Tell the user that the new workflow on the specific server is successfully configured and ready to be used.

### Step 1: Query Available Workflows (Registry)

Before attempting to generate any image, you must **first query the registry** to understand which workflows are currently supported and enabled:
```bash
python ./scripts/registry.py list --agent
```

**Return Format Parsing**:
You will receive a JSON containing all available workflows. Notice they are uniquely identified by a combination of `server_id` and `id` (or path format `<server_id>/<workflow_id>`):
- For parameters with `required: true`, if the user hasn't provided them, you must **ask the user to provide them**.
- For parameters with `required: false`, you can infer and generate them yourself based on the user's description (e.g., translating and optimizing the user's scene), or simply use empty values/random numbers (e.g., `seed` = random number).
- Never expose underlying node information to the user (do not mention Node IDs); only ask about business parameter names (e.g., prompt, style).
- If multiple workflows match the user prompt across different servers, you may list them acting as candidates, OR simply pick the most relevant one and execute it directly to provide the best user experience.

### Step 2: Parameter Assembly and Interaction

Once you have identified the workflow to use and collected/generated all necessary parameters, you need to assemble them into a compact JSON string.
For example, if the schema exposes `prompt` and `seed`, you need to construct:
`{"prompt": "A beautiful landscape, high quality, masterpiece", "seed": 40128491}`

*If critical parameters are missing, politely ask the user using `notify_user`. For example: "To generate the image you need, would you like a specific person or animal? Do you have an expected visual style?"*

### Step 3: Trigger the Image Generation Task

Once the complete parameters are collected, execute the workflow client in a command-line environment (ensure your current working directory is the project root, or navigate to it first).

Pass the full identifier as `<server_id>/<workflow_id>`.

> **Note**: Outer curly braces must be wrapped in single quotes to prevent bash from incorrectly parsing JSON double quotes.

```bash
python ./scripts/comfyui_client.py --workflow <server_id>/<workflow_id> --args '{"key1": "value1", "key2": 123}'
```

**Blocking and Result Retrieval**:
- This script will automatically submit the task to the matched server and **poll to wait** for ComfyUI to finish rendering, then download the image locally.
- If executed successfully, the standard output of the script will finally provide a JSON containing an `images` list, where the absolute paths are the generated image files.

### Step 4: Send the Image to the User

Once you obtain the absolute local path to the generated image, use your native capabilities to present the file to the user (e.g., in an OpenClaw environment, returning the path allows the client to intercept it and convert it into rich text or an image preview).

## Common Troubleshooting & Notices
1. **ComfyUI Offline**: If the script returns "Error connecting to ComfyUI", remind the user to check if the ComfyUI service is running on the specific server, or go to the Web UI panel (start with `python3 ui/app.py` or `./run_ui.sh`) to adjust the server configuration URLs.
2. **Schema Not Found**: If you directly called a workflow the user mentioned verbally, but the script reports a missing Schema, perform Step 1 `registry.py` and tell the user they need to first go to the Web UI panel to upload and configure the mapping for that workflow on the desired server.
3. **Parameter Format Error**: Ensure that the JSON passed via `--args` is a valid JSON string wrapped in single quotes.

登录后下载/点赞/收藏 ❤ 7 | ★ 0

comfyui-skill-openclaw

目录结构

SKILL.md

举报内容

提示