datarobot-agent-assist

Category: Tools & Productivity | Uploader: datarobot-oss | Downloads: 0 | Version: v1.0（Latest）
Changelog: Source: GitHub https://github.com/datarobot-oss/datarobot-agent-skills
Directory Structure

Current level: tree/main/skills/datarobot-agent-assist/
📁 references/
- 📄 agent-spec-examples.md 4.0 KB
📁 scripts/
- 📄 clone_template.py 6.5 KB
- 📄 env_utils.py 2.7 KB
- 📄 list_llm_models.py 6.8 KB
- 📄 rehearsal.py 20.2 KB
- 📄 select_framework.py 2.4 KB
- 📄 setup_template.py 8.9 KB
📄 SKILL.md 16.9 KB
SKILL.md

---
name: datarobot-agent-assist
description: >-
  Unified DataRobot agent workflow — design (agent_spec.md), optional dress-rehearsal simulation
  via built-in rehearsal engine, template-based coding, and deployment. Combines agent design guidance
  with interactive pre-code simulation (DataRobot LLM Gateway, feedback report). Use when the user
  wants to design, build, code, or deploy an AI agent for DataRobot, run a spec simulation before
  coding, mentions "agent spec", "datarobot-agent-assist", "dr-assist", "dress rehearsal", or is creating agents on DataRobot's
  platform.
---

# DataRobot Agent Assist

This skill merges **agent design, coding, and deployment** with **interactive dress-rehearsal simulation** in one place.

Assistance falls into three categories:

1. **Designing an AI agent** → Clarify requirements, build `agent_spec.md`, optionally simulate the agent before coding
2. **Coding an AI agent** → Adapt the DataRobot agent application template to the spec
3. **Deploying an AI agent** → Follow `AGENTS.md` deployment instructions

If the user's first message is simply `1`, `2`, or `3`, treat it as selecting one of these categories.

---

## On Activation

Present the three options clearly:

```
Welcome! I help you design, code, and deploy AI agents (with optional dress-rehearsal simulation before coding).

What would you like to do?
  1. Design an AI agent     → Describe your idea
  2. Code an AI agent       → Load and implement an existing agent_spec.md
  3. Deploy an AI agent     → Deploy an implemented agent to DataRobot
```

After the user selects an option, and before proceeding, run through the **[Pre-requisite Check](#pre-requisite-check)** section below.

---

## Pre-requisite Check

Run in order before proceeding:

1. **Git** — run `git --version`. If missing, tell the user to install from https://git-scm.com and stop.
2. **Python** — run `python --version`. If missing or below 3.11, tell the user to install Python 3.11+ from https://python.org and stop.
3. **DataRobot CLI** — follow **DataRobot CLI Setup** at the bottom:
   - If missing, **ALWAYS RUN** the install command before proceeding
   - **ALWAYS RUN** the upgrade command before proceeding
   - If not authenticated, **ALWAYS RUN** the auth command before proceeding

---

## 1. Designing an AI Agent

### Clarification Phase

- Ask **at most 2 rounds** of clarifying questions before proposing an initial draft spec. If tools are still ambiguous after two rounds, start simple.
- Focus questions on:
  - What the agent does and who uses it
  - What tools it needs and what external services those tools call
  - Whether those services require authentication (API key, OAuth2, bearer token, etc.)
  - Whether the user needs a custom frontend beyond the default chat UI

- If the user mentions UI-related needs early ("dashboard", "visualization", "multi-page", "admin panel", "settings page"), capture it immediately in the `frontend` field — do **not** defer.

### Model Selection

- To check available models: Run the helper script:
   ```
   python <skill_scripts_dir>/list_llm_models.py \
     --json
   ```

  **CRITICAL**: In case the script fails due to any reason, do **not** proceed. Instead, return the error message to the user and ask how they want to proceed.

- Recommend a `gpt-5`, `claude-4-5`, or `gemini-2.5` model from the list unless the user specifies cost or other constraints.
- Only display the full model catalog when the user **explicitly** asks to browse models.
- If the user's desired model is unavailable, suggest starting with an available one and updating after implementation.

### Spec Display

- **Always write the current spec to `agent_spec.md`** (YAML format) whenever showing it to the user.
- Show the spec frequently and iteratively — even if incomplete or partial.
- Do **not** summarize the spec in prose; display it as YAML in a code block.
- After displaying, invite the user to refine system prompts, add/modify tools, change the model, or update examples.

### Frontend Check (Mandatory Before Coding or Simulating)

Before offering to simulate or code, if the spec does not already have a `frontend` field set, **always ask**:

> "The template includes a default chat UI — is that sufficient, or would you like a custom frontend such as a dashboard, data visualization, or multi-page app?"

Then update the spec accordingly:
- Default UI → `frontend.type: "chat"`
- Custom UI → `frontend.type: "multi-page"` or `"custom"` with `pages` and optional `requirements`

### Agent Simulation (Before Coding)

Always offer to simulate the agent before coding it. **Run simulation** by following [Dress Rehearsal](#dress-rehearsal) in this skill end to end: initialize with `rehearsal.py --init`, drive turns with `--session`, handle `NOTE:` / `DONE`, and produce the dress rehearsal feedback report. Do not substitute improvised role-play or manual mock tool traces for this flow.

Script path (from project root):

`python <skill_scripts_dir>/rehearsal.py ...`

---

## Dress Rehearsal

Simulate an `agent_spec.md` interactively before writing any code. Responses go through the DataRobot LLM Gateway; the rehearsal script handles API calls, state, and output. You orchestrate the loop, handle out-of-character commands, and produce the feedback report at the end.

**Engine location:** `<skill_scripts_dir>/rehearsal.py` (relative to repository root).

### Step 1 — Initialize the session

```bash
python <skill_scripts_dir>/rehearsal.py --init [--spec agent_spec.md]
```

If `agent_spec.md` does not exist and no path was provided, say so and stop.

The script creates a unique session directory in the system temp dir and prints two lines:
```
session=<session_dir>
output=<output_file>
```

Retain `session_dir` for all subsequent calls. Read the `output_file` and display its contents verbatim, then say:

> You are now the **end user** of this agent. Type messages as a real user would.
>
> **Out-of-character commands:**
> - `NOTE: <text>` — record a design observation
> - `DONE` — end the session and generate your feedback report

### Step 2 — Simulation loop

Keep track of any notes and the number of turns as the session progresses — you'll need these for the report.

**On each user message:**

- If it starts with `NOTE:` — acknowledge the note, prompt for next message. Do not call the script.
- If it is `DONE` — proceed to Step 3.
- Otherwise — run the turn:

```bash
python <skill_scripts_dir>/rehearsal.py --session {session_dir} "{user_message}"
```

The script prints `output=<output_file>`. Read that file and display its contents verbatim. It will contain `[TOOL CALL]`, `[SIMULATED RETURN]`, and `[Agent]:` blocks as appropriate.

If the script exits non-zero, display the error and ask whether to continue or abort.

### Step 3 — Feedback report

Before writing the report, review the session and consider each of these areas — only surface the ones where you have something concrete to say:

- **System prompt** — wording, missing constraints, persona, tone
- **Tools** — input/output scoping, missing or redundant tools, argument naming
- **Model** — only flag if clearly wrong for the observed task complexity
- **Example prompts** — additions or revisions based on what was tested
- **Other** — edge cases, UX concerns, data dependency risks

Then write the report in this format:

```
════════════════════════════════════════════
  DRESS REHEARSAL REPORT
════════════════════════════════════════════

{1–2 sentences: what was tested and how the agent performed overall}
{If notes were recorded: "Notes: " followed by each note on its own line, prefixed with —}

Suggested changes:
1. {specific, actionable change}
2. {specific, actionable change}
…
{If nothing worth changing: "No changes recommended."}

════════════════════════════════════════════
```

Then offer to implement any changes to `agent_spec.md`.

---

## 2. Coding an AI Agent

**On Windows: coding is not supported. STOP and do NOT proceed with the next steps!**

### Pre-coding Checklist

1. Check if `agent_spec.md` exists — if so, **read it first**
2. Check if `AGENTS.md` exists in the template directory (default: current working directory)
3. If `AGENTS.md` does **not** exist, prepare the template with these steps in order. ALWAYS follow the steps in order and do not skip any, even if they seem redundant. This is critical for ensuring the template is properly set up and avoiding wasted effort coding on a broken foundation.
   a. **Check the working directory** — if it contains files other than `agent_spec.md`, warn the user and ask them to clear it before proceeding
   b. **Clone the template**: Run the helper script:
   ```
   python <skill_scripts_dir>/clone_template.py
   ```
   c. **Select the agentic framework**:

   **STOP. Do NOT proceed until the user has replied with their framework choice.**

   Ask the user (exact message):
   > Which agentic framework would you like to use?
   > 1. LangGraph
   > 2. CrewAI
   > 3. LlamaIndex
   > 4. NeMo Agent Toolkit (NAT)
   > 5. Base

   Wait for the user's reply. Do not assume or default to any framework. Once the user replies, map their choice to the corresponding value (`langgraph`, `crewai`, `llamaindex`, `nat`, `base`) and run:
   ```
   python <skill_scripts_dir>/select_framework.py \
     --target-dir . \
     --framework <value>
   ```

   d. **Validate the template**: Run `dr dependency check`. If it fails, return the error message to the user DO NOT proceed. If it succeeds, continue to the next step.
   e. **Setup the template**: Run the helper script:
   ```
   python <skill_scripts_dir>/setup_template.py \
     --llm-model <model-name> \
     --target-dir .
   ```

   **CRITICAL**: In case any of the above scripts fail due to any reason, do **not** proceed with coding. Instead, return the error message to the user and ask how they want to proceed.

   f. **Re-read `AGENTS.md`** now that the template is ready
4. Recreate the TODO list based on the `agent_spec.md` — break down the implementation into discrete steps and add them to the TodoWrite tool


### Coding Rules

- Implement by adapting the template code — do not write from scratch
- Modify files only inside the current directory and its subdirectories
- Do not view `.env` files (`.env.template` files are OK)
- Do not add code comments unless asked
- Do not mock tool implementations unless they would be complex to implement
- For tasks with 3+ steps, use the TodoWrite tool to manage your work
- Keep text responses **concise (1–3 sentences)** while coding — skip preamble and postamble

### File Write/Edit Discipline

- Always explain **why** the change is needed (purpose and impact) in 1–2 sentences before writing or editing a file
- Invoke at most **one shell command per response** — wait for the result before invoking another

### After Coding

1. **Proactively test the agent code locally** — read `AGENTS.md` and follow its local testing instructions to validate the implementation. Attempt this before suggesting next steps.

2. **If the user asks for help testing locally**:
   - Read `AGENTS.md` to find the exact shell command(s) required to run the agent locally
   - Display the command(s) in a code block
   - Tell the user to run the command in a **new terminal** in the current directory
   - Do **not** run the command yourself

3. After completing code generation and testing, present next steps:
   - Testing the agent locally
   - Revising aspects of the implementation
   - Deploying to DataRobot

---

## 3. Deploying an AI Agent

- Read `AGENTS.md` for deployment instructions
- Follow the instructions **strictly**
- Do not deviate without user confirmation

---

## Helper Scripts

The following are the examples of helper scripts used in the skill. They are located in the `scripts` directory and are designed to assist with various tasks.

### list_llm_models.py

Lists available LLM models from DataRobot LLM Gateway.

Fetches and displays active models from the DataRobot LLM Gateway catalog:
```bash
python <scripts_dir>/list_llm_models.py \
  --json
```

Requires env vars: `DATAROBOT_API_TOKEN`, `DATAROBOT_ENDPOINT`

### clone_template.py

Clones the DataRobot agent application template repository.

Clones the template to the current directory (repository URL and branch are hardcoded):
```bash
python <scripts_dir>/clone_template.py
```

Clone to a specific directory:
```bash
python <scripts_dir>/clone_template.py \
  --target-dir ./my-project
```

### setup_template.py

Sets up a template repository for initializing a new agent project.

```bash
python <scripts_dir>/setup_template.py \
  --llm-model <model-name> \
  --target-dir .
```

### select_framework.py

Saves the chosen agentic framework to `.datarobot/answers/agent-agent.yml`
(field `agent_template_framework`). Preserves all other fields in the file.

```bash
python <scripts_dir>/select_framework.py \
  --framework langgraph \
  --target-dir .
```

Valid `--framework` values: `langgraph`, `crewai`, `llamaindex`, `nat`, `base`


## Error Handling

- If a tool returns an error, read the error message carefully before responding
- For template-prep **warnings**: try to resolve yourself
- For template-prep **errors**: return the message to the user and ask how to proceed
- On unexpected errors, ask the user if they want to retry

---

## agent_spec.md Schema

Write specs in YAML to `agent_spec.md` in the working directory. Fields are optional when the spec is still evolving.

```yaml
model: "anthropic/claude-sonnet-4-5-20250929"   # DataRobot LLM Gateway model ID
system_prompt: "Your agent's instructions..."
tools:
  - function_name: tool_name
    inputs:
      - arg_name: input_arg
        type: str         # one of: str, int, float, bool, list, dict
        object_schema: "(optional: schema of dict/list contents)"
    out:
      - arg_name: output_arg
        type: str
    auth_spec:
      service_name: "External API Service"
      auth_method: api_key   # api_key | oauth2 | basic_auth | bearer_token | service_account | other
examples:
  - "Example user query 1"
  - "Example user query 2"
frontend:
  type: "chat"              # chat | multi-page | custom
  pages:
    - "Analytics - shows search history and top topics"
  requirements: "(optional additional UI requirements)"
```

When tools require external service auth, note that credentials must be configured as **runtime parameters** in the infrastructure code (see `AGENTS.md` for the pattern).

See [references/agent-spec-examples.md](references/agent-spec-examples.md) for complete working examples.

---

## Tool/Helper Scripts Timeouts

- Allow up to 10 minutes for any helper script to complete before timing out and returning an error
- Allow up to 5 minutes for any tool to return a response before timing out and returning an error
- Allow up to 30 minutes for deployment-related shell commands to complete before timing out and returning an error

---


## Tool Mapping

Claude's built-in tools replace the plugin's custom Python tools:

| Plugin Tool | Claude Tool |
|---|---|
| `read_file` | Read |
| `write_file` | Write |
| `edit_file` | Edit |
| `shell` | Bash |
| `list_dir` | Glob or Bash (`ls`) |
| `grep_files` | Grep |
| `glob` | Glob |
| `web_search` | WebSearch |
| `get_web_page` | WebFetch |
| `write_todos` / `read_todos` | TodoWrite |
| `show_agent_spec` | Write to `agent_spec.md` + display as YAML |
| `prepare_to_code` | Bash (`git clone` + `dr start`) |
| `list_available_models` | WebFetch (DataRobot API) |
| `code_research` | Agent (Explore subagent) |
| Agent simulation (dress rehearsal) | [Dress Rehearsal](#dress-rehearsal) + `<skill_scripts_dir>/rehearsal.py` in this skill directory |

---

## Behavioral Rules

- If it is unclear whether the request falls into one of the three categories, ask a clarifying question
- If the user insists on a task outside these three categories, politely decline
- If a user asks to code before designing, strongly encourage designing first
- During **coding**: keep responses to 1–3 sentences; no introductions or conclusions
- During **design**: be conversational and thorough

---

## DataRobot CLI Setup

The DataRobot CLI (`dr`) is required for managing DataRobot custom applications.

### Verify Installation

Check if the CLI is installed:

```bash
dr --version
```

Expected output: `DataRobot CLI version: v0.2.66` (or similar)

### Install DataRobot CLI

If not installed, run:

**macOS/Linux:**
```bash
curl https://cli.datarobot.com/install | sh
```

**Windows:**
```powershell
irm https://cli.datarobot.com/winstall | iex
```

### Upgrade CLI

If the CLI version is too old, run to upgrade:

```bash
dr self update --force
```

### Check Authentication Status

Verify the CLI is authenticated:

```bash
dr auth check
```

### Authenticate

If not authenticated, run:

```bash
dr auth login
```

This will guide the user through the authentication process interactively.
=======
Comments 0
Latest | Hot
Please login before commenting.
Loading comments...
datarobot-agent-assist

Directory Structure

SKILL.md

Report

Notice
