atlan-python

Category: Development & Coding | Uploader: atlanhq | Downloads: 0 | Version: v1.0（Latest）
Generate pyatlan_v9 msgspec model files by cloning the models repo and running the Pkl code generator
Changelog: Source: GitHub https://github.com/atlanhq/atlan-python
Directory Structure

Current level: tree/main/.claude/skills/generate-v9-models/
📄 SKILL.md 5.1 KB
SKILL.md

---
description: Generate pyatlan_v9 msgspec model files by cloning the models repo and running the Pkl code generator
---

# Generate v9 Models

Generates pyatlan_v9 msgspec model files from Pkl type definitions in the atlanhq/models repo.

## Usage

- `/generate-v9-models` — Clone models@master, generate and sync v9 models
- `/generate-v9-models <branch>` — Clone models@<branch> instead of master
- `/generate-v9-models <branch> test` — Also run tests after sync
- `/generate-v9-models test` — Clone models@master and run tests after sync

## Instructions

### 1. Clone or update the models repo

Parse args to determine the branch (default: `master`) and whether to run tests (args contain "test").

The models repo should be cloned as a sibling directory of this repo (atlan-python):

```bash
# Determine paths
SDK_DIR="$(pwd)"  # atlan-python root
MODELS_DIR="$(cd .. && pwd)/models"
BRANCH="master"  # override with first non-"test" arg

if [ -d "$MODELS_DIR" ]; then
  cd "$MODELS_DIR" && git fetch origin && git checkout "$BRANCH" && git pull origin "$BRANCH"
else
  git clone --branch "$BRANCH" --single-branch git@github.com:atlanhq/models.git "$MODELS_DIR"
fi
```

### 2. Run Pkl evaluation

From the models repo root, run the Pkl code generator in **SDK-only mode** (`-p sdkOnly=true`). This generates only Python SDK files — no JSON typedefs, no frontend code, no samples.

Use a temp directory for staging so no generated files land in the models repo:

```bash
cd "$MODELS_DIR"
STAGING_DIR="$(mktemp -d)"
OVERLAYS_PATH="${SDK_DIR}/pyatlan_v9/model/assets/_overlays/"

pkl eval typedefs/*.pkl -m "$STAGING_DIR" -p sdkOnly=true \
  -p targetOutputDir=pyatlan_v9/model/assets/ \
  -p internalPackage=pyatlan_v9.model \
  -p sdkOverlaysBasePath="$OVERLAYS_PATH"
```

- `-p sdkOnly=true` skips JSON typedef generation — only Python SDK files are produced
- `sdkOverlaysBasePath` must be an absolute path — Pkl resolves `read?()` relative to the module file, not CWD
- Output goes to a temp staging directory (nothing written to models repo)

### 3. Selective sync

Copy generated files from the staging dir to the SDK, **excluding** these files that have manual patches or are hand-written:

| File | Reason |
|------|--------|
| `entity.py` | Patched: `_metadata_proxies`, `type_name: Any`, `SaveSemantic` |
| `referenceable.py` | Patched: `InternalKeywordField`, field descriptors, helper exports |
| `atlas_glossary.py` | Patched: GTC anchor-in-attributes handling |
| `atlas_glossary_term.py` | Patched: GTC anchor-in-attributes handling |
| `atlas_glossary_category.py` | Patched: GTC anchor-in-attributes handling |
| `quick_sight_dataset.py` | Patched: `useLocalTypeAsPrefix` field naming |
| `quick_sight_dataset_field.py` | Patched: `useLocalTypeAsPrefix` field naming |
| `quick_sight_folder.py` | Patched: `useLocalTypeAsPrefix` field naming |
| `data_quality_rule.py` | Hand-written, not yet generated correctly |

```bash
rsync -av \
  --exclude='entity.py' \
  --exclude='referenceable.py' \
  --exclude='atlas_glossary.py' \
  --exclude='atlas_glossary_term.py' \
  --exclude='atlas_glossary_category.py' \
  --exclude='quick_sight_dataset.py' \
  --exclude='quick_sight_dataset_field.py' \
  --exclude='quick_sight_folder.py' \
  --exclude='data_quality_rule.py' \
  "${STAGING_DIR}/pyatlan_v9/model/assets/" \
  "${SDK_DIR}/pyatlan_v9/model/assets/"

# Clean up staging dir
rm -rf "$STAGING_DIR"
```

16 additional types in pyatlan_v9 (persona.py, purpose.py, badge.py, access_control.py, etc.) are hand-written and NOT generated by Pkl — rsync won't touch them since they don't exist in staging.

### 4. Post-sync patches

**related_entity.py** — ensure `relationship_attributes` field exists after `unique_attributes`. This file is not generated in sdkOnly mode, so it persists across regens. If starting from scratch, add:
```python
    # Relationship-specific attributes
    relationship_attributes: Union[dict[str, Any], None, UnsetType] = UNSET
    """Attributes of the relationship itself (e.g., description, status, etc.)."""
```

### 5. Run ruff auto-fix and format

After syncing and patching, run ruff to fix unused imports and format the generated files:

```bash
cd "${SDK_DIR}"
uv run ruff check --fix --select F401,F811 pyatlan_v9/
uv run ruff format pyatlan_v9/
```

### 6. Run tests (if args contain "test")

```bash
cd "${SDK_DIR}" && python -m pytest tests_v9/unit/ -x -q
```

### 7. Report summary

Report: how many files were generated, how many synced, how many excluded, and test results if applicable.

## Notes

- The models repo is cloned from `git@github.com:atlanhq/models.git`
- If `../models` already exists, it fetches and checks out the requested branch instead of re-cloning
- `-p sdkOnly=true` ensures only Python SDK files are generated (no JSON typedefs written to models repo)
- Generated files go to a temp staging dir, then are selectively synced to `atlan-python/pyatlan_v9/model/assets/`
- Fields with `useSetType=true` in the Pkl typedefs generate `set[str]` instead of `list[str]` (used for user/group/role fields)
- Overlay files (custom methods like `creator()`, `updater()`, policy helpers) live at `pyatlan_v9/model/assets/_overlays/` in this repo
Comments 0
Latest | Hot
Please login before commenting.
Loading comments...
atlan-python

Directory Structure

SKILL.md

Report

Notice
