graphsignal-sdk

Category: Tools & Productivity | Uploader: graphsignal | Downloads: 0 | Version: v1.0（Latest）

Changelog: Source: GitHub https://github.com/graphsignal/graphsignal-python

Directory Structure

Current level: Root

📁 .github/
- 📁 workflows/
  - 📄 daily-tests.yml 760 B
  - 📄 pypi-publish.yml 936 B
📁 graphsignal/
- 📁 _native/
  - 📁 amd64-cu12/
    
    📄 libgscuptiprof.so 71.1 KB
  - 📁 amd64-cu13/
    
    📄 libgscuptiprof.so 71.1 KB
  - 📁 arm64-cu12/
    
    📄 libgscuptiprof.so 64.9 KB
  - 📁 arm64-cu13/
    
    📄 libgscuptiprof.so 64.9 KB
  - 📄 __init__.py 124 B
- 📁 bootstrap/
  - 📄 __init__.py 0 B
  - 📄 sitecustomize.py 740 B
  - 📄 utils.py 309 B
- 📁 commands/
  - 📄 __init__.py 0 B
  - 📄 graphsignal_run.py 2.1 KB
- 📁 core/
  - 📄 __init__.py 0 B
  - 📄 config_loader.py 3.4 KB
  - 📄 sampler.py 992 B
  - 📄 signal_uploader.py 3.6 KB
  - 📄 ticker.py 21.4 KB
- 📁 otel/
  - 📄 __init__.py 0 B
  - 📄 otel_adapter.py 3.1 KB
  - 📄 otel_collector.py 6.3 KB
  - 📄 prometheus_adapter.py 5.2 KB
- 📁 profilers/
  - 📄 __init__.py 0 B
  - 📄 cupti_profiler.py 20.1 KB
  - 📄 function_profiler.py 9.8 KB
- 📁 proto/
  - 📄 __init__.py 0 B
  - 📄 signals_pb2.py 8.1 KB
- 📁 recorders/
  - 📄 __init__.py 0 B
  - 📄 base_recorder.py 278 B
  - 📄 exception_recorder.py 6.5 KB
  - 📄 instrumentation.py 8.9 KB
  - 📄 nvml_recorder.py 49.1 KB
  - 📄 process_recorder.py 10.0 KB
  - 📄 pytorch_recorder.py 8.3 KB
  - 📄 sglang_recorder.py 13.3 KB
  - 📄 vllm_recorder.py 24.2 KB
- 📁 signals/
  - 📄 __init__.py 0 B
  - 📄 logs.py 3.0 KB
  - 📄 metrics.py 7.1 KB
  - 📄 spans.py 17.0 KB
- 📄 __init__.py 4.2 KB
- 📄 __main__.py 784 B
- 📄 env_vars.py 1.9 KB
- 📄 utils.py 929 B
- 📄 version.py 23 B
📁 src/
- 📁 cupti/
  - 📄 cupti_activity.cpp 10.5 KB
  - 📄 cupti_activity.h 621 B
  - 📄 cupti_profiler.cpp 3.2 KB
  - 📄 debug_print.cpp 3.0 KB
  - 📄 debug_print.h 784 B
  - 📄 event_buckets.cpp 7.1 KB
  - 📄 event_buckets.h 1.4 KB
- 📁 test/
  - 📄 cupti_activity_test.cpp 6.9 KB
  - 📄 event_buckets_test.cpp 3.9 KB
📁 test/
- 📁 core/
  - 📄 __init__.py 2 B
  - 📄 test_config_loader.py 7.1 KB
  - 📄 test_sampler.py 627 B
  - 📄 test_signal_uploader.py 4.0 KB
  - 📄 test_ticker.py 3.1 KB
- 📁 otel/
  - 📄 __init__.py 0 B
  - 📄 test_otel_collector.py 5.4 KB
  - 📄 test_prometheus_adapter.py 4.8 KB
- 📁 profilers/
  - 📄 __init__.py 0 B
  - 📄 test_cupti_profiler.py 7.6 KB
  - 📄 test_function_profiler.py 21.3 KB
- 📁 recorders/
  - 📄 __init__.py 0 B
  - 📄 test_exception_recorder.py 13.3 KB
  - 📄 test_instrumentation.py 7.6 KB
  - 📄 test_nvml_recorder.py 9.1 KB
  - 📄 test_process_recorder.py 2.1 KB
  - 📄 test_pytorch_recorder.py 3.5 KB
  - 📄 test_sglang_recorder.py 14.2 KB
  - 📄 test_vllm_recorder.py 11.9 KB
- 📁 signals/
  - 📄 __init__.py 0 B
  - 📄 test_logs.py 2.1 KB
  - 📄 test_metrics.py 22.3 KB
  - 📄 test_spans.py 7.0 KB
- 📄 __init__.py 0 B
- 📄 http_server.py 2.6 KB
- 📄 test_env_vars.py 2.0 KB
- 📄 test_graphsignal.py 5.8 KB
- 📄 test_graphsignal_async.py 2.4 KB
- 📄 test_utils.py 953 B
📁 tools/
- 📄 __init__.py 48 B
- 📄 build-protobuf.sh 137 B
- 📄 build_protobuf.py 137 B
- 📄 init-novenv.sh 98 B
- 📄 run-vllm-tests.sh 233 B
- 📄 run_tests.py 257 B
📄 .gitignore 202 B
📄 Dockerfile.cupti 1.5 KB
📄 LICENSE 11.1 KB
📄 Makefile 9.6 KB
📄 MANIFEST.in 163 B
📄 NOTICE 133 B
📄 poetry.lock 103.8 KB
📄 pyproject.toml 2.0 KB
📄 README.md 4.4 KB
📄 SKILL.md 5.9 KB

SKILL.md

---
name: graphsignal-sdk
description: >-
  Set up and integrate Graphsignal inference observability into Python applications,
  vLLM, SGLang, and dstack services. Use when the user wants to add profiling, tracing,
  or monitoring to inference workloads, needs help configuring the Graphsignal SDK,
  or asks about graphsignal-run, CUPTI, or GPU profiling setup.
---

# Graphsignal SDK

Graphsignal captures performance profiles, traces, metrics, and errors for inference workloads. Auto-instrumentation covers vLLM, SGLang, and PyTorch out of the box.

## Install

```bash
pip install -U graphsignal
```

For CUPTI-based GPU profiling on Linux, install the extra matching the CUDA version:

```bash
pip install graphsignal[cu12]   # CUDA 12.x
pip install graphsignal[cu13]   # CUDA 13.x
```

## Configure

### Option A: In Python code

```python
import graphsignal

graphsignal.configure(api_key='my-api-key')
```

All `configure()` args can be set via env vars: `GRAPHSIGNAL_API_KEY`, `GRAPHSIGNAL_DEBUG_MODE`, etc.

`configure()` parameters:

| Arg | Env var | Purpose |
|-----|---------|---------|
| `api_key` | `GRAPHSIGNAL_API_KEY` | API key (required) |
| `api_url` | `GRAPHSIGNAL_API_URL` | On-premise server URL |
| `tags` | `GRAPHSIGNAL_TAG_{KEY}` | Process-level tags |
| `auto_instrument` | `GRAPHSIGNAL_AUTO_INSTRUMENT` | Auto-instrument libraries (default `True`) |
| `debug_mode` | `GRAPHSIGNAL_DEBUG_MODE` | Enable debug logging |

### Option B: graphsignal-run CLI

Wrap any command — no code changes needed:

```bash
export GRAPHSIGNAL_API_KEY="..."
graphsignal-run <my-app>
```

## Integrate with vLLM

Graphsignal automatically instruments vLLM (engine, scheduler, KV cache, attention, output processing, Prometheus metrics).

### In a Python app

```python
import graphsignal
graphsignal.configure(api_key='my-api-key')
# then use vLLM normally
```

### vLLM server via graphsignal-run

```bash
export GRAPHSIGNAL_API_KEY="..."
graphsignal-run vllm serve Qwen/Qwen1.5-7B-Chat --port 8000
```

### vLLM Docker

```bash
docker run --gpus all \
  -p 8000:8000 --ipc=host \
  -e GRAPHSIGNAL_API_KEY=YOUR_API_KEY \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  --entrypoint bash \
  vllm/vllm-openai:latest \
  -lc 'pip install --no-cache-dir graphsignal[cu12] \
       && exec graphsignal-run vllm serve \
           --model Qwen/Qwen2-VL-7B-Instruct \
           --trust-remote-code'
```

## Integrate with SGLang

Graphsignal automatically instruments SGLang (operations, OTEL spans, Prometheus metrics).

### In a Python app

```python
import graphsignal
graphsignal.configure(api_key='my-api-key')
# then use SGLang normally
```

### SGLang server via graphsignal-run

```bash
export GRAPHSIGNAL_API_KEY="..."
graphsignal-run sglang serve \
  --model-path Qwen/Qwen1.5-7B-Chat \
  --port 8000
```

### SGLang Docker

```bash
docker run --gpus all \
  -p 8000:8000 --ipc=host \
  -e GRAPHSIGNAL_API_KEY=YOUR_API_KEY \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  --entrypoint bash \
  your-sglang-image:latest \
  -lc 'pip install --no-cache-dir graphsignal[cu12] \
       && exec graphsignal-run sglang serve \
           --model-path Qwen/Qwen2.5-1.5B-Instruct \
           --port 8000'
```

## Integrate via dstack

dstack runs inference as services. Use `graphsignal-run` around the server command, same as bare-metal.

### dstack service config (SGLang example)

```yaml
type: service
name: deepseek-r1

image: lmsysorg/sglang:latest
env:
  - MODEL_ID=deepseek-ai/DeepSeek-R1-Distill-Llama-8B
  - GRAPHSIGNAL_API_KEY

commands:
  - |
    pip install --no-cache-dir 'graphsignal[cu12]' && \
    graphsignal-run python3 -m sglang.launch_server \
      --model-path $MODEL_ID \
      --port 8000 \
      --trust-remote-code

port: 8000
model: deepseek-ai/DeepSeek-R1-Distill-Llama-8B

resources:
  gpu: 24GB
```

Deploy:

```bash
dstack apply -f service.dstack.yml
```

If your Docker image already includes Graphsignal, skip the `pip install` step. If using the `sglang` CLI instead of `launch_server`, use `graphsignal-run sglang serve ...`.

## Manual Tracing

For code not auto-instrumented, use `trace()` or `@trace_function`:

```python
with graphsignal.trace('inference') as span:
    span.set_counter('prompt_tokens', prompt_tokens)
    span.set_counter('completion_tokens', completion_tokens)

    with span.trace('model-cold-boot') as sub_span:
        ...
```

```python
@graphsignal.trace_function
def my_function():
    ...
```

Record counter metrics within a trace:

```python
with graphsignal.trace('my-function') as span:
    span.inc_counter_metric('call_cost', price_per_call)
```

## Manual Profiling

Profile specific Python functions (3.12+):

```python
graphsignal.profile_function(func=slow_transform, category='transform', event_name='data-transform')
```

Profile by import path (avoids direct import):

```python
graphsignal.profile_function_path(path='myapp.tasks.prepare_data', category='preprocessing')
```

Profile CUDA kernels by pattern (Linux, CUPTI required):

```python
graphsignal.profile_cuda_kernel(kernel_pattern="cublas", event_name="matmul_gemm")
```

## Tags

Tags enable filtering and breakdown of traces, profiles, and metrics.

```python
graphsignal.configure(api_key='my-api-key', tags={'app_version': '1.0'})

graphsignal.set_tag('app_version', '1.0')

graphsignal.set_context_tag('user_id', current_user_id)

with graphsignal.trace('generate', tags=dict(user_id=my_user_id)):
    ...
```

Via CLI:

```bash
env GRAPHSIGNAL_TAG_APP_VERSION="1.0" graphsignal-run <my-app>
```

## Troubleshooting

Enable debug logging: `graphsignal.configure(debug_mode=True)` or `GRAPHSIGNAL_DEBUG_MODE=true`.

Ensure outgoing connections to `https://api.graphsignal.com` are allowed.

## Reference

- Full Python API: https://graphsignal.com/docs/reference/python-api/
- vLLM integration: https://graphsignal.com/docs/integrations/vllm/
- SGLang integration: https://graphsignal.com/docs/integrations/sglang/
- dstack integration: https://graphsignal.com/docs/integrations/dstack/

Comments 0

Latest | Hot

Please login before commenting.

No comments yet. Be the first one!

graphsignal-sdk

Directory Structure

SKILL.md

Report

Notice