ml-leakage-guard

分类: 数据与AI | 上传者: Furinaaa-Cancan | 下载: 0 | 版本: v1.0（最新）

出版级医学预测工作流程，具有严格的防数据泄漏控制、表型定义保护、基于谱系的泄漏检测、分割协议验证、类不平衡策略验证、超参数调整隔离检查、伪造测试和再现性门。在 EHR/索赔/注册数据中构建、审查或调试疾病风险或预后模型时使用，特别是当目标定义、诊断代码、实验室标准、药物、时间窗口和派生特征可能泄漏目标信息时。

更新日志: Source: GitHub https://github.com/Furinaaa-Cancan/medical-ml-leakage-guard

目录结构

当前层级: tree/main/

📁 .github/
- 📁 workflows/
  - 📄 ci-extended.yml 1.7 KB
  - 📄 ci-full.yml 1.8 KB
  - 📄 ci-overnight.yml 1.3 KB
  - 📄 ci-security.yml 4.8 KB
  - 📄 ci-unit.yml 730 B
📁 agents/
- 📄 claude.yaml 1.4 KB
- 📄 gemini.yaml 1.4 KB
- 📄 openai.yaml 1.2 KB
📁 docs/
- 📄 gate-framework-developer-guide.md 9.5 KB
- 📄 gate-scripts-review-summary.md 10.4 KB
- 📄 scripts_deep_review_report.md 6.0 KB
- 📄 security_hardening_skill.md 19.0 KB
📁 examples/
- 📁 medical_ml_demo/
  - 📁 01_exploration/
    
    📁 scripts/
    
    📄 explore.py 5.8 KB
  - 📁 02_splitting/
    
    📁 scripts/
    
    📄 split.py 6.6 KB
  - 📁 03_preprocessing/
    
    📁 results/
    
    📄 column_types.json 998 B
    
    📄 feature_names.json 5.2 KB
    
    📁 scripts/
    
    📄 preprocess.py 16.2 KB
  - 📁 04_feature_selection/
    
    📁 results/
    
    📄 elastic_net_cv.csv 198 B
    
    📄 group_selection_summary.csv 1.2 KB
    
    📄 selected_features.json 4.3 KB
    
    📄 stability_selection.csv 2.9 KB
    
    📁 scripts/
    
    📄 select_features.py 19.2 KB
  - 📁 05_modeling/
    
    📁 results/
    
    📄 admission_features.json 2.5 KB
    
    📄 admission_vs_discharge.csv 445 B
    
    📄 all_tuning_results.csv 4.5 KB
    
    📄 best_model_info.json 613 B
    
    📄 best_per_family.csv 353 B
    
    📄 bootstrap_optimism.csv 247 B
    
    📄 thresholds.json 124 B
    
    📁 scripts/
    
    📄 train_admission_model.py 7.7 KB
    
    📄 train_models.py 17.8 KB
  - 📁 06_evaluation/
    
    📁 results/
    
    📄 calibration_comparison.csv 262 B
    
    📄 calibration_ece.csv 92 B
    
    📄 dca.csv 1.4 KB
    
    📄 multi_seed_stability.csv 130 B
    
    📄 test_metrics.csv 833 B
    
    📄 test_metrics_ci.csv 984 B
    
    📁 scripts/
    
    📄 calibrate.py 5.4 KB
    
    📄 evaluate.py 17.3 KB
  - 📁 07_interpretability/
    
    📁 results/
    
    📄 individual_cases.json 8.5 KB
    
    📄 LightGBM_importance.csv 4.3 KB
    
    📄 LR_importance.csv 4.2 KB
    
    📄 RF_importance.csv 4.3 KB
    
    📄 XGBoost_importance.csv 4.3 KB
    
    📁 scripts/
    
    📄 interpret.py 7.0 KB
  - 📁 08_fairness/
    
    📁 results/
    
    📄 subgroup_age.csv 648 B
    
    📄 subgroup_gender.csv 202 B
    
    📄 subgroup_race.csv 465 B
    
    📁 scripts/
    
    📄 fairness.py 5.4 KB
  - 📁 09_reporting/
    
    📁 scripts/
    
    📄 report.py 13.3 KB
  - 📁 outputs/
    
    📁 tables/
    
    📄 comprehensive_metrics.csv 698 B
    
    📄 limitations.csv 1.9 KB
    
    📄 table1_cohort.csv 695 B
    
    📄 table2_performance.csv 810 B
    
    📄 table3_temporal_comparison.csv 445 B
    
    📄 tripod_ai_checklist.csv 2.3 KB
  - 📄 .gitignore 123 B
  - 📄 config.py 4.7 KB
  - 📄 README.md 1.3 KB
  - 📄 run_all.py 4.3 KB
- 📄 .gitignore 105 B
- 📄 download_cdc_data.py 15.3 KB
- 📄 download_nci_gdc.py 7.6 KB
- 📄 download_nhanes.py 11.9 KB
- 📄 download_real_data.py 40.4 KB
📁 experiments/
- 📁 authority-e2e/
  - 📄 .gitignore 202 B
  - 📄 run_adversarial_gate_checks.py 97.9 KB
  - 📄 run_authority_e2e.py 160.5 KB
  - 📄 run_release_benchmark_matrix.py 49.8 KB
  - 📄 scan_stress_diabetes_feasibility.py 6.8 KB
- 📁 paper/
  - 📁 benchmark/
    
    📁 snippets/
    
    📄 clean_correct_01_fit_train_transform_test.py 279 B
    
    📄 clean_correct_02_pipeline.py 390 B
    
    📄 clean_correct_03_imputer_train_only.py 271 B
    
    📄 clean_correct_04_pca_after_split.py 252 B
    
    📄 clean_correct_05_minmax_after_split.py 250 B
    
    📄 clean_correct_06_cv_pipeline.py 417 B
    
    📄 clean_correct_07_make_pipeline.py 306 B
    
    📄 clean_correct_08_column_transformer.py 389 B
    
    📄 clean_correct_09_smote_train_only.py 241 B
    
    📄 clean_correct_10_threshold_on_valid.py 245 B
    
    📄 clean_correct_11_robust_scaler.py 248 B
    
    📄 clean_correct_12_kfold_inside_pipeline.py 356 B
    
    📄 clean_correct_13_imblearn_pipeline.py 421 B
    
    📄 clean_correct_14_target_encoder_sklearn.py 259 B
    
    📄 clean_correct_15_gridsearch_cv.py 385 B
    
    📄 clean_correct_16_nested_cv.py 532 B
    
    📄 clean_correct_17_train_valid_test.py 537 B
    
    📄 clean_correct_18_predefined_split.py 279 B
    
    📄 clean_correct_19_iterative_imputer_correct.py 329 B
    
    📄 clean_correct_20_feature_union.py 435 B
    
    📄 clean_legit_01_label_encoder_target.py 218 B
    
    📄 clean_legit_02_roc_curve_eval_only.py 291 B
    
    📄 clean_legit_03_roc_curve_thresh_unused.py 243 B
    
    📄 clean_legit_04_model_fit_is_training.py 249 B
    
    📄 clean_legit_05_xgb_early_stop_valid.py 246 B
    
    📄 clean_legit_06_tsne_visualization.py 164 B
    
    📄 clean_legit_07_dropna_is_row_removal.py 231 B
    
    📄 clean_legit_08_ordinal_encoder_target.py 246 B
    
    📄 clean_legit_09_separate_validation_set.py 400 B
    
    📄 clean_legit_10_feature_importance.py 256 B
    
    📄 clean_legit_11_roc_multiple_models.py 344 B
    
    📄 clean_legit_12_calibration_plot.py 231 B
    
    📄 clean_legit_13_confusion_matrix.py 293 B
    
    📄 clean_legit_14_shap_explanation.py 233 B
    
    📄 clean_legit_15_bootstrap_ci.py 393 B
    
    📄 leak_edge_01_dict_fit_transform_all.py 300 B
    
    📄 leak_edge_02_loop_fit_transform.py 302 B
    
    📄 leak_edge_03_function_hides_leak.py 288 B
    
    📄 leak_edge_04_method_chain.py 212 B
    
    📄 leak_R001_01_scaler_before_split.py 341 B
    
    📄 leak_R001_02_imputer_before_split.py 226 B
    
    📄 leak_R001_03_pca_before_split.py 224 B
    
    📄 leak_R001_04_minmax_before_split.py 247 B
    
    📄 leak_R001_05_robust_scaler.py 239 B
    
    📄 leak_R001_06_power_transformer.py 234 B
    
    📄 leak_R001_07_quantile_transformer.py 250 B
    
    📄 leak_R001_08_tfidf_before_split.py 266 B
    
    📄 leak_R001_09_one_hot_before_split.py 270 B
    
    📄 leak_R001_10_multiple_transforms.py 350 B
    
    📄 leak_R002_01_fit_transform_test.py 279 B
    
    📄 leak_R002_02_imputer_fit_test.py 322 B
    
    📄 leak_R002_03_dict_comprehension.py 304 B
    
    📄 leak_R002_04_separate_fit_test.py 234 B
    
    📄 leak_R002_05_normalizer_test.py 217 B
    
    📄 leak_R002_06_valid_and_test.py 408 B
    
    📄 leak_R002_07_iterative_imputer_test.py 310 B
    
    📄 leak_R002_08_knn_imputer_test.py 230 B
    
    📄 leak_R002_09_poly_features_test.py 241 B
    
    📄 leak_R003_01_smote_full_data.py 226 B
    
    📄 leak_R003_02_smote_on_test.py 222 B
    
    📄 leak_R003_03_adasyn_full.py 222 B
    
    📄 leak_R005_01_youden_on_test.py 277 B
    
    📄 leak_R005_02_pr_curve_threshold.py 327 B
    
    📄 leak_R005_03_threshold_stored.py 280 B
    
    📄 leak_R006_01_selectkbest_full.py 253 B
    
    📄 leak_R006_02_mutual_info_full.py 271 B
    
    📄 leak_R006_03_variance_threshold_full.py 236 B
    
    📄 leak_R007_01_target_in_features.py 292 B
    
    📄 leak_R007_02_no_drop_target.py 267 B
    
    📄 leak_R007_03_same_df.py 236 B
    
    📄 leak_R017_01_eval_set_test.py 242 B
    
    📄 leak_R017_02_lgbm_test_eval.py 273 B
    
    📄 leak_R017_03_catboost_test.py 275 B
    
    📄 leak_R020_01_dropna_before_split.py 226 B
    
    📄 leak_R020_02_fillna_before_split.py 213 B
    
    📄 leak_R020_03_clip_before_split.py 339 B
    
    📄 benchmark_manifest.json 10.7 KB
    
    📄 generate_benchmark.py 28.3 KB
  - 📁 output/
    
    📁 llm_reviews/
    
    📄 aggregate.json 847 B
    
    📄 code_audit_results.json 251.7 KB
    
    📄 code_audit_v2_fixed.json 131.5 KB
    
    📄 code_audit_v3_final.json 141.2 KB
    
    📄 code_audit_v4_post_fix.json 141.8 KB
    
    📄 code_audit_verified_v2.json 132.2 KB
    
    📄 methods_vs_code_comparison.json 26.2 KB
    
    📄 statistical_results.json 4.9 KB
    
    📄 verification_report.json 143.5 KB
    
    📄 verification_report_v2.json 374.3 KB
  - 📄 __init__.py 67 B
  - 📄 blind_audit_list.jsonl 14.2 KB
  - 📄 blind_audit_results.json 1.9 KB
  - 📄 blind_pmcids.txt 579 B
  - 📄 build_leaky_pipeline.py 15.5 KB
  - 📄 collect_papers_with_code.py 13.4 KB
  - 📄 compare_methods_vs_code.py 8.2 KB
  - 📄 manual_audit_log.jsonl 22.8 KB
  - 📄 papers_verified_v2.jsonl 136.4 KB
  - 📄 papers_with_code_merged.jsonl 304.0 KB
  - 📄 review_methods_llm.py 16.0 KB
  - 📄 REVIEW_PROTOCOL.md 16.5 KB
  - 📄 run_deflation_experiment.py 13.3 KB
  - 📄 run_gate_ablation.py 14.2 KB
  - 📄 scan_published_repos.py 17.1 KB
  - 📄 statistical_analysis.py 18.3 KB
  - 📄 tool_comparison_matrix.json 10.4 KB
  - 📄 verify_repos.py 11.5 KB
- 📄 overnight_pipeline_run.sh 8.9 KB
📁 papers/
- 📁 audit_results/
  - 📄 .gitkeep 0 B
  - 📄 batch_scores.json 27.2 KB
- 📁 bmj/
  - 📁 cardiovascular/
    
    📄 .gitkeep 0 B
  - 📁 diabetes/
    
    📄 .gitkeep 0 B
  - 📁 infectious_disease/
    
    📄 .gitkeep 0 B
  - 📁 kidney_disease/
    
    📄 .gitkeep 0 B
  - 📁 neurology/
    
    📄 .gitkeep 0 B
  - 📁 oncology/
    
    📄 .gitkeep 0 B
  - 📁 other/
    
    📄 .gitkeep 0 B
  - 📁 respiratory/
    
    📄 .gitkeep 0 B
  - 📁 sepsis_icu/
    
    📄 .gitkeep 0 B
- 📁 jama/
  - 📁 cardiovascular/
    
    📄 .gitkeep 0 B
  - 📁 diabetes/
    
    📁 oconnor_2021_t2dm_readmission/
    
    📄 metadata.json 2.6 KB
    
    📄 .gitkeep 0 B
  - 📁 infectious_disease/
    
    📄 .gitkeep 0 B
  - 📁 kidney_disease/
    
    📄 .gitkeep 0 B
  - 📁 neurology/
    
    📄 .gitkeep 0 B
  - 📁 oncology/
    
    📄 .gitkeep 0 B
  - 📁 other/
    
    📄 .gitkeep 0 B
  - 📁 respiratory/
    
    📄 .gitkeep 0 B
  - 📁 sepsis_icu/
    
    📄 .gitkeep 0 B
- 📁 lancet_digital_health/
  - 📁 cardiovascular/
    
    📁 hippisley_cox_2017_qrisk3/
    
    📄 metadata.json 2.7 KB
    
    📄 .gitkeep 0 B
  - 📁 diabetes/
    
    📄 .gitkeep 0 B
  - 📁 infectious_disease/
    
    📄 .gitkeep 0 B
  - 📁 kidney_disease/
    
    📄 .gitkeep 0 B
  - 📁 neurology/
    
    📄 .gitkeep 0 B
  - 📁 oncology/
    
    📁 esteva_2017_skin_cancer/
    
    📄 metadata.json 2.6 KB
    
    📄 .gitkeep 0 B
  - 📁 other/
    
    📄 .gitkeep 0 B
  - 📁 respiratory/
    
    📄 .gitkeep 0 B
  - 📁 sepsis_icu/
    
    📄 .gitkeep 0 B
- 📁 manifests/
  - 📄 batch_manifest_all.json 2.3 KB
  - 📄 batch_manifest_jama_bmj.json 895 B
  - 📄 batch_manifest_lancet_dh.json 680 B
  - 📄 batch_manifest_nature_medicine.json 879 B
- 📁 nature_medicine/
  - 📁 cardiovascular/
    
    📁 attia_2019_af_ecg/
    
    📄 metadata.json 2.3 KB
    
    📄 .gitkeep 0 B
  - 📁 diabetes/
    
    📄 .gitkeep 0 B
  - 📁 infectious_disease/
    
    📄 .gitkeep 0 B
  - 📁 kidney_disease/
    
    📄 .gitkeep 0 B
  - 📁 neurology/
    
    📄 .gitkeep 0 B
  - 📁 oncology/
    
    📄 .gitkeep 0 B
  - 📁 other/
    
    📄 .gitkeep 0 B
  - 📁 respiratory/
    
    📄 .gitkeep 0 B
  - 📁 sepsis_icu/
    
    📁 tomaszewski_2019_sepsis/
    
    📄 metadata.json 2.7 KB
    
    📄 .gitkeep 0 B
- 📁 npj_digital_medicine/
  - 📁 cardiovascular/
    
    📄 .gitkeep 0 B
  - 📁 diabetes/
    
    📄 .gitkeep 0 B
  - 📁 infectious_disease/
    
    📄 .gitkeep 0 B
  - 📁 kidney_disease/
    
    📁 tomasev_2019_aki/
    
    📄 metadata.json 2.5 KB
    
    📄 .gitkeep 0 B
  - 📁 neurology/
    
    📄 .gitkeep 0 B
  - 📁 oncology/
    
    📄 .gitkeep 0 B
  - 📁 other/
    
    📄 .gitkeep 0 B
  - 📁 respiratory/
    
    📄 .gitkeep 0 B
  - 📁 sepsis_icu/
    
    📄 .gitkeep 0 B
- 📁 specialist_journals/
  - 📁 cardiovascular/
    
    📄 .gitkeep 0 B
  - 📁 diabetes/
    
    📄 .gitkeep 0 B
  - 📁 infectious_disease/
    
    📄 .gitkeep 0 B
  - 📁 kidney_disease/
    
    📄 .gitkeep 0 B
  - 📁 neurology/
    
    📄 .gitkeep 0 B
  - 📁 oncology/
    
    📄 .gitkeep 0 B
  - 📁 other/
    
    📄 .gitkeep 0 B
  - 📁 respiratory/
    
    📄 .gitkeep 0 B
  - 📁 sepsis_icu/
    
    📄 .gitkeep 0 B
- 📁 templates/
  - 📄 paper_metadata_template.json 3.7 KB
- 📄 .gitignore 156 B
- 📄 README.md 7.8 KB
📁 plugin/
- 📁 mlgg_lint/
  - 📁 rules/
    
    📄 __init__.py 1.2 KB
    
    📄 base.py 2.2 KB
    
    📄 r001_fit_before_split.py 5.2 KB
    
    📄 r002_scaler_on_test.py 2.7 KB
    
    📄 r003_smote_on_test.py 1.9 KB
    
    📄 r004_split_without_group.py 3.3 KB
    
    📄 r005_threshold_on_test.py 8.4 KB
    
    📄 r006_feature_selection_full.py 4.4 KB
    
    📄 r007_target_as_feature.py 6.1 KB
    
    📄 r008_temporal_split.py 3.1 KB
    
    📄 r009_no_confidence_intervals.py 3.0 KB
    
    📄 r010_train_metric_as_final.py 2.0 KB
    
    📄 r011_cv_internal_smote.py 2.5 KB
    
    📄 r012_cv_accuracy_imbalanced.py 2.6 KB
    
    📄 r013_hardcoded_threshold.py 1.9 KB
    
    📄 r014_label_encoder_features.py 3.8 KB
    
    📄 r015_small_test_set.py 1.6 KB
    
    📄 r016_no_random_state.py 1.7 KB
    
    📄 r017_early_stop_on_test.py 3.0 KB
    
    📄 r018_scaling_trees.py 2.3 KB
    
    📄 r019_multiple_comparison.py 2.9 KB
    
    📄 r020_global_clean_before_split.py 4.8 KB
  - 📄 __init__.py 114 B
  - 📄 __main__.py 95 B
  - 📄 ast_utils.py 7.4 KB
  - 📄 cli.py 4.4 KB
  - 📄 config.py 2.8 KB
  - 📄 engine.py 9.0 KB
  - 📄 formatters.py 5.1 KB
  - 📄 models.py 1.8 KB
  - 📄 notebook.py 2.8 KB
- 📁 tests/
  - 📁 rules/
    
    📄 __init__.py 0 B
  - 📁 samples/
    
    📄 leaky_notebook.ipynb 1.2 KB
    
    📄 r001_bad.py 415 B
    
    📄 r001_good.py 437 B
    
    📄 r001_good_no_split_file.py 272 B
    
    📄 r002_bad.py 393 B
    
    📄 r002_good.py 403 B
    
    📄 r003_bad.py 379 B
    
    📄 r003_bad_before_split.py 271 B
    
    📄 r003_good.py 378 B
    
    📄 r004_bad.py 322 B
    
    📄 r004_good.py 377 B
    
    📄 r005_bad.py 431 B
    
    📄 r005_good.py 385 B
    
    📄 r005_good_discard.py 337 B
    
    📄 r005_good_single_var_no_index2.py 348 B
    
    📄 r006_bad.py 415 B
    
    📄 r006_good.py 462 B
    
    📄 r007_bad.py 237 B
    
    📄 r007_bad2.py 314 B
    
    📄 r007_bad_target_in_cols.py 349 B
    
    📄 r007_good.py 243 B
    
    📄 r008_bad.py 310 B
    
    📄 r008_good.py 381 B
    
    📄 r009_bad.py 356 B
    
    📄 r009_good.py 488 B
    
    📄 r010_bad.py 350 B
    
    📄 r010_good.py 336 B
    
    📄 r011_bad.py 341 B
    
    📄 r011_good.py 390 B
    
    📄 r012_bad.py 348 B
    
    📄 r012_good.py 347 B
    
    📄 r013_bad.py 67 B
    
    📄 r013_good.py 291 B
    
    📄 r014_bad.py 256 B
    
    📄 r014_good.py 275 B
    
    📄 r015_bad.py 154 B
    
    📄 r015_good.py 153 B
    
    📄 r016_bad.py 295 B
    
    📄 r016_good.py 331 B
    
    📄 r017_bad.py 320 B
    
    📄 r017_bad_catboost_tuple.py 366 B
    
    📄 r017_good.py 420 B
    
    📄 r017_good_catboost_valid.py 359 B
    
    📄 r018_bad.py 282 B
    
    📄 r018_good.py 260 B
    
    📄 r019_bad.py 329 B
    
    📄 r019_good.py 503 B
    
    📄 r020_bad.py 326 B
    
    📄 r020_bad_clip_quantile.py 399 B
    
    📄 r020_bad_dropna.py 310 B
    
    📄 r020_good.py 430 B
    
    📄 r020_good_dropna_subset.py 369 B
  - 📄 __init__.py 0 B
  - 📄 conftest.py 48 B
  - 📄 test_cli.py 4.7 KB
  - 📄 test_engine.py 36.1 KB
  - 📄 test_formatters.py 1.3 KB
- 📁 vscode/
  - 📁 src/
    
    📄 extension.ts 5.4 KB
  - 📄 .vscodeignore 56 B
  - 📄 package.json 2.0 KB
  - 📄 tsconfig.json 271 B
- 📄 .mlgg-lint.example.toml 535 B
- 📄 .pre-commit-hooks.yaml 241 B
- 📄 pyproject.toml 1.1 KB
- 📄 README.md 6.8 KB
📁 references/
- 📄 API-Reference.md 22.0 KB
- 📄 Architecture.md 15.3 KB
- 📄 attestation-execution-log-record.example.json 1.8 KB
- 📄 attestation-execution-receipt-record.example.json 663 B
- 📄 attestation-payload.example.json 2.4 KB
- 📄 attestation-timestamp-record.example.json 350 B
- 📄 attestation-transparency-record.example.json 417 B
- 📄 attestation-witness-record.example.json 387 B
- 📄 batch-manifest.example.json 906 B
- 📄 Beginner-Quickstart.md 5.8 KB
- 📄 benchmark-registry.json 7.0 KB
- 📄 blind-audit-protocol.md 11.6 KB
- 📄 ci-matrix-report.example.json 2.5 KB
- 📄 dataset-gate-coverage-matrix.md 2.2 KB
- 📄 distribution-report.example.json 783 B
- 📄 error-knowledge-base.json 86.9 KB
- 📄 evaluation-report.example.json 5.0 KB
- 📄 execution-attestation.example.json 4.1 KB
- 📄 external-benchmark-comparison.md 7.3 KB
- 📄 external-cohort-spec.example.json 495 B
- 📄 external-validation-report.example.json 2.1 KB
- 📄 feature-engineering-report.example.json 1.5 KB
- 📄 feature-group-spec.example.json 441 B
- 📄 feature-lineage.example.json 330 B
- 📄 gate-applicability-matrix.json 15.3 KB
- 📄 gate-kapoor-coverage-analysis.md 5.2 KB
- 📄 gate-strictness-profiles.md 6.2 KB
- 📄 imbalance-policy.example.json 379 B
- 📄 interactive-profile.example.json 1.3 KB
- 📄 journal-rigor-standards.json 15.3 KB
- 📄 key-revocations.example.json 221 B
- 📄 leakage-rule-taxonomy-mapping.md 10.4 KB
- 📄 leakage-taxonomy.md 4.9 KB
- 📄 literature-knowledge-base.json 54.2 KB
- 📄 medical-disease-leakage.md 3.9 KB
- 📄 metadata-validation-rules.md 9.5 KB
- 📄 missingness-policy.example.json 7.0 KB
- 📄 mlgg-compliance-certificate.schema.json 9.3 KB
- 📄 mlgg-review-standard.json 66.4 KB
- 📄 mlgg-standard-specification.json 14.4 KB
- 📄 paper-audit-workflow.md 13.4 KB
- 📄 performance-policy.example.json 4.4 KB
- 📄 phase-sampling-strategy.md 7.0 KB
- 📄 prediction-trace.example.csv 573 B
- 📄 prediction-type-registry.json 7.8 KB
- 📄 probast-ai-signalling-questions.json 25.9 KB
- 📄 project-structure-convention.md 4.5 KB
- 📄 qwen-prompt-sensitivity-report.md 4.3 KB
- 📄 release-benchmark-suite.md 5.6 KB
- 📄 report-template.md 7.6 KB
- 📄 reporting-bias-checklist.example.json 2.0 KB
- 📄 request-schema.example.json 2.7 KB
- 📄 scoring-methodology.md 17.7 KB
- 📄 split-protocol.example.json 461 B
- 📄 stard-ai-checklist.json 11.7 KB
- 📄 stress-seed-search-report.v2.example.json 3.6 KB
- 📄 top-tier-rigor-checklist.md 9.6 KB
- 📄 tripod-ai-official-checklist.json 28.1 KB
- 📄 Troubleshooting-Top20.md 17.2 KB
- 📄 tuning-protocol.example.json 995 B
📁 scripts/
- 📄 __init__.py 0 B
- 📄 _audit_shared.py 8.9 KB
- 📄 _gate_framework.py 24.3 KB
- 📄 _gate_registry.py 23.4 KB
- 📄 _gate_utils.py 18.5 KB
- 📄 _security.py 47.8 KB
- 📄 audit_external_project.py 21.2 KB
- 📄 batch_journal_review.py 25.4 KB
- 📄 calibration_dca_gate.py 25.6 KB
- 📄 ci_matrix_gate.py 28.5 KB
- 📄 clinical_metrics_gate.py 32.0 KB
- 📄 compare_runs.py 8.9 KB
- 📄 covariate_shift_gate.py 30.1 KB
- 📄 definition_variable_guard.py 11.5 KB
- 📄 distribution_generalization_gate.py 30.8 KB
- 📄 env_doctor.py 5.9 KB
- 📄 evaluation_quality_gate.py 25.2 KB
- 📄 evidence_comparator.py 10.3 KB
- 📄 evidence_digest.py 7.9 KB
- 📄 execution_attestation_gate.py 125.5 KB
- 📄 explain_gate.py 14.4 KB
- 📄 export_latex.py 10.2 KB
- 📄 export_review_prompt.py 12.3 KB
- 📄 external_validation_gate.py 23.2 KB
- 📄 extract_paper_metadata.py 42.4 KB
- 📄 fairness_equity_gate.py 24.2 KB
- 📄 feature_engineering_audit_gate.py 14.6 KB
- 📄 feature_lineage_gate.py 18.2 KB
- 📄 fetch_papers.py 35.5 KB
- 📄 gate_applicability.py 4.9 KB
- 📄 gate_coverage_matrix.py 8.4 KB
- 📄 gate_timeline.py 10.3 KB
- 📄 generalization_gap_gate.py 9.9 KB
- 📄 generate_audit_report.py 45.8 KB
- 📄 generate_compliance_certificate.py 25.2 KB
- 📄 generate_demo_medical_dataset.py 7.3 KB
- 📄 generate_execution_attestation.py 51.4 KB
- 📄 imbalance_policy_gate.py 22.6 KB
- 📄 init_project.py 8.6 KB
- 📄 leakage_gate.py 14.6 KB
- 📄 manifest_lock.py 10.2 KB
- 📄 metric_consistency_gate.py 16.2 KB
- 📄 missingness_policy_gate.py 33.4 KB
- 📄 mlgg.py 24.9 KB
- 📄 mlgg_interactive.py 72.3 KB
- 📄 mlgg_onboarding.py 70.2 KB
- 📄 mlgg_pixel.py 248.0 KB
- 📄 mlgg_web.py 24.6 KB
- 📄 model_selection_audit_gate.py 29.3 KB
- 📄 permutation_significance_gate.py 9.0 KB
- 📄 policy_generator.py 13.7 KB
- 📄 prediction_replay_gate.py 17.6 KB
- 📄 publication_gate.py 20.3 KB
- 📄 quick_summary.py 11.3 KB
- 📄 remediation_plan.py 16.3 KB
- 📄 render_user_summary.py 12.7 KB
- 📄 report_health_check.py 7.4 KB
- 📄 reporting_bias_gate.py 15.1 KB
- 📄 request_contract_gate.py 139.5 KB
- 📄 robustness_gate.py 17.9 KB
- 📄 run_dag_pipeline.py 37.4 KB
- 📄 run_endurance_test.py 27.6 KB
- 📄 run_productized_workflow.py 10.9 KB
- 📄 run_strict_pipeline.py 45.2 KB
- 📄 sample_size_gate.py 19.4 KB
- 📄 schema_preflight.py 17.1 KB
- 📄 score_paper_metadata.py 23.4 KB
- 📄 security_audit_gate.py 13.2 KB
- 📄 seed_stability_gate.py 15.7 KB
- 📄 self_critique_gate.py 20.9 KB
- 📄 split_data.py 35.1 KB
- 📄 split_protocol_gate.py 17.6 KB
- 📄 test_gate_smoke.py 107.2 KB
- 📄 test_onboarding_smoke.py 5.7 KB
- 📄 test_play_smoke.py 97.3 KB
- 📄 test_split_smoke.py 19.3 KB
- 📄 threshold_sensitivity.py 14.3 KB
- 📄 train_select_evaluate.py 264.4 KB
- 📄 tuning_leakage_gate.py 17.8 KB
- 📄 visualize_results.py 10.5 KB
📁 tests/
- 📄 test_adversarial_e2e.py 10.7 KB
- 📄 test_audit_external_project.py 11.1 KB
- 📄 test_audit_logging.py 5.2 KB
- 📄 test_audit_shared.py 15.5 KB
- 📄 test_batch_journal_review.py 11.4 KB
- 📄 test_calibration_dca_gate.py 20.5 KB
- 📄 test_ci_matrix_gate.py 11.3 KB
- 📄 test_clinical_metrics_gate.py 15.7 KB
- 📄 test_compare_runs.py 14.3 KB
- 📄 test_covariate_shift_gate.py 28.0 KB
- 📄 test_dag_pipeline_e2e.py 13.6 KB
- 📄 test_definition_variable_guard.py 24.0 KB
- 📄 test_detect_columns.py 15.9 KB
- 📄 test_distribution_generalization_gate.py 11.2 KB
- 📄 test_download_real_data.py 4.6 KB
- 📄 test_env_doctor.py 6.6 KB
- 📄 test_evaluation_quality_gate.py 32.3 KB
- 📄 test_evidence_comparator.py 11.6 KB
- 📄 test_evidence_digest.py 8.5 KB
- 📄 test_execution_attestation_gate.py 13.7 KB
- 📄 test_explain_gate.py 11.2 KB
- 📄 test_export_latex.py 13.5 KB
- 📄 test_export_review_prompt.py 8.5 KB
- 📄 test_external_validation_gate.py 27.2 KB
- 📄 test_fairness_equity_gate.py 12.9 KB
- 📄 test_feature_engineering_audit_gate.py 15.6 KB
- 📄 test_feature_lineage_gate.py 17.1 KB
- 📄 test_feature_stats.py 10.4 KB
- 📄 test_full_matrix.py 6.6 KB
- 📄 test_gate_applicability.py 4.2 KB
- 📄 test_gate_coverage_matrix.py 10.8 KB
- 📄 test_gate_framework.py 30.3 KB
- 📄 test_gate_registry.py 9.8 KB
- 📄 test_gate_timeline.py 15.8 KB
- 📄 test_gate_utils.py 33.1 KB
- 📄 test_generalization_gap_gate.py 14.4 KB
- 📄 test_generate_audit_report.py 29.4 KB
- 📄 test_generate_compliance_certificate.py 11.2 KB
- 📄 test_generate_demo.py 7.3 KB
- 📄 test_generate_demo_dataset.py 6.6 KB
- 📄 test_generate_execution_attestation.py 5.5 KB
- 📄 test_imbalance_policy_gate.py 20.0 KB
- 📄 test_init_project.py 9.1 KB
- 📄 test_leakage_gate.py 25.6 KB
- 📄 test_manifest_lock.py 22.2 KB
- 📄 test_metric_consistency_gate.py 15.4 KB
- 📄 test_missingness_policy_gate.py 18.2 KB
- 📄 test_mlgg_cli.py 6.5 KB
- 📄 test_mlgg_cli_security.py 24.9 KB
- 📄 test_mlgg_interactive.py 10.5 KB
- 📄 test_mlgg_onboarding.py 7.8 KB
- 📄 test_model_selection_audit_gate.py 18.6 KB
- 📄 test_onboarding_e2e.py 7.3 KB
- 📄 test_packaging_contract.py 1.0 KB
- 📄 test_permutation_significance_gate.py 18.2 KB
- 📄 test_policy_generator.py 13.1 KB
- 📄 test_prediction_replay_gate.py 24.4 KB
- 📄 test_publication_gate.py 23.8 KB
- 📄 test_quick_summary.py 6.3 KB
- 📄 test_rbac_receipts.py 6.1 KB
- 📄 test_registry_cache.py 2.4 KB
- 📄 test_remediation_plan.py 11.7 KB
- 📄 test_render_user_summary.py 13.8 KB
- 📄 test_report_health_check.py 7.9 KB
- 📄 test_reporting_bias_gate.py 18.2 KB
- 📄 test_request_contract_gate.py 37.2 KB
- 📄 test_robustness_gate.py 21.7 KB
- 📄 test_run_dag_pipeline.py 16.7 KB
- 📄 test_run_productized_workflow.py 6.5 KB
- 📄 test_run_strict_pipeline.py 4.2 KB
- 📄 test_sample_size_gate.py 11.4 KB
- 📄 test_schema_preflight.py 33.0 KB
- 📄 test_score_paper_metadata.py 8.8 KB
- 📄 test_security.py 19.5 KB
- 📄 test_security_audit_gate.py 7.9 KB
- 📄 test_security_deep.py 7.8 KB
- 📄 test_security_extended.py 7.4 KB
- 📄 test_seed_stability_gate.py 15.1 KB
- 📄 test_self_critique_gate.py 16.4 KB
- 📄 test_split_data.py 33.5 KB
- 📄 test_split_e2e.py 6.1 KB
- 📄 test_split_protocol_gate.py 33.4 KB
- 📄 test_stress_audit_chain.py 9.1 KB
- 📄 test_stress_gate_cli.py 9.5 KB
- 📄 test_stress_numeric.py 13.3 KB
- 📄 test_stress_pipeline_e2e.py 14.5 KB
- 📄 test_stress_security.py 14.5 KB
- 📄 test_threshold_sensitivity.py 13.6 KB
- 📄 test_train_e2e.py 7.3 KB
- 📄 test_train_select_evaluate.py 31.9 KB
- 📄 test_tuning_leakage_gate.py 27.1 KB
- 📄 test_visualize_results.py 10.1 KB
- 📄 test_wizard_interactive.py 23.7 KB
- 📄 test_workflow_e2e.py 7.7 KB
📄 .coveragerc 254 B
📄 .gitignore 962 B
📄 CHANGELOG.md 18.4 KB
📄 CLAUDE.md 7.7 KB
📄 CONTRIBUTING.md 5.0 KB
📄 LICENSE 4.4 KB
📄 mypy.ini 192 B
📄 pyproject.toml 1.8 KB
📄 QUEUE_PROMPTS.md 63.7 KB
📄 README.md 33.0 KB
📄 requirements-optional.txt 333 B
📄 requirements.txt 131 B
📄 ruff.toml 602 B
📄 SKILL.md 64.8 KB

SKILL.md

---
name: ml-leakage-guard
description: "Publication-grade medical prediction workflow with strict anti-data-leakage controls, phenotype-definition safeguards, lineage-based leakage detection, split-protocol verification, class-imbalance policy validation, hyperparameter-tuning isolation checks, falsification tests, and reproducibility gates. Use when building, reviewing, or debugging disease risk or prognosis models in EHR/claims/registry data, especially when target definitions, diagnosis codes, lab criteria, medications, temporal windows, and derived features can leak target information."
---

# ML Leakage Guard

## AI 操作指引（Quick Dispatch）

当用户提出请求时，按以下决策树选择操作路径：

### 用户意图 → 操作命令

| 用户说的 | 你该做的 |
|---------|---------|
| "帮我训练一个模型" / "跑一下预测" | `python3 scripts/mlgg.py play` — 启动交互向导 |
| "用我的数据训练" / "我有一个 CSV" | `python3 scripts/mlgg.py play` → 选"使用自己的数据集" |
| "查看训练结果" / "结果怎么样" | `python3 scripts/quick_summary.py <output_dir>` |
| "下载一个测试数据集" | `python3 examples/download_real_data.py <name>` (heart/breast/pima/mammographic/thyroid/eeg_eye/vitaldb/framingham/diabetes130/diabetes130_full/rhc/sepsis_survival) |
| "下载 CDC 数据集" | `python3 examples/download_cdc_data.py <name>` (brfss/nhis/covid/all) |
| "下载 NHANES 数据集" | `python3 examples/download_nhanes.py --cycles both --output examples/nhanes_diabetes.csv` |
| "下载 NCI 癌症数据" | `python3 examples/download_nci_gdc.py --output examples/nci_gdc_cancer_survival.csv` |
| "审查论文 Methods (Qwen)" | `DASHSCOPE_API_KEY=sk-... python3 experiments/paper/review_methods_llm.py --pmcid PMCxxxxxx` |
| "Methods vs Code 比对" | `python3 experiments/paper/compare_methods_vs_code.py --methods-dir ... --audit-log ... --blind-list ... --output ...` |
| "统计分析" | `python3 experiments/paper/statistical_analysis.py --output experiments/paper/output/statistical_results.json` |
| "过夜批量跑 pipeline" | `nohup bash experiments/overnight_pipeline_run.sh > experiments/overnight_run.log 2>&1 &` |
| "严格审计" / "出版级验证" | `python3 scripts/mlgg.py workflow --strict` |
| "检查环境" / "安装有问题" | `python3 scripts/mlgg.py doctor` |
| "初始化项目" | `python3 scripts/mlgg.py onboarding` |
| "对比两次运行" | `python3 scripts/compare_runs.py --run-a <dir1> --run-b <dir2>` |
| "生成修复计划" | `python3 scripts/remediation_plan.py --evidence-dir <dir>` |
| "解释某个 gate 失败" | `python3 scripts/explain_gate.py --report <gate_report.json>` |
| "检查代码是否有数据泄漏" | `python3 scripts/mlgg.py lint check <file.py>` |
| "检查代码（JSON 给 agent）" | `python3 scripts/mlgg.py lint check <file.py> --format json` |
| "检查代码（CI 门控）" | `python3 scripts/mlgg.py lint check <dir> --exit-code` |
| "查看 lint 规则列表" | `python3 scripts/mlgg.py lint rules` |
| "评审一篇论文（从 metadata）" | `python3 scripts/score_paper_metadata.py --metadata <metadata.json>` |
| "批量评审论文" | `python3 scripts/score_paper_metadata.py --batch-dir papers/` |
| "从 PMC 收集有代码的论文" | `python3 experiments/paper/collect_papers_with_code.py --output <out.jsonl>` |
| "验证论文 repo 质量" | `python3 experiments/paper/verify_repos.py --input <in.jsonl> --output <out.jsonl>` |
| "批量扫描论文代码泄漏" | `python3 experiments/paper/scan_published_repos.py --manifest <verified.jsonl> --output <out.json>` |

### 五条常用命令（覆盖 90% 场景）

```bash
# 1. 新手一键体验（推荐入口）
python3 scripts/mlgg.py play

# 2. 快速查看结果
python3 scripts/quick_summary.py ~/Desktop/MLGG_Output/breast_cancer

# 3. 下载真实数据集
python3 examples/download_real_data.py breast --output /tmp/breast.csv

# 4. 严格出版级流程
python3 scripts/mlgg.py onboarding && python3 scripts/mlgg.py workflow --strict

# 5. 环境诊断
python3 scripts/mlgg.py doctor
```

### 添加新数据集的操作步骤

1. 在 `examples/download_real_data.py` 的 `URLS` 字典中添加下载 URL
2. 创建 `prepare_<name>()` 函数（参考现有函数格式）
3. 调用 `add_patient_id_and_time(df, seed=N)`（种子必须唯一）
4. 输出列顺序：`patient_id, event_time, y, features...`
5. 添加到 `PREPARE` 字典和 CLI `choices`
6. 在 `scripts/mlgg_pixel.py` 中添加 i18n 字符串 + `PLAY_DOWNLOAD_DATASETS` 条目
7. 测试：`python3 examples/download_real_data.py <name> --output /tmp/test.csv`

### 添加新模型族的操作步骤

修改 `scripts/train_select_evaluate.py` 的 5 个位置：
1. `SUPPORTED_MODEL_FAMILIES` 集合
2. `_family_grid()` — 超参数网格
3. `_build_estimator_for_family()` — Pipeline 构建
4. `_family_base_complexity()` — 复杂度排名
5. `_family_friendly_name()` — 显示名称

修改 `scripts/mlgg_pixel.py` 的 4 个位置：
6. `MODEL_POOL` 列表
7. `BASE_FAMILY_GRID_SIZES` 字典
8. `_T` i18n 字符串
9. `MODEL_PROFILE_PRESETS`（balanced/comprehensive）

### 添加新 Gate 的操作步骤

所有 gate 脚本必须遵循统一 CLI 契约：

1. **CLI 参数**：使用 `add_common_arguments(parser)` 或手动添加 `--report`、`--strict`、`--timeout`
2. **计时**：入口调用 `start_gate_timer()`
3. **报告输出**：使用 `build_report_envelope()` 生成标准信封格式
4. **终端输出**：使用 `print_gate_summary()` 打印结构化摘要
5. **退出逻辑**：`should_fail = bool(failures) or (args.strict and bool(warnings))`，返回 `2 if should_fail else 0`
6. **注册**：在 `_gate_registry.py` 中注册 gate 名称和路径
7. **无需手动同步 gate 列表**：以下工具脚本已从 `_gate_registry.py` 动态获取 gate 列表，添加新 gate 后自动生效：
   - `scripts/report_health_check.py` → `EXPECTED_REPORTS`
   - `scripts/remediation_plan.py` → `GATE_ORDER`
   - `scripts/evidence_digest.py` → `gate_files`
   - `scripts/compare_runs.py` → `REPORT_FILES`
   - 仍需手动更新：`scripts/render_user_summary.py` → `DEFAULT_GATE_FILES`（仅展示子集）、`scripts/run_strict_pipeline.py` → `gate_script_inputs`（manifest 指纹）
8. **测试**：在 `tests/` 中创建对应测试文件，覆盖率 ≥85%

**严禁**：
- 自定义 strict-mode 逻辑（如 `warning_is_blocking()` 过滤器）
- 跳过 `--strict` 对 warnings 的影响
- 手动提升 warnings 到 failures 列表（应由 `should_fail` 逻辑统一处理）

### 添加新 Lint 规则 (R0xx) 的操作步骤

1. 在 `plugin/mlgg_lint/rules/` 创建 `r0xx_rule_name.py`，继承 `BaseRule`
2. 设置 `id`、`name`、`severity`、`description`、`remediation`、`tags`
3. 在 `plugin/tests/samples/` 创建 `r0xx_bad.py`（触发诊断）和 `r0xx_good.py`（无诊断）
4. 在 `plugin/tests/test_engine.py` 添加 `test_r0xx_bad_has_diagnostics()` 和 `test_r0xx_good_no_r0xx()`
5. 运行 `python3 -m pytest plugin/tests/test_engine.py -v` 验证

**规则实现清单**：每个新规则合并前必须同时提供 bad + good 测试样本。

### 常见错误恢复

| 错误信息 | 根因 | 修复 |
|---------|------|------|
| `Unsupported model family` | 新模型未加到 `SUPPORTED_MODEL_FAMILIES` | 更新白名单（见上方 5 个位置） |
| `candidate_pool_too_small` | 候选模型少于 3 个 | 增加模型族或提高 `--max-trials-per-family` |
| `NaN to integer` | numpy 整数数组赋 NaN | 用 `DataFrame.loc[mask, col] = np.nan` |
| 训练超时（>20min） | 大数据集 + 多模型 + bootstrap | 减少模型数/trials/用保守预设 |
| `FileNotFoundError` | 路径错误或前序步骤未执行 | 检查 `data/` 目录下 CSV 是否存在 |
| R001 FP on utility files | 文件中无 train_test_split 但有 fit() | R001 已修复：skip_line is None 时跳过 (ERR-089) |
| R005 FP on unused thresholds | roc_curve 单变量捕获但未用 result[2] | R005 已修复：检查 index-2 access (ERR-090) |
| 空 metadata 通过验证 | validate_metadata({}) 返回 0 issues | 已修复：添加 REQUIRED 字段检查 (ERR-092) |
| BRFSS ZIP 文件名有空格 | CDC ZIP 中文件名尾部有空格 | 已修复：.strip() 处理 (ERR-098) |
| NCI GDC disease_type 是 list | API 返回 list 而非 string | 已修复：取 [0] 或 default (ERR-097) |

### 可用数据集清单（14 个，526K 行）

| 数据集 | 行数 | 来源 | 下载命令 | Gate 覆盖 |
|--------|------|------|---------|----------|
| Sepsis Survival | 129K | UCI | `download_real_data.py sepsis_survival` | C (39%) |
| Diabetes 130 Full | 102K | UCI | `download_real_data.py diabetes130_full` | A (94%) |
| BRFSS 2022 | 100K | CDC | `download_cdc_data.py brfss` | B (81%) |
| COVID-19 | 100K | CDC | `download_cdc_data.py covid` | C (39%) |
| NHIS 2022 | 28K | CDC | `download_cdc_data.py nhis` | A (94%) |
| NCI GDC Cancer | 25K | NCI/NIH | `download_nci_gdc.py` | A (94%) |
| NHANES | 16K | CDC | `download_nhanes.py --cycles both` | A (94%) |
| SUPPORT2 | 9K | Vanderbilt | 已下载 | A (94%) |
| RHC | 5.7K | Vanderbilt | `download_real_data.py rhc` | A (94%) |
| 4 × UCI 小型 | <1K | UCI | `download_real_data.py heart/breast/pima/ckd` | B (68-84%) |

Gate 覆盖: A=29/31可测, B=21-26/31, C=12/31。详见 `references/dataset-gate-coverage-matrix.md`。

### Gate 严格性 Profile

| Profile | 适用场景 | EPV 下限 | 最小事件数 | L3 可达？ |
|---------|---------|---------|-----------|----------|
| `standard` | N≥1000, 患病率≥10% | 10 | 100 | ✅ |
| `small_cohort` | N=200-1000 | 7 | 50 | ⚠️ 需注明 |
| `rare_disease` | N<200, 患病率<5% | 5 | 20 | ❌ |
| `exploratory` | 可行性研究 | 5 | 20 | ❌ |

在 `request.json` 中指定: `"thresholds": {"profile": "rare_disease"}`。详见 `references/gate-strictness-profiles.md`。

### 数据泄漏 & 学术诚信检测覆盖

本项目的 31 道 gate 覆盖以下学术诚信风险：

**数据泄漏检测（4 道 gate）**：
- `leakage_gate`: 行级重叠、患者 ID 重叠、时序穿越（训练数据晚于测试数据）
- `split_protocol_gate`: 分割协议验证（患者不重叠、时序有序、种子锁定）
- `definition_variable_guard`: 表型定义中的未来信息泄漏（用未来事件定义当前标签）
- `feature_lineage_gate`: 特征来源链路追溯（特征是否包含标签信息或未来数据）

**调优泄漏 / p-hacking（3 道 gate）**：
- `tuning_leakage_gate`: 超参搜索是否使用了测试数据、模型选择数据源验证
- `model_selection_audit_gate`: 候选池大小、选择标准、是否存在选择偏倚
- `evaluation_quality_gate`: 主指标是否有 CI、是否优于基线（防止挑选性报告）

**过拟合 & 泛化性（4 道 gate）**：
- `generalization_gap_gate`: train-test 性能差距是否超过阈值
- `covariate_shift_gate`: 训练/测试特征分布是否漂移
- `robustness_gate`: 时间切片和分组的性能稳健性
- `seed_stability_gate`: 不同随机种子下结果是否稳定

**统计严谨性（3 道 gate）**：
- `permutation_significance_gate`: 置换检验 p-value（模型是否优于随机）
- `ci_matrix_gate`: Bootstrap CI 完整性（所有指标都有置信区间）
- `prediction_replay_gate`: 预测结果是否可精确重现（防止结果篡改）

**临床有效性 & 报告完整性（3 道 gate）**：
- `calibration_dca_gate`: 概率校准质量 + 决策曲线分析
- `reporting_bias_gate`: TRIPOD+AI / PROBAST+AI / STARD-AI 清单合规
- `clinical_metrics_gate`: 混淆矩阵一致性、完整临床指标面板

**出版级聚合（2 道 gate）**：
- `publication_gate`: 聚合所有 gate 结果 + 执行签名验证
- `self_critique_gate`: 全局质量评分 + 审稿人级自我批评

### 缺失值插补 & Pipeline 隔离

**缺失值处理**（`train_select_evaluate.py`）：
- `SimpleImputer`（默认）：中位数填充 + 缺失指示器列
- `IterativeImputer` (MICE)：多重迭代插补（`--imputation-strategy mice`）
- 插补器在 sklearn Pipeline 内部，**只在训练集上 fit**，验证/测试集只做 transform
- 特征过滤阈值：strict 模式丢弃缺失率 >60% 的特征

**Pipeline 隔离保证**：
每个候选模型的 Pipeline 结构为 `imputer → scaler → classifier`：
- imputer 的统计量（中位数/参数）只从训练集计算
- scaler 的均值/标准差只从训练集计算
- classifier 只在训练集上拟合
- 验证/测试集只做 transform + predict，不影响任何参数

**超参数搜索隔离**（由 `tuning_leakage_gate` 强制检查）：
- `model_selection_data`: 只允许 `valid` / `cv_inner` / `nested_cv`（禁止 `test`）
- `early_stopping_data`: 只允许 `none` / `valid` / `cv_inner`（禁止 `test`）
- `preprocessing_fit_scope`: 必须是 `train_only`
- `feature_selection_scope`: 必须是 `train_only`
- `final_model_refit_scope`: 只允许 `train_only` / `train_plus_valid_no_test`

以上全部是 fail-closed 检查——违反任何一条即判定失败。

### 安全加固（Security Hardening）

本项目内置多层防御机制，覆盖以下攻击面：

**模型工件安全**：
- HMAC-SHA256 签名：训练完成后自动对 `.pkl` 文件生成签名（`.pkl.sig`）
- 安全加载：`SecureModelLoader` 在反序列化前验证签名，拒绝加载被篡改的模型
- 大小限制：模型文件超过 500MB 自动拒绝（防止 zip bomb 攻击）

**证据完整性**：
- 训练结束自动生成 SHA256 清单（`.manifest.json`），记录每个证据文件的哈希值和大小
- 可随时验证：`python3 scripts/_security.py audit evidence/`
- 检测篡改、缺失、敏感数据暴露

**输入验证**：
- `safe_path()` / `resolve_path()`: 路径穿越防护（null byte 注入、`..` 逃逸、系统目录封锁、沙箱 `Path.relative_to()` 强制检查）
- `safe_load_json()`: JSON 大小限制（100MB）+ 嵌套深度限制（50层）防止栈溢出/内存耗尽
- `check_csv_row_limit()`: CSV 行数限制防止内存耗尽 DoS

**密码学安全**：
- 所有 HMAC/签名比较必须使用 `hmac.compare_digest()`（常量时间比较，防止计时攻击）
- 禁止使用 `==` / `!=` 进行任何密码学值比较

**隐私防护**：
- `perturb_predictions()`: Laplace 机制扰动预测概率，防御成员推理攻击
- 敏感数据扫描：审计工具自动扫描证据文件中的 API key / password / token / PEM 私钥 / 医疗标识符（MRN/insurance_id）等

**供应链验证**：
- `verify_critical_imports()`: 运行时验证 sklearn/numpy/pandas 是否为真实库（非 monkey-patch）
- `.mlgg_model_key` 自动生成、权限 600、已加入 `.gitignore`

**CLI 工具**：`python3 scripts/_security.py [sign|verify|manifest|audit|check-deps]`

### 能力边界

**能做的**：
- 表格型医学二分类预测（EHR/临床/注册数据）
- 自动防泄漏分割 + 模型训练 + 评估 + 出版级审计
- 9 个真实数据集 + 自定义 CSV（支持中文列名）
- 20 个 sklearn 模型族 + 4 个可选后端
- 安全加固：HMAC 签名 + 证据清单 + 路径穿越防护 + 成员推理防御

**做不了的**：
- 图像/文本/时序等非表格数据
- 多分类/回归任务（仅二分类）
- 深度学习模型（TabNet/Transformer 等）
- 模型部署/API serving
- 交互式可视化 dashboard

---

## Objective (Goal Clarity)
Solve one narrow problem: produce leakage-safe, publication-grade medical prediction evidence.

Success is binary:
- `pass`: all hard gates pass and self-critique score reaches threshold.
- `fail`: any hard gate fails or strict review conditions are not met.

Never produce publication-grade claims without machine-checkable evidence artifacts.

## Input Contract (Structured Input)
Accept a structured request JSON, not free-form text.

Data input modes:
- **Pre-split mode**: user provides separate train/valid/test CSV files.
- **Single-file mode**: user provides one complete CSV; use `scripts/split_data.py` to auto-split with patient-level disjoint, temporal ordering, and prevalence checks. The interactive wizard (`mlgg interactive --command train`) and onboarding (`mlgg onboarding --input-csv`) support this mode natively.

Required fields:
- `study_id`
- `run_id`
- `target_name`
- `prediction_unit`
- `index_time_col`
- `label_col`
- `patient_id_col`
- `primary_metric`
- `claim_tier_target` (`leakage-audited` or `publication-grade`)
- `phenotype_definition_spec`
- `split_paths.train`
- `split_paths.test`

Publication-grade required fields:
- `feature_lineage_spec`
- `feature_group_spec`
- `split_protocol_spec`
- `imbalance_policy_spec`
- `missingness_policy_spec`
- `tuning_protocol_spec`
- `performance_policy_spec`
- `reporting_bias_checklist_spec`
- `execution_attestation_spec`
- `model_selection_report_file`
- `feature_engineering_report_file`
- `distribution_report_file`
- `robustness_report_file`
- `seed_sensitivity_report_file`
- `evaluation_report_file`
- `prediction_trace_file`
- `external_cohort_spec`
- `external_validation_report_file`
- `ci_matrix_report_file`
- `evaluation_metric_path`
- `permutation_null_metrics_file`
- `actual_primary_metric`
- `primary_metric` must be `pr_auc` for publication-grade strict mode.
- `evaluation_metric_path` terminal token must match `primary_metric` (after normalization).

Optional threshold keys under `thresholds`:
- `alpha` and `min_delta` for permutation significance gate.
- `min_baseline_delta`, `ci_min_resamples`, and `ci_max_width` for evaluation quality gate.

Path semantics:
- All relative paths in request JSON are resolved relative to the request file directory.

Template:
- `references/request-schema.example.json`
- `references/feature-lineage.example.json`
- `references/split-protocol.example.json`
- `references/imbalance-policy.example.json`
- `references/missingness-policy.example.json`
- `references/tuning-protocol.example.json`
- `references/performance-policy.example.json`
- `references/external-cohort-spec.example.json`
- `references/reporting-bias-checklist.example.json`
- `references/execution-attestation.example.json`
- `references/attestation-payload.example.json`
- `references/key-revocations.example.json`
- `references/attestation-timestamp-record.example.json`
- `references/attestation-transparency-record.example.json`
- `references/attestation-execution-receipt-record.example.json`
- `references/attestation-execution-log-record.example.json`
- `references/attestation-witness-record.example.json`
- `references/evaluation-report.example.json`
- `references/external-validation-report.example.json`
- `references/prediction-trace.example.csv`

Validate request first:

```bash
python3 scripts/request_contract_gate.py \
  --request configs/request.json \
  --report evidence/request_contract_report.json \
  --strict
```

## Hidden Workflow (Internal, Fail-Closed)
Use this internal sequence in order:
1. Validate request contract.
2. Lock data/config fingerprints (`manifest_lock.py`).
3. Run execution attestation gate (`execution_attestation_gate.py`).
4. Run split/time leakage gate (`leakage_gate.py`).
5. Run split protocol gate (`split_protocol_gate.py`).
6. Run covariate-shift gate (`covariate_shift_gate.py`).
7. Run reporting/bias checklist gate (`reporting_bias_gate.py`).
8. Run phenotype-definition leakage gate (`definition_variable_guard.py`).
9. Run lineage leakage gate (`feature_lineage_gate.py`).
10. Run imbalance policy gate (`imbalance_policy_gate.py`).
11. Run missingness policy gate (`missingness_policy_gate.py`).
12. Run tuning leakage gate (`tuning_leakage_gate.py`).
13. Run model-selection audit gate (`model_selection_audit_gate.py`).
14. Run feature-engineering audit gate (`feature_engineering_audit_gate.py`).
15. Run clinical-metrics gate (`clinical_metrics_gate.py`).
16. Run prediction-replay gate (`prediction_replay_gate.py`).
17. Run distribution-generalization gate (`distribution_generalization_gate.py`).
18. Run generalization-gap gate (`generalization_gap_gate.py`).
19. Run robustness gate (`robustness_gate.py`).
20. Run seed-stability gate (`seed_stability_gate.py`).
21. Run external-validation gate (`external_validation_gate.py`).
22. Run calibration+DCA gate (`calibration_dca_gate.py`).
23. Run CI-matrix gate (`ci_matrix_gate.py`).
24. Run metric consistency gate (`metric_consistency_gate.py`).
25. Run evaluation quality gate (`evaluation_quality_gate.py`).
26. Run permutation falsification gate (`permutation_significance_gate.py`).
27. Aggregate publication gate (`publication_gate.py`).
28. Run self-critique scoring gate (`self_critique_gate.py`).
29. Run security audit gate (`security_audit_gate.py`).
30. Run fairness & equity gate (`fairness_equity_gate.py`).
31. Run sample size adequacy gate (`sample_size_gate.py`).
32. Emit final report only if all strict gates pass.

Treat execution-attestation failures (signature/fingerprint/key-revocation/timestamp/transparency/execution-receipt/execution-log/witness-quorum/cross-role-authority-distinctness), disease-definition leakage, lineage ambiguity, metric-source ambiguity, split protocol violations, covariate-shift anomalies, class-imbalance misuse, missingness/imputation misuse, and tuning/test leakage as critical failures in strict mode.

## Output Contract (Machine-Parseable)
Produce these deterministic artifacts:
1. `evidence/request_contract_report.json`
2. `evidence/manifest.json`
3. `evidence/execution_attestation_report.json`
4. `evidence/reporting_bias_report.json`
5. `evidence/leakage_report.json`
6. `evidence/split_protocol_report.json`
7. `evidence/covariate_shift_report.json`
8. `evidence/definition_guard_report.json`
9. `evidence/lineage_report.json`
10. `evidence/imbalance_policy_report.json`
11. `evidence/missingness_policy_report.json`
12. `evidence/tuning_leakage_report.json`
13. `evidence/model_selection_audit_report.json`
14. `evidence/feature_engineering_audit_report.json`
15. `evidence/clinical_metrics_report.json`
16. `evidence/prediction_replay_report.json`
17. `evidence/distribution_generalization_report.json`
18. `evidence/generalization_gap_report.json`
19. `evidence/robustness_gate_report.json`
20. `evidence/seed_stability_report.json`
21. `evidence/external_validation_gate_report.json`
22. `evidence/calibration_dca_report.json`
23. `evidence/ci_matrix_gate_report.json`
24. `evidence/metric_consistency_report.json`
25. `evidence/evaluation_quality_report.json`
26. `evidence/permutation_report.json`
27. `evidence/publication_gate_report.json`
28. `evidence/self_critique_report.json`
29. `evidence/security_audit_gate_report.json`
30. `evidence/fairness_equity_report.json`
31. `evidence/sample_size_report.json`
32. `evidence/dag_pipeline_report.json`

Report status from each file must be machine-readable (`pass` or `fail`) with issue codes.

## Quality Control (Self-Critique)
Do not stop at initial gate pass.
Run `self_critique_gate.py` to score evidence quality and produce recommendations.

Publication-grade readiness requires:
- Strict-mode component reports.
- No blocking failures.
- Self-critique score at or above threshold (default `95`).

## Composability (Workflow Node Ready)
Each script is a composable node:
- Deterministic CLI interface.
- Deterministic JSON output.
- Deterministic exit code (`0` pass, `2` fail).

Use one-command orchestration for production use:

```bash
python3 scripts/run_strict_pipeline.py \
  --request configs/request.json \
  --evidence-dir evidence \
  --compare-manifest evidence/manifest_baseline.json \
  --strict
```

Productized one-command wrapper:

```bash
python3 scripts/run_productized_workflow.py \
  --request configs/request.json \
  --evidence-dir evidence \
  --allow-missing-compare \
  --strict
```

Novice onboarding wrapper (guided 8-step flow):

```bash
python3 scripts/mlgg.py onboarding \
  --project-root /tmp/mlgg_demo \
  --mode guided \
  --yes
```

Onboarding contract:
- `scripts/mlgg_onboarding.py` is strict-only (no policy downgrade path).
- Failure behavior:
  - default `--stop-on-fail` (fail-fast)
  - optional `--no-stop-on-fail` (collect full diagnostics while keeping fail-closed result)
  - guided mode without interactive stdin fails closed with `onboarding_interactive_input_unavailable` (use `--yes` or `--mode auto`)
  - wrapper route-conflict failure code: `authority_preset_route_override_forbidden`
- Modes:
  - `guided`: step-by-step command preview + confirmation.
  - `preview`: print the full 8-step command plan only; report includes `preview_only=true` and `display_status=preview`.
  - `auto`: execute all steps non-interactively.
- Step order is fixed:
  1. `env_doctor.py`
  2. `init_project.py`
  3. `generate_demo_medical_dataset.py`
  4. config alignment to demo schema (`request/lineage/group/external spec`)
  5. `train_select_evaluate.py`
  6. `generate_execution_attestation.py` (+ keypair bootstrap if needed)
  7. `run_productized_workflow.py --strict --allow-missing-compare`
  8. `run_productized_workflow.py --strict --compare-manifest ...`
- Required report:
  - `evidence/onboarding_report.json` (`contract_version=onboarding_report.v2`)
  - report fields include `stop_on_fail`, `termination_reason`, `failure_codes`, `next_actions`, `copy_ready_commands`, `preview_only`, `display_status`
  - `copy_ready_commands` uses absolute `mlgg.py` path so commands are runnable from any working directory.
- Offline demo data artifacts:
  - `data/train.csv`, `data/valid.csv`, `data/test.csv`
  - `data/external_2025_q4.csv` (`cross_period`)
  - `data/external_site_b.csv` (`cross_institution`)

This wrapper runs:
1. `env_doctor.py`
2. `schema_preflight.py`
3. `run_strict_pipeline.py`
4. `render_user_summary.py`

For first-run baseline bootstrap, you may omit `--compare-manifest` only with:
- `--allow-missing-compare`
- `run_strict_pipeline.py` always enforces `--strict` for publication-grade execution.
- `--allow-missing-compare` is bootstrap-only for artifact generation; publication-grade readiness still fails until baseline manifest comparison exists.
- `run_strict_pipeline.py` is publication-grade only; non-publication claim tiers are rejected.

## Personal UX Quickstart (Signed Attestation)
Create keypair once:

```bash
mkdir -p keys
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:3072 -out keys/attestation_priv.pem
openssl pkey -in keys/attestation_priv.pem -pubout -out keys/attestation_pub.pem
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:3072 -out keys/timestamp_priv.pem
openssl pkey -in keys/timestamp_priv.pem -pubout -out keys/timestamp_pub.pem
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:3072 -out keys/execution_priv.pem
openssl pkey -in keys/execution_priv.pem -pubout -out keys/execution_pub.pem
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:3072 -out keys/execution_log_priv.pem
openssl pkey -in keys/execution_log_priv.pem -pubout -out keys/execution_log_pub.pem
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:3072 -out keys/witness_a_priv.pem
openssl pkey -in keys/witness_a_priv.pem -pubout -out keys/witness_a_pub.pem
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:3072 -out keys/witness_b_priv.pem
openssl pkey -in keys/witness_b_priv.pem -pubout -out keys/witness_b_pub.pem
```

Generate payload + signature + spec in one command:

```bash
python3 scripts/generate_execution_attestation.py \
  --study-id sepsis-risk-icu-v1 \
  --run-id sepsis-risk-icu-v1-train-2026-02-24-001 \
  --payload-out evidence/attestation_payload.json \
  --signature-out evidence/attestation.sig \
  --spec-out configs/execution_attestation.json \
  --private-key-file keys/attestation_priv.pem \
  --public-key-file keys/attestation_pub.pem \
  --timestamp-private-key-file keys/timestamp_priv.pem \
  --timestamp-public-key-file keys/timestamp_pub.pem \
  --execution-private-key-file keys/execution_priv.pem \
  --execution-public-key-file keys/execution_pub.pem \
  --execution-log-private-key-file keys/execution_log_priv.pem \
  --execution-log-public-key-file keys/execution_log_pub.pem \
  --require-independent-timestamp-authority \
  --require-independent-execution-authority \
  --require-independent-log-authority \
  --require-witness-quorum \
  --min-witness-count 2 \
  --require-independent-witness-keys \
  --require-witness-independence-from-signing \
  --witness "witness-a|keys/witness_a_pub.pem|keys/witness_a_priv.pem" \
  --witness "witness-b|keys/witness_b_pub.pem|keys/witness_b_priv.pem" \
  --command "python train.py --config configs/train_config.json --seed 42" \
  --artifact training_log=evidence/train.log \
  --artifact training_config=configs/train_config.json \
  --artifact model_artifact=models/model_v1.bin \
  --artifact evaluation_report=evidence/evaluation_report.json \
  --artifact prediction_trace=evidence/prediction_trace.csv.gz \
  --artifact external_validation_report=evidence/external_validation_report.json
```

This command also creates:
- `configs/key_revocations.json` (bootstrapped if missing)
- `evidence/attestation_timestamp_record.json` + `.sig`
- `evidence/attestation_transparency_record.json` + `.sig`
- `evidence/attestation_execution_receipt_record.json` + `.sig`
- `evidence/attestation_execution_log_record.json` + `.sig`
- `evidence/attestation_witness_record_1.json` + `.sig`
- `evidence/attestation_witness_record_2.json` + `.sig`

## Manual Strict Execution Order
If orchestration is unavailable, run in this exact order:
1. `request_contract_gate.py`
2. `manifest_lock.py` (with optional `--compare-with`)
3. `execution_attestation_gate.py`
4. `leakage_gate.py`
5. `split_protocol_gate.py`
6. `covariate_shift_gate.py`
7. `reporting_bias_gate.py`
8. `definition_variable_guard.py`
9. `feature_lineage_gate.py`
10. `imbalance_policy_gate.py`
11. `missingness_policy_gate.py`
12. `tuning_leakage_gate.py`
13. `model_selection_audit_gate.py`
14. `feature_engineering_audit_gate.py`
15. `clinical_metrics_gate.py`
16. `prediction_replay_gate.py`
17. `distribution_generalization_gate.py`
18. `generalization_gap_gate.py`
19. `robustness_gate.py`
20. `seed_stability_gate.py`
21. `external_validation_gate.py`
22. `calibration_dca_gate.py`
23. `ci_matrix_gate.py`
24. `metric_consistency_gate.py`
25. `evaluation_quality_gate.py`
26. `permutation_significance_gate.py`
27. `publication_gate.py`
28. `self_critique_gate.py`
29. `security_audit_gate.py`
30. `fairness_equity_gate.py`
31. `sample_size_gate.py`

Note: Steps 30-31 run in METRIC_VALIDATION layer (parallel with steps 16-26 in DAG mode). In manual sequential mode, run them after step 29 to ensure all dependencies are available.

If any step returns non-zero, stop and block claim release.

## Medical Non-Negotiable Rules
- Never tune on test data.
- Never fit preprocessors on combined train+validation+test.
- Never apply resampling/SMOTE on validation or test splits.
- Never select thresholds or calibrate probabilities on test split.
- Never fit imputers on validation/test distributions.
- Never use target/outcome information for feature imputation.
- Never run MICE at oversized scale without audited fallback evidence (`mice_with_scale_guard`).
- Never ignore severe train-vs-holdout distribution separability without explicit mitigation and downgrade.
- Never perform model ranking/selection with any test-derived signal.
- Never release without full split-level clinical metrics (accuracy/precision/PPV/NPV/sensitivity/specificity/F1/F2-beta/ROC-AUC/PR-AUC/Brier).
- Never ignore train/valid/test gap breaches beyond configured fail thresholds.
- Never claim publication-grade without signed execution attestation proving run command, timing, and artifact hashes.
- Never reuse revoked/expired/over-age signing keys for publication-grade claims.
- Never omit trusted timestamp or transparency-log records for publication-grade claims.
- Never omit signed execution-receipt proof (with exit code and timing consistency) for publication-grade claims.
- Never omit signed execution-log attestation binding `training_log` to payload hash for publication-grade claims.
- Never omit witness-quorum evidence with independent witness keys and minimum validated witness count for publication-grade claims.
- Never claim publication-grade if TRIPOD+AI/PROBAST+AI checklist has unmet required items.
- Never accept publication-grade primary metrics from non-test evaluation splits; evaluation report must explicitly declare `split=test`.
- Never claim publication-grade without valid primary-metric confidence interval and explicit baseline comparison in the evaluation artifact.
- Never include variables used to define the disease label as model predictors.
- Never include derived features whose lineage contains disease-defining variables.
- Never include post-index features for pre-index prediction tasks.
- Never report point estimates without uncertainty and robustness checks.
- Never claim causality from predictive associations.
- Never publish subgroup predictions without fairness/equity assessment (equalized odds, disparate impact).
- Never claim adequate sample size without EPV ≥ 10 justification (Riley et al. 2019).
- Never omit IDI/NRI when comparing against baseline models for top-tier journals.
- Never use ICD diagnostic codes from the same admission as predictors without verifying temporal precedence.
- Never claim TRIPOD+AI adherence without the 2024 expanded 27-item checklist (BMJ 2024;385:e078378).

## Resources

### scripts/
- `scripts/run_strict_pipeline.py`: single-entry strict orchestrator.
- `scripts/request_contract_gate.py`: request schema/path validation and publication-policy anti-downgrade checks.
- `scripts/mlgg.py`: unified command entrypoint (`onboarding`, `interactive`, `init`, `train`, `workflow`, ...).
- `scripts/mlgg_onboarding.py`: novice-guided strict onboarding flow and report emitter.
- `scripts/split_data.py`: split a single CSV into train/valid/test with patient-level disjoint, temporal ordering, prevalence safety checks, NaN patient_id/target exclusion, row count preservation, SHA256 input fingerprint, min 10 pos/neg per split, min 5 patients per split, and prevalence shift warning.
- `scripts/generate_demo_medical_dataset.py`: offline reproducible demo dataset generator.
- `scripts/manifest_lock.py`: dataset/protocol/evaluation/gate-script fingerprint and baseline comparison.
- `scripts/execution_attestation_gate.py`: signed run-attestation and artifact-hash verification gate.
- `scripts/generate_execution_attestation.py`: one-command payload/signature/spec/timestamp/transparency/execution-receipt/execution-log/witness-quorum generator for personal users.
- `scripts/reporting_bias_gate.py`: TRIPOD+AI / PROBAST+AI / STARD-AI checklist hard gate.
- `scripts/leakage_gate.py`: split contamination, ID overlap, and temporal boundary checks.
- `scripts/split_protocol_gate.py`: enforce split protocol consistency and temporal/group safeguards.
- `scripts/covariate_shift_gate.py`: train-vs-holdout covariate-shift and split separability risk gate.
- `scripts/definition_variable_guard.py`: hard gate against disease-definition variable leakage.
- `scripts/feature_lineage_gate.py`: hard gate against lineage-derived leakage.
- `scripts/imbalance_policy_gate.py`: validate class-imbalance strategy and train-only resampling policy.
- `scripts/missingness_policy_gate.py`: validate missing-data strategy, large-scale method suitability, and imputer isolation policy.
- `scripts/tuning_leakage_gate.py`: validate hyperparameter tuning/test-isolation protocol.
- `scripts/model_selection_audit_gate.py`: validate candidate pool, one-SE replay, and test-isolated model selection.
- `scripts/feature_engineering_audit_gate.py`: validate feature-group provenance, train-only engineering scope, stability evidence, and reproducibility fields.
- `scripts/clinical_metrics_gate.py`: validate clinical metric completeness and confusion-matrix consistency per split.
- `scripts/distribution_generalization_gate.py`: train-vs-holdout distribution shift, split separability, and transport-readiness gate.
- `scripts/generalization_gap_gate.py`: fail-closed overfitting gap checks across train/valid/test.
- `scripts/ci_matrix_gate.py`: bootstrap CI matrix gate for primary metric and transport-drop CI on internal and external cohorts.
- `scripts/metric_consistency_gate.py`: extract and validate metric from evaluation report.
- `scripts/evaluation_quality_gate.py`: enforce primary-metric CI quality and baseline improvement checks.
- `scripts/permutation_significance_gate.py`: falsification significance gate.
- `scripts/publication_gate.py`: aggregate fail-closed publication gate.
- `scripts/self_critique_gate.py`: quality scoring and reviewer-grade self-critique gate.
- `scripts/train_select_evaluate.py`: terminal-ready training, model selection, threshold selection, and evaluation artifact generator.
- `scripts/train_select_evaluate.py` model-pool controls: `--model-pool`, `--include-optional-models`, `--max-trials-per-family`, `--hyperparam-search`, `--n-jobs`.
- `scripts/train_select_evaluate.py` optional model backends: `xgboost` and `catboost` are auto-detected and fail-closed when explicitly requested but unavailable.
- `scripts/init_project.py`: one-command initialization for `configs/`, `data/`, `evidence/`, `models/`, `keys/`, plus `configs/request.json`.
- `scripts/schema_preflight.py`: train/valid/test schema checks with semantic column auto-mapping report.
- `scripts/env_doctor.py`: dependency and environment diagnostics with optional-backend checks.
- `scripts/render_user_summary.py`: user-facing markdown/json summary from strict evidence artifacts.
- `scripts/run_productized_workflow.py`: full UX wrapper (doctor -> preflight -> strict pipeline -> user summary).
- `scripts/mlgg_interactive.py`: terminal interactive wizard for core commands (`init/workflow/train/authority`) with command preview, confirm-before-run, and profile save/load.
- `scripts/mlgg_pixel.py`: pixel-art interactive CLI wizard (`mlgg.py play`) for guided pipeline setup and execution with bilingual (en/zh) support, dataset-size-aware defaults, small-sample strict mode, and play-mode quick-readiness card.
- `scripts/_gate_utils.py`: shared utility functions (`add_issue`, `load_json`, `write_json`, `to_float`) for gate scripts.
- `scripts/_security.py`: security hardening module — HMAC model signing, path traversal protection, secure JSON loading, artifact integrity manifest, membership inference defense, dependency verification, security audit CLI.
- `scripts/security_audit_gate.py`: 29th pipeline gate (FINAL layer) — verifies model HMAC signatures, evidence manifest integrity, dependency authenticity, file permissions, sensitive data exposure, artifact sizes.
- `scripts/fairness_equity_gate.py`: 30th pipeline gate (METRIC_VALIDATION layer) — equalized odds gap across demographic/clinical subgroups, disparate impact ratio (four-fifths rule), per-subgroup PR-AUC validation.
- `scripts/sample_size_gate.py`: 31st pipeline gate (METRIC_VALIDATION layer) — EPV (Riley et al. 2019/2025), shrinkage factor, minimum events/non-events adequacy.
- `scripts/policy_generator.py`: generate recommended `performance_policy.json` from evidence reports with configurable margin and presets.
- `scripts/gate_timeline.py`: analyze gate execution timeline, identify bottleneck gates, compute wall-clock span.
- `scripts/gate_coverage_matrix.py`: scan evidence directory against full gate registry to produce coverage matrix.
- `scripts/evidence_comparator.py`: compare two evidence directories side-by-side showing improved/regressed/new/removed gates.
- `scripts/evidence_digest.py`: generate compact one-page summary from evidence directory.
- `scripts/report_health_check.py`: scan all gate reports for completeness and pass rate.
- `scripts/remediation_plan.py`: generate prioritized remediation plan from gate failures.
- `scripts/threshold_sensitivity.py`: analyze how close metrics sit to pass/fail thresholds.
- `scripts/compare_runs.py`: compare two pipeline runs side-by-side.
- `scripts/export_latex.py`: generate LaTeX tables from evaluation/CI/model-selection reports.
- `scripts/explain_gate.py`: explain a single gate result in human-readable form.
- `scripts/quick_summary.py`: one-command training results viewer with key metrics, overfitting risk, model selection top-10.
- `scripts/audit_external_project.py`: 10-dimension quantitative audit tool for evaluating medical ML projects (100-point scale) with journal-specific gap analysis.
- `scripts/fairness_equity_gate.py`: fail-closed fairness and equity gate — equalized odds gap, disparate impact ratio (four-fifths rule), per-subgroup PR-AUC validation.
- `scripts/sample_size_gate.py`: fail-closed sample size adequacy gate — EPV (Riley et al. 2019/2025), shrinkage factor, min events/non-events.
- `scripts/batch_journal_review.py`: batch audit N projects in parallel with comparison matrix, cross-cutting analysis, and aggregated remediation priorities.
- `experiments/authority-e2e/scan_stress_diabetes_feasibility.py`: stress-case diabetes feasibility scanner across target modes and row caps; outputs a fail-closed feasibility report.

### plugin/
- `plugin/mlgg_lint/`: AST-based static analysis for ML Python code (10 rules: R001–R010, 57 tests).
- R001 fit-before-split (ERROR), R002 scaler-on-test (ERROR), R003 resample-on-test (ERROR), R004 split-without-group (WARNING), R005 threshold-on-test (ERROR), R006 feature-selection-on-full (ERROR), R007 target-as-feature (ERROR), R008 temporal-split-shuffle (WARNING), R009 no-confidence-intervals (INFO), R010 train-metric-as-final (WARNING).
- Detection: keyword args (`fit(X=X_test)`), chained calls (`SMOTE().fit_resample()`), DataFrame origin tracking + `.drop()` re-assignment, Pipeline exclusion, word-boundary variable classification.
- CLI: `python3 scripts/mlgg.py lint check [--format text|json|sarif] [--exit-code] [--severity warning] [--disable R004,R008] PATH...`
- Supports `# noqa: R001` / `# noqa` inline suppression and `.mlgg-lint.toml` config auto-discovery.
- Output: relative paths (no absolute path leakage), ANSI-stripped in no-color mode.
- Security: 16 MB file limit, 1 MB config limit, symlink skip, stat-error handling, malformed TOML graceful fallback.
- VS Code extension at `plugin/vscode/` (SARIF-based diagnostics on save/open).
- Pre-commit hook at `plugin/.pre-commit-hooks.yaml`.

### examples/
- `examples/download_real_data.py`: download and prepare 9 real medical datasets (UCI/PhysioNet/GitHub) + 2 synthetic generators.
  - Real datasets: heart(297), breast(569), pima(768), mammographic(961), framingham(4240), vitaldb(6388), thyroid(7200), diabetes130(10000), eeg_eye(14980).
  - All produce pipeline-ready CSV with `patient_id`, `event_time`, `y` columns.

### tests/
- `tests/`: 2905+ pytest unit tests covering all gate scripts and analysis tools.
  - Direct `main()` tests for 20+ gate scripts (bypass subprocess for in-process coverage).
  - All gate modules ≥86% coverage; publication_gate 97%, evaluation_quality_gate 94%.
  - Run: `python3 -m pytest tests/ -q --tb=short` (~10 min for full suite).

### references/
- `references/Beginner-Quickstart.md`: bilingual novice quickstart (minimal loop + publication-grade loop).
- `references/Troubleshooting-Top20.md`: high-frequency failure code to diagnosis/fix/verify mapping.
- `references/request-schema.example.json`: structured request template.
- `references/feature-lineage.example.json`: lineage map template.
- `references/split-protocol.example.json`: split protocol template.
- `references/imbalance-policy.example.json`: class-imbalance policy template.
- `references/missingness-policy.example.json`: missing-data/imputation policy template.
- `references/tuning-protocol.example.json`: hyperparameter tuning protocol template.
- `references/performance-policy.example.json`: metric panel/threshold/gap policy template.
- `references/reporting-bias-checklist.example.json`: TRIPOD+AI / PROBAST+AI / STARD-AI checklist template.
- `references/execution-attestation.example.json`: signed execution-attestation spec template.
- `references/attestation-payload.example.json`: signed payload template with artifact hashes.
- `references/key-revocations.example.json`: key revocation list template.
- `references/attestation-timestamp-record.example.json`: trusted timestamp record template.
- `references/attestation-transparency-record.example.json`: transparency log record template.
- `references/attestation-execution-receipt-record.example.json`: execution receipt record template.
- `references/attestation-execution-log-record.example.json`: execution-log attestation record template.
- `references/attestation-witness-record.example.json`: witness attestation record template.
- `references/feature-group-spec.example.json`: feature group specification template (groups, train-only scope).
- `references/feature-engineering-report.example.json`: feature-engineering audit report template.
- `references/distribution-report.example.json`: distribution/shift report template.
- `references/ci-matrix-report.example.json`: CI matrix report template.
- `references/external-validation-report.example.json`: external validation report template.
- `references/evaluation-report.example.json`: evaluation metrics report template.
- `references/interactive-profile.example.json`: interactive CLI profile contract example (`contract_version/command/saved_at_utc/argument_values/python/cwd`).
- `references/benchmark-registry.json`: frozen benchmark dataset registry (contract `benchmark_registry.v1`).
- `references/stress-seed-search-report.v2.example.json`: stress seed/profile search contract template.
- `references/medical-disease-leakage.md`: medical phenotype leakage patterns and controls.
- `references/leakage-taxonomy.md`: leakage classes, red flags, and mitigations.
- `references/top-tier-rigor-checklist.md`: submission-grade hard gates.
- `references/external-benchmark-comparison.md`: external tool/guideline comparison and gap map.
- `references/release-benchmark-suite.md`: structured benchmark profile matrix and pass contract.
- `references/report-template.md`: reporting template for methods/results/robustness.
- `references/error-knowledge-base.json`: self-improving error pattern database with 25 known patterns, agent-appendable.
- `references/journal-rigor-standards.json`: top-tier journal requirements mapped to gates (Nature Medicine, Lancet DH, JAMA, BMJ, npj DM).
- `references/literature-knowledge-base.json`: curated top-journal literature database (30 entries, LIT-001–LIT-030), searchable by category/gate/dimension.
- `references/mlgg-review-standard.json`: independent MLGG Medical ML Review Standard — 10 dimensions × 73 criteria across 3 review levels (quick/standard/comprehensive).
- `references/batch-manifest.example.json`: batch manifest template for multi-project review.

## Authority E2E Execution Notes
- Recommended single-entry CLI:
  - `python3 scripts/mlgg.py <command> [command-args]`
  - Examples:
    - `python3 scripts/mlgg.py init --project-root /tmp/mlgg_demo`
    - `python3 scripts/mlgg.py train --interactive`
    - `python3 scripts/mlgg.py interactive --command workflow --profile-name demo --save-profile`
    - `python3 scripts/mlgg.py workflow --request /tmp/mlgg_demo/configs/request.json --strict --allow-missing-compare`
    - `python3 scripts/mlgg.py authority --include-stress-cases`
    - `python3 scripts/mlgg.py benchmark-suite --profile release` (recommended multi-dataset stability verdict)
    - `python3 scripts/mlgg.py benchmark-suite --profile release --repeat 3 --registry-file references/benchmark-registry.json`
    - `python3 scripts/mlgg.py authority-release` (recommended release stress path)
    - `python3 scripts/mlgg.py authority-research-heart --stress-seed-min 20250003 --stress-seed-max 20250060` (research/high-pressure mode)
    - preset wrappers are fixed-route; conflicting route flags are rejected fail-closed
    - add `--error-json` for machine-readable failures (`contract_version=mlgg_error.v1`)

- New-user order of operations:
  - `init` -> place split CSVs -> `train` (emit required evidence artifacts) -> `workflow --strict --allow-missing-compare`.
  - Follow-up reproducible runs should pass `--compare-manifest <project>/evidence/manifest_baseline.bootstrap.json`.

- Interactive wizard defaults:
  - Supports `init/workflow/train/authority`.
  - Preview command before execution, then require one confirm step.
  - Train wizard defaults `--include-optional-models` to off; enable manually only when optional backends are installed.
  - Train wizard defaults `--n-jobs` to `1` for cross-platform stability; increase manually for multi-core runs.
  - Train wizard default artifact outputs are auto-scoped to split project base (`<project>/evidence`) inferred from train split path.
  - Train wizard emits `--external-validation-report-out` only when `external_cohort_spec` is provided.
  - Train wizard emits `--feature-engineering-report-out` only when `feature_group_spec` is provided.
  - Profile reuse:
    - `--profile-name <name> --save-profile`
    - `--profile-name <name> --load-profile`
    - `--accept-defaults` for non-blocking execution with defaults/profile values
  - Profile path defaults to `~/.mlgg/profiles` (override with `--profile-dir`).
  - For workflow wizard, `--strict` is always injected and cannot be bypassed by interactive mode.
  - Workflow wizard first-run default enables `--allow-missing-compare` when no baseline manifest is provided/found.
  - Workflow wizard now auto-suggests evidence output under request project base (`<project>/evidence` when request is under `configs/`).
  - Authority wizard now defaults to release-grade stress path (`--include-stress-cases --stress-case-id uci-chronic-kidney-disease`);
    selecting `uci-heart-disease` is treated as advanced research/high-pressure mode.

- Use isolated output paths in concurrent runs:
  - `--summary-file`
  - `--stress-seed-cache-file`
  - `--stress-selection-file`
- Optional benchmark case switches:
  - `--include-ckd-case` (UCI Chronic Kidney Disease)
  - `--include-large-cases` (Diabetes130 large-cohort path)
  - `--diabetes-target-mode {lt30,gt30,any}` and `--diabetes-max-rows`
- Stress dataset selection:
  - `--stress-case-id {uci-diabetes-130-readmission,uci-heart-disease,uci-chronic-kidney-disease,uci-breast-cancer-wdbc}`
  - default is `uci-chronic-kidney-disease` (most stable publication-grade stress path in current benchmark set)
- Release benchmark blocking suites are `authority_release_core` + `adversarial_fail_closed`; `authority_release_extended` (Diabetes130) is kept as observational/non-blocking in release profile.
- Non-blocking authority failures are summarized as `observational_diagnostics` in matrix report and written to `*.observational_diagnostics.json` sidecar.
- Case-specific training configuration is enabled in authority E2E:
  - larger cohorts (e.g., Diabetes130) use expanded model pool (includes `xgboost` when installed), higher `max-trials-per-family`, and multi-core `--n-jobs`.
- Use `--run-tag` to bind all generated stress artifacts to a unique execution token.
- Stress seed-search profile bundles are selected with `--stress-profile-set` (default `strict_v1`).
- `--stress-seed-search` applies only to `--stress-case-id uci-heart-disease`; other stress cases run without seed search.
- CI coverage:
  - `.github/workflows/ci-smoke.yml` (push/PR/workflow_dispatch)
  - `.github/workflows/ci-full.yml` (nightly/workflow_dispatch release blocking benchmark-suite)
  - `.github/workflows/ci-extended.yml` (weekly/workflow_dispatch extended observational benchmark-suite)
- Optional diabetes feasibility auto-scan on failure:
  - `--auto-scan-diabetes-feasibility`
  - `--diabetes-feasibility-target-modes`
  - `--diabetes-feasibility-max-rows-options`
  - `--diabetes-feasibility-summary-dir`
  - `--diabetes-feasibility-report-file`
- Summary rows now include strict-pipeline root-cause fields for failed cases:
  - `root_failure_code_primary`
  - `root_failure_codes`
  - `failed_steps`
- Summary rows now also include `clinical_floor_gap_summary` with internal/external floor margins
  (`observed - required_min`) for `sensitivity/npv/specificity/ppv`.
- `stress_seed_search_report` v2 contract requires:
  - `contract_version`
  - `run_tag`
  - `policy_sha256`
  - `search_profile_set`
  - `selected_profile`
  - `dataset_fingerprint`
  - `code_revision_hint`

## Deep Review Fix Log

### Session 1 (Fixes applied to request_contract_gate.py, train_select_evaluate.py)

**Fix 1 — `request_contract_gate.py`: wrong error code in `validate_feature_engineering_report_shape`**
- The `except` block for JSON parse failure used `feature_group_spec_missing_or_invalid` instead of `feature_engineering_report_invalid`.
- Fixed: error code now correctly reflects `feature_engineering_report_invalid`.

**Fix 2 — `train_select_evaluate.py`: misleading hard-coded CI bounds in `transport_drop_ci`**
- `ci_95` and `ci_width` in the transport drop block were hard-coded to `[0.0, 0.0]` / `0.0`, falsely implying CIs were bootstrapped.
- Fixed: replaced with `null` and added `ci_note: "not_computed_point_estimate_only"`.
- Verified: `ci_matrix_gate.py` independently recomputes these CIs from prediction traces; downstream not affected.

### Session 2 (Fixes applied to feature_engineering_audit_gate.py, generalization_gap_gate.py, robustness_gate.py, seed_stability_gate.py)

**Fix 3 — `feature_engineering_audit_gate.py`: wrong error code for `feature_engineering_report` parse failure**
- Mirror of Fix 1: the `except` block used `feature_group_spec_missing_or_invalid` when parsing `feature_engineering_report` JSON.
- Fixed: error code now correctly set to `feature_engineering_report_invalid`.

**Fix 4 — `feature_engineering_audit_gate.py`: `to_float` missing `math.isfinite` guard**
- `to_float` accepted `inf` and `nan` as valid float values, inconsistent with all other gate scripts.
- Fixed: added `math.isfinite` guard and added `import math`.

**Fix 5 — `generalization_gap_gate.py`: `finish()` ignored `--strict` for warning escalation**
- `should_fail = bool(failures)` silently swallowed warnings even in strict mode.
- Fixed: `should_fail = bool(failures) or (args.strict and bool(warnings))`.

**Fix 6 — `robustness_gate.py`: same strict-mode bug as Fix 5**
- Fixed: `should_fail = bool(failures) or (args.strict and bool(warnings))`.

**Fix 7 — `seed_stability_gate.py`: same strict-mode bug as Fix 5**
- Fixed: `should_fail = bool(failures) or (args.strict and bool(warnings))`.

### Verified clean (no bugs found)
- `execution_attestation_gate.py`: `finish()` already correct; all validation logic and key/timestamp/transparency/receipt/log/witness-quorum checks are robust.
- `generalization_gap_gate.py`: `to_float` already had `math.isfinite`.
- All 27 gate scripts now uniformly use `bool(failures) or (args.strict and bool(warnings))` in `finish()`.
- All 11 `to_float` implementations across gate scripts now reject `inf`/`nan`.

## Agent Skill Protocol (Agent 技能协议)

本节定义 AI Agent 如何使用本项目作为 skill 快速构建和审计医疗 ML 项目。

### 三种操作模式

#### 模式 A：从零构建科研项目 (Build)
当用户说"帮我做一个预测模型"或"build a medical prediction project"时：

**标准化 8 步流程**：
```
Step 1: 环境检查     → python3 scripts/mlgg.py doctor
Step 2: 项目初始化   → python3 scripts/mlgg.py init --project-root <dir>
Step 3: 数据准备     → 下载数据集或放入用户数据，用 split_data.py 分割
Step 4: 配置对齐     → 确保 request.json + 所有 spec 文件正确
Step 5: 模型训练     → python3 scripts/mlgg.py train ...
Step 6: 执行认证     → python3 scripts/generate_execution_attestation.py ...
Step 7: 严格审计     → python3 scripts/mlgg.py workflow --strict
Step 8: 质量报告     → python3 scripts/quick_summary.py + python3 scripts/audit_external_project.py
```

**Agent 决策点**：
- Step 3 数据不足 (<100行)？→ 警告并建议更大数据集
- Step 5 候选模型不足？→ 自动扩大 model-pool
- Step 7 某个 gate 失败？→ 查询 `references/error-knowledge-base.json` 定位修复方案
- Step 8 得分 <90？→ 生成 remediation_plan 并逐项修复

#### 模式 B：审计他人项目 (Audit)
当用户说"帮我审查这个项目"或"review this ML project"时：

```bash
# 1. 量化评分
python3 scripts/audit_external_project.py --project-dir <dir> --target-journal nature_medicine --json

# 2. 如果已有 evidence 目录，运行完整 gate
python3 scripts/report_health_check.py --evidence-dir <dir>/evidence

# 3. 生成修复计划
python3 scripts/remediation_plan.py --evidence-dir <dir>/evidence
```

**审计输出**：12 维度量化评分 (满分100) + 期刊差距分析 + 优先修复清单

#### 模式 C：增量修复 (Fix)
当某个 gate 失败时：

```
1. 读取 gate report JSON → 提取 failure codes
2. 在 references/error-knowledge-base.json 中查找 → 获取修复方案
3. 如果找不到 → 诊断根因 → 应用修复 → 追加到 error-knowledge-base.json
4. 重跑失败的 gate → 验证通过
5. 重跑 publication_gate → 验证全链路通过
```

#### 模式 D：LLM 评审 Skill（零部署，带自己的 LLM）
当用户说"帮我生成评审 prompt"、"我想用 ChatGPT/Gemini 评审" 或 "export review prompt"时：

```bash
# 1. 快速红线检查 prompt（18条，粘贴到任意 LLM）
python3 scripts/export_review_prompt.py --level quick --output review_prompt_quick.md

# 2. 标准评审 prompt（53条）
python3 scripts/export_review_prompt.py --level standard --output review_prompt.md

# 3. 顶刊级 prompt，附 Nature Medicine 特定要求
python3 scripts/export_review_prompt.py --level comprehensive \
  --journal nature_medicine --output review_prompt_nm.md

# 4. JSON 格式（适合 API 调用）
python3 scripts/export_review_prompt.py --level standard --format json \
  --journal jama --output review_payload.json

# 5. 附文献引用
python3 scripts/export_review_prompt.py --level comprehensive \
  --include-literature --output review_with_refs.md
```

用法：将生成的 `.md` 文件内容粘贴到任意 LLM 对话框（Claude、GPT-4、Gemini 均可），然后粘贴论文 PDF 的文字内容，LLM 将输出结构化 JSON 评分报告。

**支持的期刊** `--journal` 参数：`nature_medicine` · `jama` · `lancet_digital_health` · `bmj` · `npj_digital_medicine`

#### 模式 E：批量评审 (Batch Review)
当用户说"帮我批量评审"或"review these projects"时：

```bash
# 1. 准备评审清单 (参考 references/batch-manifest.example.json)
# 2. 运行批量评审
python3 scripts/mlgg.py batch-review \
  --manifest batch_manifest.json \
  --target-journal nature_medicine \
  --workers 4 \
  --format json \
  --output batch_report.json

# 3. 可选：输出 CSV 摘要
python3 scripts/mlgg.py batch-review \
  --manifest batch_manifest.json \
  --summary-csv batch_summary.csv
```

**批量评审输出**：
- 对比矩阵：每个项目的 12 维度评分 + 总分 + 等级
- 跨项目分析：最常失败的维度 + 最普遍的差距
- 聚合修复优先级：去重后按严重性 × 影响项目数排序

**文献检索协议**：
- 查询 `references/literature-knowledge-base.json`（30 条顶刊文献）
- 按类别 (`category`)、实现的门控 (`gates_implementing`)、影响维度 (`dimensions_affected`) 搜索
- 在评审报告中引用 `LIT-NNN` 编号
- 新增文献须符合：IF>10 期刊 / EQUATOR 指南 / PRISMA 系统评价

### 12 维度量化评分标准 (100分制)

用于量化评判任何医疗 ML 项目的质量：

| # | 维度 | 权重 | 评分要点 |
|---|------|------|---------|
| 1 | 数据完整性 | 12 | Split 隔离、患者级不重叠、时序有序、无行重叠 |
| 2 | 防泄漏 | 15 | 无目标泄漏、无定义变量泄漏、无谱系泄漏、无未来特征 |
| 3 | 流水线隔离 | 12 | 预处理器仅在训练集 fit、插补器隔离、重采样仅在训练集 |
| 4 | 模型选择严谨性 | 10 | 候选池≥3、one-SE 规则、不窥探测试集、有基线比较 |
| 5 | 统计有效性 | 12 | Bootstrap CI、置换检验、校准、DCA、指标一致性 |
| 6 | 泛化证据 | 10 | Train-test gap、外部队列、Transport-drop CI、种子稳定性 |
| 7 | 临床完整性 | 7 | 完整指标面板、混淆矩阵一致性、阈值可行性 |
| 8 | 报告标准 | 7 | TRIPOD+AI、PROBAST+AI、STARD-AI、排除标准文档、局限性文档 |
| 9 | 可重复性 | 6 | 种子记录、版本追踪、执行认证、清单锁定 |
| 10 | 安全与溯源 | 3 | 模型签名、工件完整性、敏感数据保护 |
| 11 | 公平性与公正 | 3 | 均等化优势差距、差异影响比率、亚组性能最低标准 |
| 12 | 样本量充分性 | 3 | EPV≥10、收缩因子≥0.90、最小事件/非事件数≥100 |

**评分解读**：
- 90-100: 顶刊级 (Publication-grade) — 可直接投稿 Nature Medicine / Lancet DH / JAMA / BMJ
- 75-89: 有基础但需补充 (Solid but gaps) — 需要补充特定维度
- 60-74: 重大缺陷 (Major issues) — 需要系统性修复
- <60: 不可发表 (Not publishable) — 需要重新设计

### 顶刊级标准映射

各顶级期刊的核心要求已映射到本框架的 gate：
- 详见 `references/journal-rigor-standards.json`
- 支持期刊：Nature Medicine, Lancet Digital Health, JAMA, BMJ, npj Digital Medicine
- Agent 可自动运行差距分析：`audit_external_project.py --target-journal <name>`

### 自改进错误知识库协议

本项目维护一个结构化的错误模式数据库 (`references/error-knowledge-base.json`)：

**Agent 操作规范**：
1. 遇到新错误 → 先查知识库是否已有记录
2. 已有记录 → 按 `fix` 字段操作 → 验证修复
3. 未找到 → 诊断根因 → 应用修复 → 验证 → 追加新条目（ERR-NNN 格式）
4. 提交：`git commit -m "knowledge-base: add ERR-NNN <description>"`

**条目结构**：
```json
{
  "id": "ERR-NNN",
  "code": "error_code_string",
  "symptom": "用户看到的症状",
  "root_cause": "根因分析",
  "fix": "具体修复步骤",
  "prevention": "如何预防此类问题",
  "category": "data|leakage|pipeline|model|gate|config|environment|attestation|security|statistical",
  "severity": "CRITICAL|ERROR|WARNING|INFO",
  "affected_files": ["file1.py"],
  "first_seen": "YYYY-MM",
  "resolved": true
}
```

### Agent 快速参考卡

```
┌─────────────────────────────────────────────────────────────┐
│  ML Leakage Guard — Agent Quick Reference                   │
├─────────────────────────────────────────────────────────────┤
│  构建新项目:  python3 scripts/mlgg.py onboarding --mode auto│
│  审计项目:    python3 scripts/audit_external_project.py     │
│  错误查询:    references/error-knowledge-base.json          │
│  期刊标准:    references/journal-rigor-standards.json       │
│  修复计划:    python3 scripts/remediation_plan.py           │
│  健康检查:    python3 scripts/report_health_check.py        │
│  证据对比:    python3 scripts/evidence_comparator.py        │
│  阈值敏感:    python3 scripts/threshold_sensitivity.py      │
│  LaTeX导出:   python3 scripts/export_latex.py               │
├─────────────────────────────────────────────────────────────┤
│  评分工具:    audit_external_project.py --target-journal X  │
│  支持期刊:    nature_medicine | lancet_digital_health |     │
│               jama | bmj | npj_digital_medicine             │
├─────────────────────────────────────────────────────────────┤
│  Gate 失败?   1. 读报告 2. 查知识库 3. 修复 4. 重跑         │
│  得分 <90?    1. 运行 remediation_plan 2. 逐项修复          │
│  新增错误?    追加到 error-knowledge-base.json               │
└─────────────────────────────────────────────────────────────┘
```

### 标准化交付物清单 (Publication-Ready Deliverables)

Agent 完成完整流程后应产出以下交付物：

```
<project>/
├── data/
│   ├── train.csv, valid.csv, test.csv          # 分割后数据
│   └── external_*.csv                          # 外部验证队列
├── configs/
│   ├── request.json                            # 实验请求合同
│   ├── execution_attestation.json              # 执行认证规范
│   └── *.json                                  # 各类 spec 文件
├── evidence/
│   ├── *_report.json (×31)                     # 31 个 gate 报告
│   ├── manifest.json                           # SHA256 工件清单
│   ├── prediction_trace.csv.gz                 # 行级预测追踪
│   ├── evaluation_report.json                  # 评估指标报告
│   ├── model_selection_report.json             # 模型选择报告
│   └── audit_report.json                       # 12维量化审计报告
├── models/
│   ├── model.pkl + model.pkl.sig               # 签名模型工件
│   └── .mlgg_model_key                         # HMAC 密钥
├── keys/
│   └── *.pem                                   # 认证密钥对
└── results/
    ├── summary.md                              # 人类可读摘要
    └── tables.tex                              # LaTeX 表格
```

登录后下载/点赞/收藏 ❤ 8 | ★ 0

ml-leakage-guard

目录结构

SKILL.md

举报内容

提示