Skip to content

Pipeline Overview

The AutoClean pipeline is a comprehensive EEG processing system that automates the preprocessing and artifact rejection workflow. This document provides a high-level overview of the pipeline stages.

Pipeline Stages

1. Data Import

  • Raw data loading
  • Channel configuration
  • Montage setup
  • BIDS conversion

2. Preprocessing

  • Resampling
  • Channel type marking
  • Data cropping
  • Reference setting
  • Bad channel detection

3. Artifact Rejection

  • ICA decomposition
  • Component classification
  • Muscle artifact detection
  • Automated rejection
  • Quality metrics calculation

4. Quality Control

  • PSD analysis
  • Topographical mapping
  • Report generation
  • Data validation

Pipeline Flow

graph TD
    A[Raw Data] --> B[Data Import]
    B --> C[Preprocessing]
    C --> D[Artifact Rejection]
    D --> E[Quality Control]
    E --> F[Final Output]

    subgraph "Data Import"
    B1[Load Raw] --> B2[Configure Channels]
    B2 --> B3[Setup Montage]
    B3 --> B4[BIDS Convert]
    end

    subgraph "Preprocessing"
    C1[Resample] --> C2[Mark Channels]
    C2 --> C3[Crop Data]
    C3 --> C4[Set Reference]
    end

    subgraph "Artifact Rejection"
    D1[ICA] --> D2[Component Classification]
    D2 --> D3[Muscle Detection]
    D3 --> D4[Auto Rejection]
    end

    subgraph "Quality Control"
    E1[PSD Analysis] --> E2[Topomaps]
    E2 --> E3[Reports]
    E3 --> E4[Validation]
    end

Key Functions

Data Import

  • step_import_raw(): Import and configure raw EEG data
  • step_convert_to_bids(): Convert data to BIDS format

Preprocessing

  • step_resample_data(): Resample the data
  • step_mark_eog_channels(): Mark EOG channels
  • step_crop_data(): Crop the data
  • step_set_reference(): Set EEG reference

Artifact Rejection

  • step_clean_bad_channels(): Detect and interpolate bad channels
  • step_run_pylossless(): Run PyLossless pipeline
  • step_run_ll_rejection_policy(): Apply rejection criteria

Quality Control

  • step_psd_topo_figure(): Generate PSD and topographical plots
  • plot_bad_channels_with_topography(): Visualize bad channels
  • generate_ica_reports(): Create ICA component reports

Output Structure

The pipeline generates a structured output directory:

output/
├── raw/
│   └── sub-{participant}/
├── preprocessed/
│   └── sub-{participant}/
├── artifacts/
│   └── sub-{participant}/
└── reports/
    └── sub-{participant}/

Error Handling

The pipeline includes comprehensive error handling:

  • Input validation
  • Configuration verification
  • Processing stage monitoring
  • Error logging and reporting

Performance Considerations

  • Memory usage optimization
  • Parallel processing capabilities
  • Efficient data handling
  • Progress tracking and logging