full_process()

Process items through a configurable pipeline.

Usage

full_process(
    items,
    pipeline=None,
    verbose=False,
    max_workers=1,
    timeout=30.0,
)

Applies each stage in pipeline to the items sequentially (or in parallel if max_workers > 1).

Parameters

items: list

List of items to process.

pipeline: list = None

List of callable stages. Each receives items and returns modified items. If None, uses a default pipeline.

verbose: bool = False

If True, print progress information.

max_workers: int = 1

Number of parallel workers. Use 1 for sequential.

timeout: float = 30.0

Maximum processing time in seconds per stage.

Returns

dict

Processing results with keys:

• "items" — processed items
• "stages_run" — number of stages executed
• "elapsed" — total time in seconds

Raises

TimeoutError

If any stage exceeds the timeout.

RuntimeError

If a pipeline stage fails.

Notes

Parallel processing uses a thread pool. For CPU-bound stages, consider using max_workers=1 to avoid GIL contention.

Examples

>>> full_process([1, 2, 3])
{'items': [1, 2, 3], 'stages_run': 0, 'elapsed': 0.0}

>>> full_process([1, 2, 3], pipeline=[str], verbose=True)
Processing stage 1/1...
{'items': ['1', '2', '3'], 'stages_run': 1, 'elapsed': 0.0}