col_vals_decreasing()`method`

Are column data decreasing by row?

USAGE

Validate.col_vals_decreasing(
    columns,
    allow_stationary=False,
    increasing_tol=None,
    na_pass=False,
    pre=None,
    segments=None,
    thresholds=None,
    actions=None,
    brief=None,
    active=True,
)

The col_vals_decreasing() validation method checks whether column values in a table are decreasing when moving down a table. There are options for allowing missing values in the target column, allowing stationary phases (where consecutive values don’t change), and even one for allowing increasing movements up to a certain threshold. This validation will operate over the number of test units that is equal to the number of rows in the table (determined after any pre= mutation has been applied).

Parameters

columns : str | list[str] | Column | ColumnSelector | ColumnSelectorNarwhals: A single column or a list of columns to validate. Can also use col() with column selectors to specify one or more columns. If multiple columns are supplied or resolved, there will be a separate validation step generated for each column.
allow_stationary : bool = False: An option to allow pauses in decreasing values. For example, if the values for the test units are [88, 85, 85, 82, 80] then the third unit (85, appearing a second time) would be marked as failing when allow_stationary is False. Using allow_stationary=True will result in all the test units in [88, 85, 85, 82, 80] to be marked as passing.
increasing_tol : float | None = None: An optional threshold value that allows for movement of numerical values in the positive direction. By default this is None but using a numerical value will set the absolute threshold of positive travel allowed across numerical test units. Note that setting a value here also has the effect of setting allow_stationary to True.
na_pass : bool = False: Should any encountered None, NA, or Null values be considered as passing test units? By default, this is False. Set to True to pass test units with missing values.
pre : Callable | None = None: An optional preprocessing function or lambda to apply to the data table during interrogation. This function should take a table as input and return a modified table. Have a look at the Preprocessing section for more information on how to use this argument.
segments : SegmentSpec | None = None: An optional directive on segmentation, which serves to split a validation step into multiple (one step per segment). Can be a single column name, a tuple that specifies a column name and its corresponding values to segment on, or a combination of both (provided as a list). Read the Segmentation section for usage information.
thresholds : int | float | bool | tuple | dict | Thresholds = None: Set threshold failure levels for reporting and reacting to exceedences of the levels. The thresholds are set at the step level and will override any global thresholds set in Validate(thresholds=...). The default is None, which means that no thresholds will be set locally and global thresholds (if any) will take effect. Look at the Thresholds section for information on how to set threshold levels.
actions : Actions | None = None: Optional actions to take when the validation step(s) meets or exceeds any set threshold levels. If provided, the Actions class should be used to define the actions.
brief : str | bool | None = None: An optional brief description of the validation step that will be displayed in the reporting table. You can use the templating elements like "{step}" to insert the step number, or "{auto}" to include an automatically generated brief. If True the entire brief will be automatically generated. If None (the default) then there won’t be a brief.
active : bool = True: A boolean value indicating whether the validation step should be active. Using False will make the validation step inactive (still reporting its presence and keeping indexes for the steps unchanged).

Returns

Validate: The Validate object with the added validation step.

Examples

For the examples here, we’ll use a simple Polars DataFrame with a numeric column (a). The table is shown below:

import pointblank as pb
import polars as pl

tbl = pl.DataFrame(
    {
        "a": [6, 5, 4, 3, 2, 1],
        "b": [5, 4, 4, 3, 2, 1],
        "c": [5, 4, 5, 3, 2, 1],
    }
)

pb.preview(tbl)

	a Int64	b Int64	c Int64
1	6	5	5
2	5	4	4
3	4	4	5
4	3	3	3
5	2	2	2
6	1	1	1

Let’s validate that values in column a are decreasing. We’ll determine if this validation had any failing test units (there are six test units, one for each row).

validation = (
    pb.Validate(data=tbl)
    .col_vals_decreasing(columns="a")
    .interrogate()
)

validation

		STEP	COLUMNS	VALUES	TBL	EVAL	UNITS	PASS	FAIL	W	E	C	EXT
#4CA64C	1	col_vals_decreasing()	a			✓	6	6 1.00	0 0.00	—	—	—	—

The validation passed as all values in column a are decreasing. Now let’s check column b which has a stationary value:

validation = (
    pb.Validate(data=tbl)
    .col_vals_decreasing(columns="b")
    .interrogate()
)

validation

		STEP	COLUMNS	VALUES	TBL	EVAL	UNITS	PASS	FAIL	W	E	C	EXT
#4CA64C66	1	col_vals_decreasing()	b			✓	6	5 0.83	1 0.17	—	—	—

This validation fails at the third row because the value 4 is repeated. If we want to allow stationary values, we can use allow_stationary=True:

validation = (
    pb.Validate(data=tbl)
    .col_vals_decreasing(columns="b", allow_stationary=True)
    .interrogate()
)

validation

		STEP	COLUMNS	VALUES	TBL	EVAL	UNITS	PASS	FAIL	W	E	C	EXT
#4CA64C	1	col_vals_decreasing()	b			✓	6	6 1.00	0 0.00	—	—	—	—