slitflow.setreqs module

This module includes functions used in the set_reqs method of the Data class. The required data must be sorted to align the correspondence between the data.

allocate_data(reqs, param)

run_cycle(Data, reqs, param, load_splits, data_splits, run_mode)

make_index_list(reqs)

Make index list from reqs. Initial data: [“_file”] Loaded data: [“_file”, “_split”, “_keep”]

make_splits_list(reqs, load_splits=None, data_splits=None)

Generate a list of splits specifications for each request.

validate_splits(splits_list)

Validate the list of splits to ensure they meet specific criteria.

add_splits(index_list, splits_list)

Add split columns to a DataFrame based on the provided split values.

get_index_columns(index_list, splits_list)

Extract index columns based on data split values.

validate_index_columns(index_data_col_list, index_col_max_list, splits_list)

Validate the consistency of index columns based on the provided lists.

filter_and_merge_index_data(index_list, splits_list, index_col_max, save_split=None)

Filter and merge index based on the provided lists of index and splits.

add_mrg_id(merged_df, index_list, index_data_col_list)

Add a ‘_mrg_id’ column to each DataFrame in index_list.

make_index_dest_setreq(index_list)

Make a list of index DataFrames with ‘_dest’ column added.

merge_index_data(index_list)

Merge and sort index_list based on a common ‘_mrg_id’ column

make_load_index(merged_index)

Extract columns containing ‘_load’ and compute ‘_cycle’ and ‘_dest’

make_index_cycle(load_index, index_list)

Merge each index DataFrame with the load index and make cycle_no list.

make_save_no_list(load_index, merged_index_data)

Create a list of ‘_save’ values.

make_index_dest_cycle(index_split, index_data_cycle)

Add the _dest column to index_split.