• Roque's avatar
    erp5_wendelin_data_lake_ingestion: split files are no longer... · aca468c6
    Roque authored
    erp5_wendelin_data_lake_ingestion:  split files are no longer appended/processed and no data streams are removed anymore
    - all ingestions and data streams corresponding to split parts are kept
    - client will receive the list of all data streams and it will be in charge of merging the parts during the download
    - validate chunk data streams only when full file was ingested
    - only process split ingestions when full file was ingested
    - calculate full split file size
    - calculate hash and add state control during data stream validation
    - stop invalidated ingestions
    aca468c6
ERP5Site_stopIngestionList.py 5.48 KB