• Douglas's avatar
    Inventory Pandas: added an initial prototype of a Pandas-based Inventory API · 0ba29803
    Douglas authored
    The implementation relies on the Data Array Module. It imports data from the
    stocks table through a zSQL Method. Category information is added later in a
    column-wise way, so it can be easily done in parallel and query Portal Catalog
    once for each category column in the array. This category processing needs to be
    done only once, when the array is created, and to new data as it is added.
    
    But there is a catch: each entity that belongs to the movement can have many
    categories. So the row can be duplicated for each entity's categories and
    searched by equality, or they can be stored as comma-separated values and
    searched using a regular expression. Regular expression seems faster for
    datasets up to 1M rows.
    
    Some unit tests were also added.
    
    These are the external methods created and their purposes:
    
    - Base_filterInventoryDataFrame is there just to parse keyword arguments and
    forward them to Base_getInventoryDataFrame. This is used for the non-programmer
    interface of Pandas-based getMovementHistoryList implementation and can be
    used as an external method in other scripts too.
    
    - Base_convertResultsToBigArray will convert results of Portal Catalog and ZSQL
    Method to a Data Array with a proer transformation of the schema to a
    compatible NumPy data type.
    
    - Base_extendBigArray will extend a Data Array with a Portal Catalog query or
    ZSQL Method result. Raise errors when the extension data type doesn't match
    the source.
    
    - Base_fillPandasInventoryCategoryList will fill category information in a Data
    Array which has stock movements information.
    
    -Base_zGetStockByResource is used in a test case as source to create a Data
    Array with stock data.
    0ba29803
extension.erp5.PandasInventory.py 23.3 KB