Making "modified_object_list" consistent by always containing lists and not
sometimes lists and sometimes tuples.

As now, we expect our catalog object to be erp5 catalog and not indexable,
so its important to not reindex or unindex the old object.

Required as most of the changes we have are in erp5_core and that's
why its important to install/update it before we install erp5_catalog_storage
bt5.

For compatibilty, we have added 2 extra functions, but in no way we need
to use this attribute as it doesn't give any meaning for ERP5CatalogTool
to have a default SQL Catalog.

This would increase compatibility between accessors in erp5_catalog and
remove the need of copy and patch. At the same time, it also helps in
increasing performance as using _getOb is always less costly than
getattr, as it reduces the need to look in acquisition.

One of the difference between SQL and ERP5 Catalog is how they
handle their properties. For ERP5Catalog, we use property_sheets
which generate setters and getters, for SQLCatalog, properties
just act as attributes(as one can expect from a property set
on a class).

This creates a difference on how we use them, especially for
list_type properties, which have accessors like get<PropertyName>List
to get the list of objects. For SQLCatalog, there is no such thing.
This creates problem whereever 'get<PropertyName>' or getProperty(<name>)
was being used to get multiple property types.

After adding document component object, _p_oid was bing generated at the
last step of commit hook. This led to problem that _registry_dict for
dynamic_class 'erp5.component.document' wasn't getting updated with
the _p_oid of the component created. This led to failing of test because
to validate object with same reference, the checkConsistency function
checks in the _registry_dict for reference and _p_oid.

Explicilty calling commit here generates the oid before validation, hence
make it available for the registry_dict in time.

But, this clearly is a workaround which is clearly not solving the real
problem of why the _p_oid isn't being generated at the right step.

Also, me(Ayush), tried to recreate this error on a running instance, but
it worked fine there.

Earlier, class for any Catalog object used to be Products.ZSQLCatalog.SQLCatalog,
but now as we have shifted SQLCatalog to ERP5Catalog, the catalog objects have
lazy_class 'erp5.portal_type.Catalog' as their 1st class in mro. This class
don't have the required attributes.

The lazy_class is dynamic, so its not possible for them to save reserved IDs
as their attributes. So, we use the next class in mro, which would be
class 'Products.ERP5Catalog.ERP5Catalog' and save attributes in it.

Notice that this change of 'self.__klass__' is not required if
we are starting a lock with '_reserved_uid_lock' and then acquire-release it
consicutively in one go(transaction).

And, Patch changeObjectClass extension to remove useless attributes

Copying __dict__ from one object to another brings us to situation where
we don have many objects which we don't need at all, for example, migrating
objects with subclasses who were initially OFS objects and later an ERP5
object can lead to adding subobjects as attributes of the new object, which
is completely undesirable. To handle this, it is important to delete the
sub-objects as the attributes for those migrated classes.

Old Catalog Tool didn't have portal_type attribute, so while migrating
via synchronizeDynamicModule, after _bootstrap, we expect the tool to
have a portal_type to finalize migration.

This step is now being done only at the end of _bootstrap after we
change the classes for portal_catalog and its sub-objects.

Use filter attributes as property. Same as done for ERP5 SQL Methods.
This way maintains consistency between sub-objects of Catalog, i.e,
SQL Method and Python Script.

Contains views, property sheets,  portal types concerning migration of
portal_catalog, sql catalog and their subobjects to erp5 object.

Portal Types:
1. Catalog : ERP5 Catalog object (an ERP5 Folder), which earlier used to be OFS Folder.
	Actions:
		Update Catalog
		Clear Catalog
		Clear Reserved
		Export Properties

2. Catalog Tool: Portal Catalog where we can add and use multiple ERP5 catalogs.
	Actions:
		Hot ReindexAll

3. SQL Method: SQL methods with their views inside erp5.
	Actions:
		Run Method

Property Sheet:
1. Catalog
2. CatalogTool
3. SQLMethod
	Containing properties for the various portal_types/classes respectively.
4. CatalogFilter

Also, filter_dict for erp5_catalog would now not be a Persistent Mapping
object. The info inside filter_dict is being saved inside the SQL Method
objects as their properties.

Views:
1. Catalog (View)
2. CatalogTool(View, Properties, Filtered Items), Object Actions
3. SQLMethod (View, Filter)
4. Python Script (Filter)

Extras:

	- Dialog view for catalog before clear_catalog

	- Add typeBaseMethod(s) for PythonScript and SQLMethod portal types:
The methods <portal_type>_getRedirectParameterDictAfterAdd are required for these
portal_types, cause their __call__ methods are overridden, so after addition
of new objects for them, it was taking to the url concerning wherever __call__
methods directed them.
Now, we change them to redirect to 'abosulte_url+'/view'' which is basically
the homepage for these catalog methods.

	- Do not use default ERP5 Catalog to show properties:
Earlier, in tales for properties for any erp5 catalog, we showed
the values for the default_erp5_catalog. But keeping in mind that
we can have multiple catalogs at same time, the old approach
was wrong. Hence, we now display properties for the current catalog

	- Search fo Catalog_viewContentList and Catalog_viewFilterList

- This step is needed due to the use of BaseTool as Base class for CatalogTool
  due to which there were conflict between reindexObject from the Base and the one
  from the BaseTool.

	- Remove copy-pasting all code from CatalogTool, better to rely on inheritence
	- Remove unnecessary imports
	- Add argument id in __init__ class
	- Add functions _isBootstrapRequired and _bootstrap
	- Update BusinessTemplate installation according to changes made in ERP5Catalog and Tool
	- Explicilty add manage option tabs in Catalog Tool

- Update testCopySupport according to changes in portal_catalog

Its better to change the tests where they need to call getpath function of
portal_catalog to use portal_catalog.getpath instead of portal_catalog.getPath,
as we have overridden the 'getPath' function for CatalogTool class due to change
in inherited class.

Move from SQLCatalog to ERP5Catalog as the default Catalog inside ERP5.
The major difference is use of Products.ERP5Type.Core.Folder as Catalog
base class.

Significant changes:
	-Inherit from Catalog class from Products.ZSQLCatalog.SQLCatalog instead of copy-pasting the whole code again.
	-Add allowed_types for ERP5Catalog tool
	-Monkey patch some property setters and getters to maintain consistency
	-Update id and title for ERP5Catlog while class initialization
	-Set declarative securities and solve some inheritance conflicts
	-Add isRADContent for ERP5Catalog Class
 	-Solve inheritence conflict for _setPropValue function in ERP5Catalog class
	-Add SQL Method portal_type in allowed_types for ERP5Catalog class
	-Override getCatalogMethodIds cause it uses global variable in SQLCatalog.Catalog
	-Redefine security declarations
	-Add functions for object_actions of Catalog portal_type in ERP5Catalog object
	-Add filter_dict attribute for compatibilty

Also,
- Update BusinessTemplate installation with updated filter_dict
This removes the need to copy-patch or if-else on meta_type of catalog.
Use dynamic migration while installing the catalog method objects for
bt5.
- Update tests according to changes in portal_catalog
- Create FilterDict and Filter class which would be used to imitate the behaviour
of filter_dict for Catalog.

Create the SQLMethod class based on ZSQLMethods.SQL
class and XMLObject. Also, move attributes to
property in 'SQL Method' property sheets.

Also, we don't need to look for `transactionless` connection-id here, as we don't use them in
catalog methods.

And, update the files where we can use the new function.

/cc @aurel 

/reviewed-on nexedi/erp5!411

To generate (and execute) SQL, use catalog tool.

/reviewed-on nexedi/erp5!408

This reverts commit 206fa603 (which was
itself a revert commit), re-applying the change now that surrounding
code is ready for it.

Ignored columns are produced when aliasing a column. For example,
aliasing "catalog.reference" as "reference".
Before this change, this would cause conditions on "reference" to be
rendered non-mapped, which can cause SQL execution issues when there is
more than one "reference" column available (catalog.reference and its
alias counting as only one), which is the case when
catalog-category-catalog joins happen.

Instead, render all columns which could be mapped, independently from
their "ignored" status.

Also, use a different local variable for table aliases than for column
aliases.
Also, use more "return" statements, and simplify conditional structure.

As per Jérome, who implemented the test, it was written to test the
current state rather than testing the desired outcome. And it makes
little sense to have (and test for) 100 being present in both debit and
credit columns ("normal" lines), and 0 to be present in the stat line.

Update test to check for a more consistent outcome.
Acked-by: Jérome Perrin <jerome@nexedi.com>

Explicitely state which values represent empty values. Coercing to boolean is not sufficient.

-  Remove field_json.value because that one is never send by ERP5 backend
-  Set comprehensive initial state and avoid sneaking state variables afterwards
-  Handle better NaNs which represent empty numerical value
-  Refactor for shorter and simpler code
-  Rename "percents" -> "percentage" according to coding style guidelines