Skip to content

M
MariaDB
Commit e4d72dfa authored by joreland@mysql.com's avatar joreland@mysql.com
Browse files
Options

ndb - Update documentation wrt scans

parent b8ce3833
Hide whitespace changes
Inline Side-by-side
Showing with 471 additions and 515 deletions
ndb/include/ndbapi/Ndb.hpp
View file @ e4d72dfa
...@@ -52,15 +52,6 @@ ...@@ -52,15 +52,6 @@
The execution can be of two different types, The execution can be of two different types,
<var>Commit</var> or <var>NoCommit</var>. <var>Commit</var> or <var>NoCommit</var>.
*/
#ifndef DOXYGEN_SHOULD_SKIP_INTERNAL
/**
(A transaction's execution can also be divided into three
steps: prepare, send, and poll. This allows us to perform asynchronous
transactions. More about this later.)
*/
#endif
/**
If the execution is of type <var>NoCommit</var>, If the execution is of type <var>NoCommit</var>,
then the application program executes part of a transaction, then the application program executes part of a transaction,
but without committing the transaction. but without committing the transaction.
...@@ -94,28 +85,13 @@ ...@@ -94,28 +85,13 @@
To execute several parallel synchronous transactions, one can either To execute several parallel synchronous transactions, one can either
use multiple <code>Ndb</code> objects in several threads, or start multiple use multiple <code>Ndb</code> objects in several threads, or start multiple
applications programs. applications programs.
*/
#ifndef DOXYGEN_SHOULD_SKIP_INTERNAL
/**
Another way to execute several parallel transactions is to use
asynchronous transactions.
*/
#endif
/**
@section secNdbOperations Operations @section secNdbOperations Operations
Each <code>NdbTransaction</code> (that is, a transaction) Each <code>NdbTransaction</code>
consists of a list of operations which are represented by instances consists of a list of operations which are represented by instances
of <code>Ndb*Operation</code>. of <code>Ndb*Operation</code>.
*/
#ifndef DOXYGEN_SHOULD_SKIP_INTERNAL
/**
Operations are of two different kinds:
-# standard operations, and
-# interpreted program operations.
*/
#endif
/**
<h3>Single row operations</h3> <h3>Single row operations</h3>
After the operation is created using <code>NdbTransaction::getNdbOperation()</code> After the operation is created using <code>NdbTransaction::getNdbOperation()</code>
(or <code>NdbTransaction::getNdbIndexOperation()</code>), it is defined in the following (or <code>NdbTransaction::getNdbIndexOperation()</code>), it is defined in the following
...@@ -163,11 +139,8 @@ ...@@ -163,11 +139,8 @@
We will now discuss in somewhat greater detail each step involved in the creation We will now discuss in somewhat greater detail each step involved in the creation
and use of synchronous transactions. and use of synchronous transactions.
*/
// Edit stop point - JS, 20041228 0425+1000 // Edit stop point - JS, 20041228 0425+1000
/**
<h4>Step 1: Define single row operation type</h4> <h4>Step 1: Define single row operation type</h4>
The following types of operations exist: The following types of operations exist:
-# NdbOperation::insertTuple : -# NdbOperation::insertTuple :
...@@ -207,14 +180,13 @@ ...@@ -207,14 +180,13 @@
Normally the attribute is defined by its name but it is Normally the attribute is defined by its name but it is
also possible to use the attribute identity to define the also possible to use the attribute identity to define the
attribute. attribute.
The mapping from name to identity is performed by the Table object.
NdbIndexOperation::getValue returns an NdbRecAttr object NdbOperation::getValue returns an NdbRecAttr object
containing the read value. containing the read value.
To get the value, there is actually two methods. To get the value, there is actually two methods.
The application can either The application can either
- use its own memory (passed through a pointer aValue) to - use its own memory (passed through a pointer aValue) to
NdbIndexOperation::getValue, or NdbOperation::getValue, or
- receive the attribute value in an NdbRecAttr object allocated - receive the attribute value in an NdbRecAttr object allocated
by the NDB API. by the NDB API.
...@@ -224,367 +196,176 @@ ...@@ -224,367 +196,176 @@
Ndb::closeTransaction have been called. Ndb::closeTransaction have been called.
The result of reading data from an NdbRecAttr object before The result of reading data from an NdbRecAttr object before
calling NdbTransaction::execute is undefined. calling NdbTransaction::execute is undefined.
*/
#ifndef DOXYGEN_SHOULD_SKIP_INTERNAL
/**
<h3>Interpreted Program Operations</h3>
The following types of interpreted program operations exist:
-# NdbOperation::interpretedUpdateTuple :
updates a tuple using an interpreted program
-# NdbOperation::interpretedDeleteTuple :
delete a tuple using an interpreted program
-# NdbOperation::openScanRead :
scans a table with read lock on each tuple
-# NdbOperation::openScanExclusive :
scans a table with exclusive update lock on each tuple
The operations interpretedUpdateTuple and interpretedDeleteTuple both
work using the unique tuple key.
These <em>interpreted programs</em> @subsection secScan Scan Operations
make it possible to perform computations
inside the NDB Cluster Kernel instead of in the application Scans are roughly the equivalent of SQL cursors.
program.
This is sometimes very effective, since no intermediate results
are sent to the application, only the final result.
Scans can either be performed on a table (@ref NdbScanOperation) or
on an ordered index (@ref NdbIndexScanOperation).
<h3>Interpreted Update and Delete</h3> Scan operation are characteriesed by the following:
- They can only perform reads (shared, exclusive or dirty)
- They can potentially work with multiple rows
- They can be used to update or delete multiple rows
- They can operate on several nodes in parallell
Operations for interpreted updates and deletes must follow a After the operation is created using <code>NdbTransaction::getNdbScanOperation()</code>
certain order when defining operations on a tuple. (or <code>NdbTransaction::getNdbIndexScanOperation()</code>), it is defined in the following
As for read and write operations, three steps:
one must first define the operation type and then the search key. -# Define the standard operation type, using <code>NdbScanOperation::readTuples()</code>
-# The first step is to define the initial readings. -# Specify search conditions, using @ref NdbScanFilter and/or @ref NdbIndexScanOperation::setBound
In this phase it is only allowed to use the -# Specify attribute actions, using <code>NdbOperation::getValue()</code>
NdbOperation::getValue method. -# Executing the transaction, using <code>NdbTransaction::execute()</code>
This part might be empty. -# Iterating through the result set using <code>NdbScanOperation::nextResult</code>
-# The second step is to define the interpreted part.
The methods supported are the methods listed below except
NdbOperation::def_subroutine and NdbOperation::ret_sub
which can only be used in a subroutine.
NdbOperation::incValue and NdbOperation::subValue
increment and decrement attributes
(currently only unsigned integers supported).
This part can also be empty since interpreted updates
can be used for reading and updating the same tuple.
<p>
Even though getValue and setValue are not really interpreted
program instructions, it is still allowed to use them as
the last instruction of the program.
(If a getValue or setValue is found when an interpret_exit_ok
could have been issued then the interpreted_exit_ok
will be inserted.
A interpret_exit_ok should be viewed as a jump to the first
instruction after the interpreted instructions.)
-# The third step is to define all updates without any
interpreted program instructions.
Here a set of NdbOperation::setValue methods are called.
There might be zero such calls.
-# The fourth step is the final readings.
The initial readings reads the initial value of attributes
and the final readings reads them after their updates.
There might be zero NdbOperation::getValue calls.
-# The fifth step is possible subroutine definitions using
NdbOperation::def_subroutine and NdbOperation::ret_sub.
*/
#endif
/**
@subsection secScan Scanning
The most common use of interpreted programs is for scanning
tables. Scanning is a search of all tuples in a table.
Tuples which satisfy conditions (a search filter)
stated in the interpreted program
are sent to the application.
Reasons for using scan transactions include
need to use a search key different from the primary key
and any secondary index.
Or that the query needs to access so many tuples so that
it is more efficient to scan the entire table.
Scanning can also be used to update information.
The scanning transaction itself is however
not allowed to update any tuples.
To do updates via scanning transactions, the tuples
need to be handed over to another transaction which is
executing the actual update.
Even though a scan operation is part of a transaction,
the scan transaction is not a normal transaction.
The locks are <em>not</em> kept throughout the entire
scan transaction, since this would imply non-optimal performance.
<em>
A transaction containing a scan operation can only
contain that operation.
No other operations are allowed in the same transaction.
</em>
The NdbOperation::openScanRead operation
only sets a temporary read lock while
reading the tuple.
The tuple lock is released already when the
result of the read reaches the application.
The NdbOperation::openScanExclusive operation sets an
exclusive lock on the tuple
and sends the result to the application.
Thus when the application reads the data it is still
locked with the exclusive lock.
If the application desires to update the tuple it may transfer
the tuple to another transaction which updates the tuple.
The updating transaction can consist of a combination of tuples
received from the scan and normal operations.
For transferred operations it is not necessary to provide the
primary key. It is part of the transfer.
You only need to give the operation type and the
actions to perform on the tuple.
The scan transaction starts like a usual transaction,
but is of the following form:
-# Start transaction
-# Get NdbOperation for the table to be scanned
-# Set the operation type using NdbOperation::openScanRead or
NdbOperation::openScanExclusive
-# Search conditions are defined by an interpreted program
(setValue and write_attr are not allowed, since scan transactions
are only allowed to read information).
The instruction interpret_exit_nok does in this case
not abort the transaction, it only skips the tuple and
proceeds with the next.
The skipped tuple will not be reported to the application.
-# Call NdbTransaction::executeScan to define (and start) the scan.
-# Call NdbTransaction::nextScanResult to proceed with next tuple.
When calling NdbTransaction::nextScanResult, the lock on any
previous tuples are released.
<br>
If the tuple should be updated then it must be transferred over
to another updating transaction.
This is performed by calling
NdbOperation::takeOverForUpdate or takeOverForDelete on
the scanning transactions NdbOperation object with the updating
transactions NdbTransaction object as parameter.
<p>
If NdbOperation::takeOverFor* returns NULL then the
operation was not successful, otherwise it returns a reference
to the NdbOperation which the updating transaction has received
-# Use Ndb::closeTransaction as usual to close the transaction.
This can be performed even if there are more tuples to scan.
See also example program in section @ref select_all.cpp.
However, a new scan api is under development, using NdbScanOperation
and NdbScanFilter. NdbScanFilter makes it easier to define a search
criteria and is recommended instead of using Interpreted Programs.
The scan transaction starts like a usual transaction,
but is of the following form:
-# Start transaction
-# Get NdbScanOperation for the table to be scanned
-# NdbScanOperation::readTuples(NdbOperation::LM_Exclusive) returns a handle to a
NdbResultSet.
-# Search conditions are defined by NdbScanFilter
-# Call NdbTransaction::execute(NoCommit) to start the scan.
-# Call NdbResultSet::nextResult to proceed with next tuple.
When calling NdbResultSet::nextResult(false), the lock on any
previous tuples are released and the next tuple cached in the API
is fetched.
<br>
If the tuple should be updated then define a new update operation
(NdbOperation) using NdbResultSet::updateTuple().
The new update operation can the be used to modify the tuple.
When nextResult(false) returns != 0, then no more tuples
are cached in the API. Updated tuples is now commit using
NdbTransaction::execute(Commit).
After the commit, more tuples are fetched from NDB using
nextResult(true).
-# Use Ndb::closeTransaction as usual to close the transaction.
This can be performed even if there are more tuples to scan.
See the scan example program in @ref ndbapi_scan.cppn for example
usage of the new scan api.
*/
#ifndef DOXYGEN_SHOULD_SKIP_INTERNAL
/**
<h3>Interpreted Programs</h3>
Interpretation programs are executed in a
register-based virtual machine.
The virtual machine has eight 64 bit registers numbered 0-7.
Each register contains type information which is used both
for type conversion and for type checking.
@note Arrays are currently <b>not</b> supported in the virtual machine. Here are two brief examples illustrating this process. For the sake of brevity,
Currently only unsigned integers are supported and of size we omit error-handling.
maximum 64 bits.
This first example uses an <code>NdbScanOperation</code>:
@code
// 1. Create
MyOperation= MyTransaction->getNdbScanOperation("MYTABLENAME");
// 2. Define type of operation and lock mode
MyOperation->readTuples(NdbOperation::LM_Read);
All errors in the interpretation program will cause a // 3. Specify Search Conditions
transaction abort, but will not affect any other transactions. NdbScanFilter sf(MyOperation);
sf.begin(NdbScanFilter::OR);
sf.eq(0, i); // Return rows with column 0 equal to i or
sf.eq(1, i+1); // column 1 equal to (i+1)
sf.end();
The following are legal interpreted program instructions: // 4. Attribute Actions
-# incValue : Add to an attribute MyRecAttr= MyOperation->getValue("ATTR2", NULL);
-# subValue : Subtract from an attribute @endcode
-# def_label : Define a label in the interpreted program
-# add_reg : Add two registers
-# sub_reg : Subtract one register from another
-# load_const_u32 : Load an unsigned 32 bit value into a register
-# load_const_u64 : Load an unsigned 64 bit value into a register
-# load_const_null : Load a NULL value into a register
-# read_attr : Read attribute value into a register
-# write_attr : Write a register value into an attribute
-# branch_ge : Compares registers and possibly jumps to specified label
-# branch_gt : Compares registers and possibly jumps to specified label
-# branch_le : Compares registers and possibly jumps to specified label
-# branch_lt : Compares registers and possibly jumps to specified label
-# branch_eq : Compares registers and possibly jumps to specified label
-# branch_ne : Compares registers and possibly jumps to specified label
-# branch_ne_null : Jumps if register does not contain NULL value
-# branch_eq_null : Jumps if register contains NULL value
-# branch_label : Unconditional jump to label
-# interpret_exit_ok : Exit interpreted program
(approving tuple if used in scan)
-# interpret_exit_nok : Exit interpreted program
(disqualifying tuple if used in scan)
There are also three instructions for subroutines, which The second example uses an <code>NdbIndexScanOperation</code>:
are described in the next section. @code
// 1. Create
MyOperation= MyTransaction->getNdbIndexScanOperation("MYORDEREDINDEX", "MYTABLENAME");
@subsection subsubSub Interpreted Programs: Subroutines // 2. Define type of operation and lock mode
MyOperation->readTuples(NdbOperation::LM_Read);
The following are legal interpreted program instructions for // 3. Specify Search Conditions
subroutines: // All rows with ATTR1 between i and (i+1)
-# NdbOperation::def_subroutine : MyOperation->setBound("ATTR1", NdbIndexScanOperation::BoundGE, i);
Defines start of subroutine in interpreted program code MyOperation->setBound("ATTR1", NdbIndexScanOperation::BoundLE, i+1);
-# NdbOperation::call_sub :
Calls a subroutine
-# NdbOperation::ret_sub :
Return from subroutine
The virtual machine executes subroutines using a stack for // 4. Attribute Actions
its operation. MyRecAttr = MyOperation->getValue("ATTR2", NULL);
The stack allows for up to 24 subroutine calls in succession. @endcode
Deeper subroutine nesting will cause an abort of the transaction.
All subroutines starts with the instruction <h4>Step 1: Define scan operation operation type</h4>
NdbOperation::def_subroutine and ends with the instruction Scan operations only support 1 operation, @ref NdbScanOperation::readTuples or @ref NdbIndexScanOperation::readTuples
NdbOperation::ret_sub.
If it is necessary to return earlier in the subroutine
it has to be done using a branch_label instruction
to a label defined right before the
NdbOperation::ret_sub instruction.
@note The subroutines are automatically numbered starting with 0. @note If you want to define multiple scan operations within the same transaction,
The parameter used by NdbOperation::def_subroutine then you need to call NdbTransaction::getNdb*ScanOperation for each
should match the automatic numbering to make it easier to operation.
debug the interpreted program.
*/
#endif
#ifndef DOXYGEN_SHOULD_SKIP_INTERNAL <h4>Step 2: Specify Search Conditions</h4>
/** The search condition is used to select tuples.
@section secAsync Asynchronous Transactions If no search condition is specified, the scan will return all rows
The asynchronous interface is used to increase the speed of in the table.
transaction executing by better utilizing the connection
between the application and the NDB Kernel.
The interface is used to send many transactions
at the same time to the NDB kernel.
This is often much more efficient than using synchronous transactions.
The main reason for using this method is to ensure that
Sending many transactions at the same time ensures that bigger
chunks of data are sent when actually sending and thus decreasing
the operating system overhead.
The synchronous call to NdbTransaction::execute Search condition can be @ref NdbScanFilter which can be used on both
normally performs three main steps:<br> @ref NdbScanOperation and @ref NdbIndexScanOperation or bounds which
-# <b>Prepare</b> can only be used on index scans, @ref NdbIndexScanOperation::setBound.
Check transaction status An index scan can have both NdbScanFilter and bounds
- if problems, abort the transaction
- if ok, proceed
-# <b>Send</b>
Send the defined operations since last execute
or since start of transaction.
-# <b>Poll</b>
Wait for response from NDB kernel.
The asynchronous method NdbTransaction::executeAsynchPrepare @note When NdbScanFilter is used each row is examined but maybe not
only perform step 1. returned. But when using bounds, only rows within bounds will be examined.
(The abort part in step 1 is only prepared for. The actual
aborting of the transaction is performed in a later step.)
Asynchronous transactions are defined and executed <h4>Step 3: Specify Attribute Actions</h4>
in the following way.
-# Start (create) transactions (same way as for the
synchronous transactions)
-# Add and define operations (also as in the synchronous case)
-# <b>Prepare</b> transactions
(using NdbTransaction::executeAsynchPrepare or
NdbTransaction::executeAsynch)
-# <b>Send</b> transactions to NDB Kernel
(using Ndb::sendPreparedTransactions,
NdbTransaction::executeAsynch, or Ndb::sendPollNdb)
-# <b>Poll</b> NDB kernel to find completed transactions
(using Ndb::pollNdb or Ndb::sendPollNdb)
-# Close transactions (same way as for the synchronous transactions)
See example program in section @ref ndbapi_example2.cpp. Now it is time to define which attributes should be read.
Normally the attribute is defined by its name but it is
also possible to use the attribute identity to define the
attribute.
NdbOperation::getValue returns an NdbRecAttr object
containing the read value.
To get the value, there is actually two methods.
The application can either
- use its own memory (passed through a pointer aValue) to
NdbOperation::getValue, or
- receive the attribute value in an NdbRecAttr object allocated
by the NDB API.
The NdbRecAttr object is released when Ndb::closeTransaction
is called.
Thus, the application can not reference this object after
Ndb::closeTransaction have been called.
The result of reading data from an NdbRecAttr object before
calling NdbTransaction::execute is undefined.
<h3> Using Scan to update/delete </h3>
Scanning can also be used to update/delete rows.
This is performed by
-# Scan using exclusive locks, NdbOperation::LM_Exclusive
-# When iterating through the result set, for each row optionally call
either NdbScanOperation::updateCurrentTuple or
NdbScanOperation::deleteCurrentTuple
-# If performing <code>NdbScanOperation::updateCurrentTuple</code>,
set new values on record using ordinary @ref NdbOperation::setValue.
NdbOperation::equal should _not_ be called as the primary key is
retreived from the scan.
@note that the actual update/delete will not be performed until next
NdbTransaction::execute (as with single row operations),
NdbTransaction::execute needs to be called before locks are released,
see @ref secScanLocks
<h4> Index scans specific features </h4>
The following features are available when performing an index scan
- Scan subset of table using @ref NdbIndexScanOperation::setBound
- Ordering result set ascending or descending, @ref NdbIndexScanOperation::readTuples
- When using NdbIndexScanOperation::BoundEQ on distribution key
only fragment containing rows will be scanned.
This prepare-send-poll protocol actually exists in four variants: Rows are returned unordered unless sorted is set to true.
- (Prepare-Send-Poll). This is the one-step variant provided @note When performing sorted scan, parameter parallelism to readTuples will
by synchronous transactions. be ignored and max parallelism will be used instead.
- (Prepare-Send)-Poll. This is the two-step variant using
NdbTransaction::executeAsynch and Ndb::pollNdb.
- Prepare-(Send-Poll). This is the two-step variant using
NdbTransaction::executeAsynchPrepare and Ndb::sendPollNdb.
- Prepare-Send-Poll. This is the three-step variant using
NdbTransaction::executeAsynchPrepare, Ndb::sendPreparedTransactions, and
Ndb::pollNdb.
Transactions first has to be prepared by using method
NdbTransaction::executeAsynchPrepare or NdbTransaction::executeAsynch.
The difference between these is that
NdbTransaction::executeAsynch also sends the transaction to
the NDB kernel.
One of the arguments to these methods is a callback method.
The callback method is executed during polling (item 5 above).
Note that NdbTransaction::executeAsynchPrepare does not
send the transaction to the NDB kernel. When using
NdbTransaction::executeAsynchPrepare, you either have to call
Ndb::sendPreparedTransactions or Ndb::sendPollNdb to send the
database operations.
(Ndb::sendPollNdb also polls Ndb for completed transactions.)
The methods Ndb::pollNdb and Ndb::sendPollNdb checks if any
sent transactions are completed. The method Ndb::sendPollNdb
also send all prepared transactions before polling NDB.
Transactions still in the definition phase (i.e. items 1-3 above,
transactions which has not yet been sent to the NDB kernel) are not
affected by poll-calls.
The poll method invoked (either Ndb::pollNdb or Ndb::sendPollNdb)
will return when:
-# at least 'minNoOfEventsToWakeup' of the transactions
are finished processing, and
-# all of these transactions have executed their
callback methods.
The poll method returns the number of transactions that
have finished processing and executed their callback methods.
@note When an asynchronous transaction has been started and sent to @subsection secScanLocks Lock handling with scans
the NDB kernel, it is not allowed to execute any methods on
objects belonging to this transaction until the transaction
callback method have been executed.
(The transaction is stated and sent by either
NdbTransaction::executeAsynch or through the combination of
NdbTransaction::executeAsynchPrepare and either
Ndb::sendPreparedTransactions or Ndb::sendPollNdb).
More about how transactions are sent the NDB Kernel is When scanning a table or an index potentially
available in section @ref secAdapt. a lot of records will be returned.
*/
#endif But Ndb will only lock a batch of rows per fragment at a time.
How many rows will be locked per fragment is controlled by the
<code>batch</code> parameter to @ref NdbScanOperation::readTuples.
To let the application handle how locks are released
@ref NdbScanOperation::nextResult have a parameter <code>fetch_allow</code>.
If NdbScanOperation::nextResult is called with fetch_allow = false, no
locks may be released as result of the function call. Otherwise the locks
for the current batch may be released.
This example shows scan delete, handling locks in an efficient manner.
For the sake of brevity, we omit error-handling.
@code
int check;
// Outer loop for each batch of rows
while((check = MyScanOperation->nextResult(true)) == 0)
{
do
{
// Inner loop for each row within batch
MyScanOperation->deleteCurrentTuple();
} while((check = MyScanOperation->nextResult(false) == 0));
// When no more rows in batch, exeute all defined deletes
MyTransaction->execute(NoCommit);
}
@endcode
See @ref ndbapi_scan.cpp for full example of scan.
/**
@section secError Error Handling @section secError Error Handling
Errors can occur when Errors can occur when
...@@ -659,18 +440,6 @@ ...@@ -659,18 +440,6 @@
* @include ndbapi_example4.cpp * @include ndbapi_example4.cpp
*/ */
#ifndef DOXYGEN_SHOULD_SKIP_INTERNAL
/**
* @page select_all.cpp select_all.cpp
* @include select_all.cpp
*/
/**
* @page ndbapi_async.cpp ndbapi_async.cpp
* @include ndbapi_async.cpp
*/
#endif
/** /**
* @page ndbapi_scan.cpp ndbapi_scan.cpp * @page ndbapi_scan.cpp ndbapi_scan.cpp
* @include ndbapi_scan.cpp * @include ndbapi_scan.cpp
...@@ -803,24 +572,10 @@ ...@@ -803,24 +572,10 @@
and thus each record of the table has the same schema. and thus each record of the table has the same schema.
@subsection secKeys Tuple Keys @subsection secKeys Primary Keys
Each record has from zero up to four attributes which belong Each record has from 1 up to 32 attributes which belong
to the primary key of the table. to the primary key of the table.
If no attribute belongs to the primary key, then
the NDB Cluster creates an attribute named <em>NDB$TID</em>
which stores a tuple identity.
The <em>tuple key</em> of a table is thus either
the primary key attributes or the special NDB$TID attribute.
@subsection secArrays Array Attributes
A table attribute in NDB Cluster can be of <em>array type</em>.
This means that the attribute consists of an array of
<em>elements</em>. The <em>attribute size</em> is the size
of one element of the array (expressed in bits) and the
<em>array size</em> is the number of elements of the array.
@section secTrans Transactions @section secTrans Transactions
Transactions are committed to main memory, Transactions are committed to main memory,
...@@ -881,6 +636,280 @@ ...@@ -881,6 +636,280 @@
of the primary key and keyLen will be 4. of the primary key and keyLen will be 4.
*/ */
#ifndef DOXYGEN_SHOULD_SKIP_INTERNAL
/**
(A transaction's execution can also be divided into three
steps: prepare, send, and poll. This allows us to perform asynchronous
transactions. More about this later.)
*/
#endif
#ifndef DOXYGEN_SHOULD_SKIP_INTERNAL
/**
Another way to execute several parallel transactions is to use
asynchronous transactions.
*/
#endif
#ifndef DOXYGEN_SHOULD_SKIP_INTERNAL
/**
Operations are of two different kinds:
-# standard operations, and
-# interpreted program operations.
*/
#endif
#ifndef DOXYGEN_SHOULD_SKIP_INTERNAL
/**
<h3>Interpreted Program Operations</h3>
The following types of interpreted program operations exist:
-# NdbOperation::interpretedUpdateTuple :
updates a tuple using an interpreted program
-# NdbOperation::interpretedDeleteTuple :
delete a tuple using an interpreted program
The operations interpretedUpdateTuple and interpretedDeleteTuple both
work using the unique tuple key.
These <em>interpreted programs</em>
make it possible to perform computations
inside the NDB Cluster Kernel instead of in the application
program.
This is sometimes very effective, since no intermediate results
are sent to the application, only the final result.
<h3>Interpreted Update and Delete</h3>
Operations for interpreted updates and deletes must follow a
certain order when defining operations on a tuple.
As for read and write operations,
one must first define the operation type and then the search key.
-# The first step is to define the initial readings.
In this phase it is only allowed to use the
NdbOperation::getValue method.
This part might be empty.
-# The second step is to define the interpreted part.
The methods supported are the methods listed below except
NdbOperation::def_subroutine and NdbOperation::ret_sub
which can only be used in a subroutine.
NdbOperation::incValue and NdbOperation::subValue
increment and decrement attributes
(currently only unsigned integers supported).
This part can also be empty since interpreted updates
can be used for reading and updating the same tuple.
<p>
Even though getValue and setValue are not really interpreted
program instructions, it is still allowed to use them as
the last instruction of the program.
(If a getValue or setValue is found when an interpret_exit_ok
could have been issued then the interpreted_exit_ok
will be inserted.
A interpret_exit_ok should be viewed as a jump to the first
instruction after the interpreted instructions.)
-# The third step is to define all updates without any
interpreted program instructions.
Here a set of NdbOperation::setValue methods are called.
There might be zero such calls.
-# The fourth step is the final readings.
The initial readings reads the initial value of attributes
and the final readings reads them after their updates.
There might be zero NdbOperation::getValue calls.
-# The fifth step is possible subroutine definitions using
NdbOperation::def_subroutine and NdbOperation::ret_sub.
*/
#endif
#ifndef DOXYGEN_SHOULD_SKIP_INTERNAL
/**
<h3>Interpreted Programs</h3>
Interpretation programs are executed in a
register-based virtual machine.
The virtual machine has eight 64 bit registers numbered 0-7.
Each register contains type information which is used both
for type conversion and for type checking.
@note Arrays are currently <b>not</b> supported in the virtual machine.
Currently only unsigned integers are supported and of size
maximum 64 bits.
All errors in the interpretation program will cause a
transaction abort, but will not affect any other transactions.
The following are legal interpreted program instructions:
-# incValue : Add to an attribute
-# subValue : Subtract from an attribute
-# def_label : Define a label in the interpreted program
-# add_reg : Add two registers
-# sub_reg : Subtract one register from another
-# load_const_u32 : Load an unsigned 32 bit value into a register
-# load_const_u64 : Load an unsigned 64 bit value into a register
-# load_const_null : Load a NULL value into a register
-# read_attr : Read attribute value into a register
-# write_attr : Write a register value into an attribute
-# branch_ge : Compares registers and possibly jumps to specified label
-# branch_gt : Compares registers and possibly jumps to specified label
-# branch_le : Compares registers and possibly jumps to specified label
-# branch_lt : Compares registers and possibly jumps to specified label
-# branch_eq : Compares registers and possibly jumps to specified label
-# branch_ne : Compares registers and possibly jumps to specified label
-# branch_ne_null : Jumps if register does not contain NULL value
-# branch_eq_null : Jumps if register contains NULL value
-# branch_label : Unconditional jump to label
-# interpret_exit_ok : Exit interpreted program
(approving tuple if used in scan)
-# interpret_exit_nok : Exit interpreted program
(disqualifying tuple if used in scan)
There are also three instructions for subroutines, which
are described in the next section.
@subsection subsubSub Interpreted Programs: Subroutines
The following are legal interpreted program instructions for
subroutines:
-# NdbOperation::def_subroutine :
Defines start of subroutine in interpreted program code
-# NdbOperation::call_sub :
Calls a subroutine
-# NdbOperation::ret_sub :
Return from subroutine
The virtual machine executes subroutines using a stack for
its operation.
The stack allows for up to 24 subroutine calls in succession.
Deeper subroutine nesting will cause an abort of the transaction.
All subroutines starts with the instruction
NdbOperation::def_subroutine and ends with the instruction
NdbOperation::ret_sub.
If it is necessary to return earlier in the subroutine
it has to be done using a branch_label instruction
to a label defined right before the
NdbOperation::ret_sub instruction.
@note The subroutines are automatically numbered starting with 0.
The parameter used by NdbOperation::def_subroutine
should match the automatic numbering to make it easier to
debug the interpreted program.
*/
#endif
#ifndef DOXYGEN_SHOULD_SKIP_INTERNAL
/**
@section secAsync Asynchronous Transactions
The asynchronous interface is used to increase the speed of
transaction executing by better utilizing the connection
between the application and the NDB Kernel.
The interface is used to send many transactions
at the same time to the NDB kernel.
This is often much more efficient than using synchronous transactions.
The main reason for using this method is to ensure that
Sending many transactions at the same time ensures that bigger
chunks of data are sent when actually sending and thus decreasing
the operating system overhead.
The synchronous call to NdbTransaction::execute
normally performs three main steps:<br>
-# <b>Prepare</b>
Check transaction status
- if problems, abort the transaction
- if ok, proceed
-# <b>Send</b>
Send the defined operations since last execute
or since start of transaction.
-# <b>Poll</b>
Wait for response from NDB kernel.
The asynchronous method NdbTransaction::executeAsynchPrepare
only perform step 1.
(The abort part in step 1 is only prepared for. The actual
aborting of the transaction is performed in a later step.)
Asynchronous transactions are defined and executed
in the following way.
-# Start (create) transactions (same way as for the
synchronous transactions)
-# Add and define operations (also as in the synchronous case)
-# <b>Prepare</b> transactions
(using NdbTransaction::executeAsynchPrepare or
NdbTransaction::executeAsynch)
-# <b>Send</b> transactions to NDB Kernel
(using Ndb::sendPreparedTransactions,
NdbTransaction::executeAsynch, or Ndb::sendPollNdb)
-# <b>Poll</b> NDB kernel to find completed transactions
(using Ndb::pollNdb or Ndb::sendPollNdb)
-# Close transactions (same way as for the synchronous transactions)
See example program in section @ref ndbapi_example2.cpp.
This prepare-send-poll protocol actually exists in four variants:
- (Prepare-Send-Poll). This is the one-step variant provided
by synchronous transactions.
- (Prepare-Send)-Poll. This is the two-step variant using
NdbTransaction::executeAsynch and Ndb::pollNdb.
- Prepare-(Send-Poll). This is the two-step variant using
NdbTransaction::executeAsynchPrepare and Ndb::sendPollNdb.
- Prepare-Send-Poll. This is the three-step variant using
NdbTransaction::executeAsynchPrepare, Ndb::sendPreparedTransactions, and
Ndb::pollNdb.
Transactions first has to be prepared by using method
NdbTransaction::executeAsynchPrepare or NdbTransaction::executeAsynch.
The difference between these is that
NdbTransaction::executeAsynch also sends the transaction to
the NDB kernel.
One of the arguments to these methods is a callback method.
The callback method is executed during polling (item 5 above).
Note that NdbTransaction::executeAsynchPrepare does not
send the transaction to the NDB kernel. When using
NdbTransaction::executeAsynchPrepare, you either have to call
Ndb::sendPreparedTransactions or Ndb::sendPollNdb to send the
database operations.
(Ndb::sendPollNdb also polls Ndb for completed transactions.)
The methods Ndb::pollNdb and Ndb::sendPollNdb checks if any
sent transactions are completed. The method Ndb::sendPollNdb
also send all prepared transactions before polling NDB.
Transactions still in the definition phase (i.e. items 1-3 above,
transactions which has not yet been sent to the NDB kernel) are not
affected by poll-calls.
The poll method invoked (either Ndb::pollNdb or Ndb::sendPollNdb)
will return when:
-# at least 'minNoOfEventsToWakeup' of the transactions
are finished processing, and
-# all of these transactions have executed their
callback methods.
The poll method returns the number of transactions that
have finished processing and executed their callback methods.
@note When an asynchronous transaction has been started and sent to
the NDB kernel, it is not allowed to execute any methods on
objects belonging to this transaction until the transaction
callback method have been executed.
(The transaction is stated and sent by either
NdbTransaction::executeAsynch or through the combination of
NdbTransaction::executeAsynchPrepare and either
Ndb::sendPreparedTransactions or Ndb::sendPollNdb).
More about how transactions are sent the NDB Kernel is
available in section @ref secAdapt.
*/
#endif
/**
Put this back when real array ops are supported
i.e. get/setValue("kalle[3]");
@subsection secArrays Array Attributes
A table attribute in NDB Cluster can be of <em>array type</em>.
This means that the attribute consists of an array of
<em>elements</em>. The <em>attribute size</em> is the size
of one element of the array (expressed in bits) and the
<em>array size</em> is the number of elements of the array.
*/
#ifndef Ndb_H #ifndef Ndb_H
#define Ndb_H #define Ndb_H
......
ndb/include/ndbapi/NdbIndexScanOperation.hpp
View file @ e4d72dfa
...@@ -34,16 +34,15 @@ class NdbIndexScanOperation : public NdbScanOperation { ...@@ -34,16 +34,15 @@ class NdbIndexScanOperation : public NdbScanOperation {
public: public:
/** /**
* readTuples returns a NdbResultSet where tuples are stored. * readTuples using ordered index
* Tuples are not stored in NdbResultSet until execute(NoCommit)
* has been executed and nextResult has been called.
* *
* @param parallel Scan parallelism * @param lock_mode Lock mode
* @param batch No of rows to fetch from each fragment at a time * @param batch No of rows to fetch from each fragment at a time
* @param LockMode Scan lock handling * @param parallel No of fragments to scan in parallel
* @param order_by Order result set in index order * @param order_by Order result set in index order
* @param order_desc Order descending, ignored unless order_by * @param order_desc Order descending, ignored unless order_by
* @returns NdbResultSet. * @param read_range_no Enable reading of range no using @ref get_range_no
* @returns 0 for success and -1 for failure
* @see NdbScanOperation::readTuples * @see NdbScanOperation::readTuples
*/ */
int readTuples(LockMode = LM_Read, int readTuples(LockMode = LM_Read,
...@@ -53,16 +52,6 @@ public: ...@@ -53,16 +52,6 @@ public:
bool order_desc = false, bool order_desc = false,
bool read_range_no = false); bool read_range_no = false);
#ifndef DOXYGEN_SHOULD_SKIP_DEPRECATED
inline int readTuples(int parallell){
return readTuples(LM_Read, 0, parallell, false);
}
inline int readTuplesExclusive(int parallell = 0){
return readTuples(LM_Exclusive, 0, parallell, false);
}
#endif
/** /**
* Type of ordered index key bound. The values (0-4) will not change * Type of ordered index key bound. The values (0-4) will not change
* and can be used explicitly (e.g. they could be computed). * and can be used explicitly (e.g. they could be computed).
...@@ -134,7 +123,14 @@ public: ...@@ -134,7 +123,14 @@ public:
*/ */
int get_range_no(); int get_range_no();
/**
* Is current scan sorted
*/
bool getSorted() const { return m_ordered; } bool getSorted() const { return m_ordered; }
/**
* Is current scan sorted descending
*/
bool getDescending() const { return m_descending; } bool getDescending() const { return m_descending; }
private: private:
NdbIndexScanOperation(Ndb* aNdb); NdbIndexScanOperation(Ndb* aNdb);
......
ndb/include/ndbapi/NdbOperation.hpp
View file @ e4d72dfa
...@@ -265,21 +265,6 @@ public: ...@@ -265,21 +265,6 @@ public:
int equal(Uint32 anAttrId, Int64 aValue); int equal(Uint32 anAttrId, Int64 aValue);
int equal(Uint32 anAttrId, Uint64 aValue); int equal(Uint32 anAttrId, Uint64 aValue);
/**
* Generate a tuple id and set it as search argument.
*
* The Tuple id has NDB$TID as attribute name and 0 as attribute id.
*
* The generated tuple id is returned by the method.
* If zero is returned there is an error.
*
* This is mostly used for tables without any primary key
* attributes.
*
* @return Generated tuple id if successful, otherwise 0.
*/
Uint64 setTupleId();
/** @} *********************************************************************/ /** @} *********************************************************************/
/** /**
* @name Specify Attribute Actions for Operations * @name Specify Attribute Actions for Operations
...@@ -708,6 +693,7 @@ public: ...@@ -708,6 +693,7 @@ public:
/** @} *********************************************************************/ /** @} *********************************************************************/
#ifndef DOXYGEN_SHOULD_SKIP_INTERNAL
/** /**
* Type of operation * Type of operation
*/ */
...@@ -723,11 +709,16 @@ public: ...@@ -723,11 +709,16 @@ public:
NotDefined2, ///< Internal for debugging NotDefined2, ///< Internal for debugging
NotDefined ///< Internal for debugging NotDefined ///< Internal for debugging
}; };
#endif
/**
* Return lock mode for operation
*/
LockMode getLockMode() const { return theLockMode; } LockMode getLockMode() const { return theLockMode; }
void setAbortOption(Int8 ao) { m_abortOption = ao; }
#ifndef DOXYGEN_SHOULD_SKIP_INTERNAL #ifndef DOXYGEN_SHOULD_SKIP_INTERNAL
void setAbortOption(Int8 ao) { m_abortOption = ao; }
/** /**
* Set/get distribution/partition key * Set/get distribution/partition key
*/ */
...@@ -758,8 +749,10 @@ protected: ...@@ -758,8 +749,10 @@ protected:
void next(NdbOperation*); // Set next pointer void next(NdbOperation*); // Set next pointer
NdbOperation* next(); // Get next pointer NdbOperation* next(); // Get next pointer
public: public:
#ifndef DOXYGEN_SHOULD_SKIP_INTERNAL
const NdbOperation* next() const; const NdbOperation* next() const;
const NdbRecAttr* getFirstRecAttr() const; const NdbRecAttr* getFirstRecAttr() const;
#endif
protected: protected:
enum OperationStatus enum OperationStatus
......
ndb/include/ndbapi/NdbRecAttr.hpp
View file @ e4d72dfa
...@@ -245,7 +245,9 @@ public: ...@@ -245,7 +245,9 @@ public:
~NdbRecAttr(); ~NdbRecAttr();
public: public:
#ifndef DOXYGEN_SHOULD_SKIP_INTERNAL
const NdbRecAttr* next() const; const NdbRecAttr* next() const;
#endif
private: private:
NdbRecAttr(); NdbRecAttr();
......
ndb/include/ndbapi/NdbScanOperation.hpp
View file @ e4d72dfa
...@@ -37,16 +37,14 @@ class NdbScanOperation : public NdbOperation { ...@@ -37,16 +37,14 @@ class NdbScanOperation : public NdbOperation {
public: public:
/** /**
* readTuples returns a NdbResultSet where tuples are stored. * readTuples
* Tuples are not stored in NdbResultSet until execute(NoCommit)
* has been executed and nextResult has been called.
* *
* @param parallel Scan parallelism * @param lock_mode Lock mode
* @param batch No of rows to fetch from each fragment at a time * @param batch No of rows to fetch from each fragment at a time
* @param LockMode Scan lock handling * @param parallel No of fragments to scan in parallell
* @note specifying 0 for batch and parallall means max performance * @note specifying 0 for batch and parallall means max performance
*/ */
int readTuples(LockMode = LM_Read, int readTuples(LockMode lock_mode = LM_Read,
Uint32 batch = 0, Uint32 parallel = 0); Uint32 batch = 0, Uint32 parallel = 0);
#ifndef DOXYGEN_SHOULD_SKIP_DEPRECATED #ifndef DOXYGEN_SHOULD_SKIP_DEPRECATED
...@@ -67,7 +65,7 @@ public: ...@@ -67,7 +65,7 @@ public:
/** /**
* Get the next tuple in a scan transaction. * Get the next tuple in a scan transaction.
* *
* After each call to NdbResult::nextResult * After each call to nextResult
* the buffers and NdbRecAttr objects defined in * the buffers and NdbRecAttr objects defined in
* NdbOperation::getValue are updated with values * NdbOperation::getValue are updated with values
* from the scanned tuple. * from the scanned tuple.
...@@ -114,53 +112,31 @@ public: ...@@ -114,53 +112,31 @@ public:
int nextResult(bool fetchAllowed = true, bool forceSend = false); int nextResult(bool fetchAllowed = true, bool forceSend = false);
/** /**
* Close result set (scan) * Close scan
*/ */
void close(bool forceSend = false); void close(bool forceSend = false);
/** /**
* Restart * Update current tuple
*/
int restart(bool forceSend = false);
/**
* Transfer scan operation to an updating transaction. Use this function
* when a scan has found a record that you want to update.
* 1. Start a new transaction.
* 2. Call the function takeOverForUpdate using your new transaction
* as parameter, all the properties of the found record will be copied
* to the new transaction.
* 3. When you execute the new transaction, the lock held by the scan will
* be transferred to the new transaction(it's taken over).
*
* @note You must have started the scan with openScanExclusive
* to be able to update the found tuple.
* *
* @param updateTrans the update transaction connection.
* @return an NdbOperation or NULL. * @return an NdbOperation or NULL.
*/ */
NdbOperation* updateCurrentTuple(); NdbOperation* updateCurrentTuple();
NdbOperation* updateCurrentTuple(NdbTransaction* updateTrans); NdbOperation* updateCurrentTuple(NdbTransaction* updateTrans);
/** /**
* Transfer scan operation to a deleting transaction. Use this function * Delete current tuple
* when a scan has found a record that you want to delete. * @return 0 on success or -1 on failure
* 1. Start a new transaction.
* 2. Call the function takeOverForDelete using your new transaction
* as parameter, all the properties of the found record will be copied
* to the new transaction.
* 3. When you execute the new transaction, the lock held by the scan will
* be transferred to the new transaction(its taken over).
*
* @note You must have started the scan with openScanExclusive
* to be able to delete the found tuple.
*
* @param deleteTrans the delete transaction connection.
* @return an NdbOperation or NULL.
*/ */
int deleteCurrentTuple(); int deleteCurrentTuple();
int deleteCurrentTuple(NdbTransaction* takeOverTransaction); int deleteCurrentTuple(NdbTransaction* takeOverTransaction);
/**
* Restart scan with exactly the same
* getValues and search conditions
*/
int restart(bool forceSend = false);
protected: protected:
NdbScanOperation(Ndb* aNdb); NdbScanOperation(Ndb* aNdb);
virtual ~NdbScanOperation(); virtual ~NdbScanOperation();
......
ndb/include/ndbapi/NdbTransaction.hpp
View file @ e4d72dfa
...@@ -110,23 +110,6 @@ enum ExecType { ...@@ -110,23 +110,6 @@ enum ExecType {
* -# AbortOption::IgnoreError * -# AbortOption::IgnoreError
* Continue execution of transaction even if operation fails * Continue execution of transaction even if operation fails
* *
* NdbTransaction::execute can sometimes indicate an error
* (return with -1) while the error code on the NdbTransaction is 0.
* This is an indication that one of the operations found a record
* problem. The transaction is still ok and can continue as usual.
* The NdbTransaction::execute returns -1 together with error code
* on NdbTransaction object equal to 0 always means that an
* operation was not successful but that the total transaction was OK.
* By checking error codes on the individual operations it is possible
* to find out which operation was not successful.
*
* NdbTransaction::executeScan is used to setup a scan in the NDB kernel
* after it has been defined.
* NdbTransaction::nextScanResult is used to iterate through the
* scanned tuples.
* After each call to NdbTransaction::nextScanResult, the pointers
* of NdbRecAttr objects defined in the NdbOperation::getValue
* operations are updated with the values of the new the scanned tuple.
*/ */
/* FUTURE IMPLEMENTATION: /* FUTURE IMPLEMENTATION:
...@@ -376,6 +359,7 @@ public: ...@@ -376,6 +359,7 @@ public:
#endif #endif
void close(); void close();
#ifndef DOXYGEN_SHOULD_SKIP_INTERNAL
/** /**
* Restart transaction * Restart transaction
* *
...@@ -385,6 +369,7 @@ public: ...@@ -385,6 +369,7 @@ public:
* Note this method also releases completed operations * Note this method also releases completed operations
*/ */
int restart(); int restart();
#endif
/** @} *********************************************************************/ /** @} *********************************************************************/
...@@ -494,7 +479,7 @@ public: ...@@ -494,7 +479,7 @@ public:
*/ */
const NdbOperation * getNextCompletedOperation(const NdbOperation * op)const; const NdbOperation * getNextCompletedOperation(const NdbOperation * op)const;
#ifndef DOXYGEN_SHOULD_SKIP_INTERNAL
const NdbOperation* getFirstDefinedOperation()const{return theFirstOpInList;} const NdbOperation* getFirstDefinedOperation()const{return theFirstOpInList;}
const NdbOperation* getLastDefinedOperation()const{return theLastOpInList;} const NdbOperation* getLastDefinedOperation()const{return theLastOpInList;}
...@@ -508,6 +493,7 @@ public: ...@@ -508,6 +493,7 @@ public:
* ops are used (read, insert, update, delete). * ops are used (read, insert, update, delete).
*/ */
int executePendingBlobOps(Uint8 flags = 0xFF); int executePendingBlobOps(Uint8 flags = 0xFF);
#endif
private: private:
/** /**
......
ndb/src/ndbapi/NdbOperationSearch.cpp
View file @ e4d72dfa
...@@ -300,32 +300,6 @@ NdbOperation::equal_impl(const NdbColumnImpl* tAttrInfo, ...@@ -300,32 +300,6 @@ NdbOperation::equal_impl(const NdbColumnImpl* tAttrInfo,
return -1; return -1;
} }
/******************************************************************************
* Uint64 setTupleId( void )
*
* Return Value: Return > 0: OK
* Return 0 : setTupleId failed
* Parameters:
* Remark:
*****************************************************************************/
Uint64
NdbOperation::setTupleId()
{
if (theStatus != OperationDefined)
{
return 0;
}
Uint64 tTupleId = theNdb->getTupleIdFromNdb(m_currentTable->m_tableId);
if (tTupleId == ~(Uint64)0){
setErrorCodeAbort(theNdb->theError.code);
return 0;
}
if (equal((Uint32)0, tTupleId) == -1)
return 0;
return tTupleId;
}
/****************************************************************************** /******************************************************************************
* int insertKEYINFO(const char* aValue, aStartPosition, * int insertKEYINFO(const char* aValue, aStartPosition,
* anAttrSizeInWords, Uint32 anAttrBitsInLastWord); * anAttrSizeInWords, Uint32 anAttrBitsInLastWord);
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7