Batch Processing
From: Mega Visual DataFlex Class of Stark Data/Boston by Pete DonovanDescription: A BPO (business process object) is a standalone piece of code that is used in your DataFlex program that exists for the purposes of updating files for a specific purpose. When the BPO is executed, a popup dialog shows the user the progress, and usually all errors are supressed until the processing is complete, when the user is informed of the processing statistics and success or failure. A BPO can use visible elements to gather input for processing but is an invisible object by default.
The BPO is a desireable element to your Visual DataFlex system when you want any of these major advantages:
The contacts sample example in Visual DataFlex contains a BPO written by John Tuohy that you can use as a guideline, and here is a blank BPO that I use as a template when creating a new BPO:
Use BatchDD.Pkg Use StatLog.Pkg // Interface: // Send DoProcess To _______ Object ______ Is A BusinessProcess Set Allow_Cancel_State To True // Properties // BPO Runtime Properties: // DDO's // Main Interface With BPO: Procedure Start_Process Procedure OnProcess // Main processing
logic here. Procedure End_Process //Errors will not be displayed during
execution. //Validation Errors (Of DDO's) Will Hit
The Log Showing File And Field Of Error End_Object // BPO |
Use DataDictionaries Or Not?
A BPO can utilize your Visual DataFlex (Database Builder) datadictionary classes and use DDOs (data dictionary objects) to do the processing, or a procedural coding style is also supported. The most important aspect of choosing a methodology is to determine which of these methods fits best with the objective of the BPO.I find that my overall objective in batch processing is usually a single transaction that I want to succeed totally or fail/rollback all changes, or a combination of these transactions. Frequently, I will want multiple transactions to happen together and desire them all to succeed or fail/rollback.
Using DDOs (data dictionary objects) is the most desireable way to code since you can utilize all of your current business rules for a file (if you can structure a DDO tree (connected DDOs) to handle your transaction objective). Here is such an example:
| Processing DELETES!: (Using the power of Data
Dictionaries)Situation:
The EuroSoft Vending Company has sales orders that automatically pull the required items
from the inventory file. When inventory is pulled, the Vendor File is updated with the
amount of sales for the year. Then, the salesrep assigned to the sales order is credited
with the sales volume for their end-of-year bonus. Lastly, the delivery person’s file
is updated with the dollar volume they delivered which counts towards a bonus prize for a
vacation in Miami Florida to visit Data Access Corp!
Here is the problem: Each week, a few orders are refused by the customers because the delivery man oversleeps and gets there late. Objective: Write a batch process to delete the undelivered sales orders and undo the updates to inventory, vendors, salesreps, and delivery people! //DDO TREE: Object SalesRep_DD Is A SalesRep_DataDictionary // Using your existing DD class rules, this code backs-out
all of the previous transactions! |
This BPO meets my objective because if any one transaction fails then it does not affect the integrity of the entire process. The code that makes it work is centralized in the DD class where it should be which means that any future adjustments to the DD classes will automatically flow to the BPO.
Here is an example of data architecture that does not appear to lend itself to using DDOs: (Process Paychecks)
In this example,
using OOPS DataFlex, no ONE transaction will save all of these files because saves travel
upwards only. In this case, it is preferable to process paychecks with DDOs by
altering the GL Master DDO to save the GL transaction file manually (instead of using a
DDO for the GL transaction). The 2 points here are that DDOs are preferable to procedural
("saverecord") code, and that instead of having 2 separate transactions (one to
paycheck and another to GL transaction) you need to change your environment so that one
save will complete the transaction. This example is halfway between using DDOs and using
procedural code.
Here is a code example of how this could be accomplished:
| //(From the Process Paychecks
BPO) Object GLMast_DD Is A GLMast_DataDictionary Procedure Update Clear GLTrans Move 105 To GLTrans.Account Move Paycheck.Amount To GLTrans.Amount Move Paycheck.Date To GLTrans.Date Saverecord GLTrans Forward Send Update End_Procedure End_Object |
Using procedural code to handle multiple transactions as one transaction is frequently desireable because many times you cannot get a DDO tree to handle multiple transactions as one ; Here is an example:
Objective: In a variable amount of (4 here)
separate transactions, make the bride and groom co-owners of these bank accounts. If we
used DDOs for this, we would have 4 separate transactions which could not be rolled back
if the fourth one failed… therefore procedural code is desireable.
When we use procedural code, there are two aspects of DDOs that we need to duplicate:
First, please consider the following code: The way that DDOs control which files get locked is to first set all files to (filemode) readonly and then reset the files participating in the transaction (filemode) back to their original condition. When the transaction is done, the files (filemode) is set back. Sometimes, as with alias files, their original condition is not "default" so you have to remember what the original setting was. The purpose of the following code is to allow you to use a reread command without locking all the files you have open, and to duplicate what the DDO tree does during a transaction:
| // This would be inserted in your
main package to be used in all programs: // Statement of purpose: // When using a reread command in VDF you will unfortunately lock many more // files than are necessary. In order to only lock the files necessary I // use the following procedure to set all files to readonly, manually set // the files being worked on to (default) and then reset all back at the end. // This is what a data dictionary does in a VDF program for it's file structure. Object FileModeArray Is An Array Procedure Write_Attribute_Array Integer iFile Integer
iFileMode Procedure Clear_FileModeArray Function Is_NonStandardFileMode Integer iFile Returns Integer //Here is the main procedure: Procedure ChangeAllFileModes Integer iMode Send Write_Attribute_Array iFile iExistingFileMode End If (iMode = DF_Filemode_Default) Begin // Setting All Files Back To Default Get Is_NonStandardFileMode iFile To iNonStandardFileMode End If (iNonStandardFileMode) Set_Attribute DF_File_Mode Of iFile To iNonStandardFileMode Else Set_Attribute DF_File_Mode Of iFile To iMode Loop End_Procedure |
Secondly, we can use a manual transaction block to process our data:
| Begin_Transaction Reread Repeat //Process Code Here If <Condition> Error 300 "Aborting Transaction!" Loop Unlock End_Transaction // Any Error (other than finderr) Will Exit The Transaction Block To Here. |
Here is a procedural code BPO that uses both of these techniques:
Here is another example when procedural coding is preferable: The parent file A changes
the ID field from "OLDVALUE" to "NEWVALUE" but child files B and C
have this as their key field and must be changed or else they will be orphaned as child
records.
There are 2 reasons
why DDOs would not be a good choice here:
Here is a procedural BPO that uses my BPO template and mimics the reread and rollback behaviors of a DDO tree:
| Use Batch.Pkg // File_Mode
Change Utility Package. Use BatchDD.Pkg Use StatLog.Pkg // Interface: Object Change_ID_BPO Is A BusinessProcess // Block, Execution Skips To Here. If (Err) Set piErrors To 1 Send ChangeAllFileModes DF_Filemode_Default //Set Back From Readonly To Normal End_Procedure Procedure End_Process String E# B# C# T# Forward Send End_Process // This closes the sentinel object. Get piErrors To E# Get piB_Records To B# Get piC_Records To C# Get piCounter To T# If (E# > 0) Send Stop_Box ("Process Failed: ERRORS WERE FOUND... " + ; "\nAll Changes Were Rolled Back: See StatusLog For Details") Else If (T#=0) Send Info_Box "No Errors: No Child Records Were Found" Else Send Info_Box ("Process Complete! (Ok)" + ; "\n" + B# + " B Records Were Updated" + ; "\n" + C# + " C Records Were Updated") End_Procedure //Errors will not be displayed during execution. Procedure OnError Integer iErr# String sErr_Description Set Error_Check_State To False End_Procedure End_Object // BPO |