mDesktop in Depth - Creating or Modifying a Job Script

 

Creating a Job Script

To the right of the main panel are the buttons associated with editing the job scripts.  To create a NEW job script, click on the New Job button.

You will now see a small window asking you for a new reference.  Type in something here which will be the name of the job script.  We suggest that you call it something descriptive.  The only limitation on this name is that it must be a name you have not used before (i.e. it must be unique).

You will now see the Job Editor Screen, which is described below.

Additionally it’s now possible to automatically generate a job script at the end of Wizard Processing using the Save Job button which appears on the Wizard Processing Complete window. You can also use the Job Overview report to check that the automatically generated job contains all the processing steps that you’re expecting.

Modifying a Job Script

At the Job Maintenance screen, you can choose to edit an existing job script (instead of creating a new one which is described above).  To do this, first choose the job you want to modify from the list of jobs available, then click on the Modify button on the right hand panel.

Whether you have chosen to modify a job, or create a new one, the next screen you will see is the Job Editor:

Explanation of Job Editor Layout

The main concept behind the job editor is that it is designed to represent a list of files to be processed.  Each line will correspond to the processing done on one file.  The processing is generally done in left-to-right, top-to-bottom order (like reading a book).  Processing may include importing a source file (into a predefined Main File layout), followed by internally deduping it, followed by finding its overlap with the master file (the one at the top of the list), and the purging of duplicate records from the file, or merging the records into the master file.

Therefore, the columns correspond to these processes – first you have the Source Name, which will be the file name of the source text file, followed by the Main DBF, which will be the Main File layout that the data is imported into.  This is followed by the Priority column, which indicated the files relevant priority in the list (low numbers = high priority).  Next, we have the column for import & matching configuration, Options, and then columns for internal deduplication (Match Keys) and then finding overlap between two files (Overlap Keys).  Finally, we have a column for the Flag Score to be used for processing.

In terms of file order, the first file is the Master file.  This is the file that each find overlap is done against.  Also, if files are merged, they will be merged into this file.

The files that follow this will be Merge/Purge files.  Each of these files can be imported and internally deduped according to how you set up that line of the job, and then will be compared with the master file.  (Note: they will NOT be compared to each other).  These files will then be purged of the overlapping records, or merged into the master file.

Following this comes Suppression files.  These files can be imported, deduped and compared to the master in the same way as the Merge/Purge files. However, these files are special in that they contain exclusion records.  That is, when the overlap is complete, the records will be removed from the master file, not the secondary file (which is the case if you select Purge Records from the Merge/Purge menu).  Files such as the MPS (Mailing Preference Service) file are suppression files.

You can also add bespoke program files to the job scripts at any point.  These files are not imported or processed in any way, but contain FoxPro program code.  They can be used to insert special programming steps and options into the job scripts which would not normally be possible.  A good understanding of the FoxPro programming language is normally required to create and use these kinds of files.

Therefore, a common job script could look like this:

The following sections will explain the columns and settings above in more detail.

Adding and Removing Files (Rows)

To add or remove a merge file or a suppression file, you must right-click the mouse in the file listing part of the window, to see the menu.  If you are removing a row that already exists, you must select that row by left-clicking on it first.  Note that when you start editing a new job script, mDesktop will automatically add a row for you.  

To add a merge/purge file, click on the Add Row option.  This will add a new blank row at the bottom of the list.  Similarly, you can add a Add Suppression Row with the equivalent option.  Note that when suppression files are added, they are always added to the bottom of the whole list, whilst merge/purge files will be added to the end of the merge/purge list only (i.e.  before suppression files).

If you want to remove the currently selected row, click on Delete Row.

With this menu, you can also Rename Job (click on it and it will ask you for the new name – the same naming conventions apply as when you create a new job script).  Furthermore, you can also Clone Job (copy the current job).  When you do this, mDesktop will again ask you for the new job script name.  Note that when the copying of the job has been done, the job you will be editing will be the new job.


Explanation of Columns

Source Name

This column specifies the file name of the original data – the data before you have done anything to it.  This is the file which will be imported into mDesktop.

If you double-click on an entry in this column, you will see the following screen:

You can either type directly into the field to change the file name, or you can click on the Choose source file button to choose another one.  Click on Edit source file to view (and perhaps change) the data file.

Special Cases

There are two special uses for this screen that the user should be aware of:

  • If you choose a file with extension .FXP (a Visual FoxPro compiled executable file) or .PRG (a text file which contains a FoxPro program – see later for more details), then this file will not be treated as an import file, but as an executable program file.  This means that mDesktop will not process the data in the file, but will attempt to run it instead.  In this case, all the other columns in the row are irrelevant, and will be ignored by mDesktop.  Note that compiled Visual FoxPro programs must be compiled using the same version of Visual FoxPro as your version of mDesktop – you can ascertain this via the Help menu, About mDesktop.
  • If you have an import format other than EXT or Regenerate Keys (see Setup below), and the source file name is blank, then mDesktop will not do any import processing on the file. mDesktop will assume the file has already been imported, and will not try to do so again.  In the case of the EXT format, the source file should always be empty.  (This format means do the generate keys step only – don't add records from a source file previous to this).  For more information on this format, please consult the Import Options section.


Main File

This column corresponds to the Main File that will contain the data after it has been imported.  To choose a file for this, just double-click on the field, and mDesktop will ask you for a file name.  Note that the file must exist before the job script is run – the job script cannot automatically create Main File layouts.  The File can of course be empty.  When the job is run, any data in this file will be over-written by the data in the source file.

Priority

The priority of a file corresponds to its place in the processing order.  The lower this number, the more important a file is.  The file with the highest priority (usually 1), will be at the top of the list.  This file will be the Master File, with which all other files are compared, and into which files can be merged.

Priorities are automatically assigned when you add merge or suppression files, in the order that they were added to the job.  You can change any of these priorities by typing into an appropriate number into this field.  This field supports decimal numbers if necessary.

For suppression files, these will have the priority set to "SUPPRESS".  You can make a merge file a suppression file by typing in the word SUPPRESS into this field.  Similarly, you can turn a suppression file into a merge file by typing a number into this field.

Suppression files do not have priorities amongst themselves, as they only act to remove records from the master file (when a record is removed from the master file, it doesn't matter if suppression file A or suppression file B removed it – the result is the same).

If you blank out the priority field, this will be treated as priority zero by mDesktop.  If you have two files with the same priority, the order they will be processed in will be the order that they were added (i.e. the one you added to the job first will be processed first).

Options

The Options column is where most of the mDesktop processing options are set.  These include options such as operation parameters and Import Options – please consult the mDesktop Options section for more information on the options available.

Double-clicking on a row in the Options column will give you a choice of three options:

Change

Select Change to view the standard mDesktop Options screen (with a couple of tabs that do not relate to job script processing removed).  From here, you can change processing options as required.

As Is

This option has two slightly different meanings, depending on whether it is chosen for the Master File or any other file.

If As Is is chosen for the Master File, this means that the mDesktop options chosen will be the same as the current configuration.  (Check this via the Jobs/Setup menu, Options before starting the job script).

If As Is is chosen for any file other than the first one on the list, the job script processing will take the mDesktop configuration from whatever was set up in the previous line in the script.  (That is, it remains unchanged).

Default

Whenever this option is chosen, the job script processing will take the configuration from the current setup of mDesktop.  (Note that this has the same effect as As Is for the first file in the list).

Notes

One pitfall for the unwary in job script configurations is the generation of unique reference numbers.  As has been stated elsewhere, it is vital that within a single file, all unique reference numbers remain unique.  This is simple within only one file, but when you are cross matching and merging two or more files together, then maintenance of the reference number becomes more complicated.  

One way to resolve this problem is to assign a new unique reference starting number for each file that is being imported.  You can do this by setting the Options column to Change for every file that is being imported, and by manually setting the starting reference number in the mDesktop options for each file.   Alternatively, if you set the job script up using the Multiple File Wizard, it will ensure that the unique references are really unique across all the records in all the files in the job – otherwise it will regenerate them starting from 1.

Another feature of the job script processing that can be used is the Input Format option.  As mentioned above, if you set this to anything except EXT, and have a blank Source File name, then no import processing will be done.  This can be very useful for quicker processing of files that have already been imported.  Furthermore, if you do not want to add new records into a file, but just want to regenerate the mDesktop fields, you can set the Input Format to EXT.

Match Keys

Match Keys are used for the internal deduplication step of mDesktop.  As with normal use of mDesktop, you should specify a set of Match Keys in order to carry out the deduplication.  This corresponds to the Find Matches menu option.

In a similar way to the Setup column, double-clicking on this field will show you an options screen:

Change

This option works in an equivalent way to the Change option in the Setup column.  Choose this option and click on Continue to go to the Match Keys screen where you can edit the Match Keys that will be used in the find matches step in the job.

As Is

Again, this works in a similar way to the As Is from the Setup column.  Selecting it means that the Match Keys used for this file will be the same as for the previous file in the list.

Note that this option does not apply to the first file in the list.

None

If you select this, then no internal Find Matches steps will take place.  Use this for files that have already been deduped by mDesktop, or files which do not need to be cleaned.

Setting the options in this column will only cause matches to be searched for.  If you want to actually delete the matches as well, you must set the Deletion Score to a non-zero value.

Overlap Keys

This sets the keys that will be used to find overlap between the master file, and the current file.  This works almost exactly the same way as the Match Keys option works.

The only real difference is that the first file can only have this set to None, as it can't find overlap with itself.

As for the Match Keys, this will only search for the overlap, it will not actually process the results – so if you want to merge the file with the master file or purge records from this file, set the deletion score column to the appropriate score value.

Deletion Score

This determines the score at which a potential match is considered to be a true match.  It is used in two ways.  Firstly, for internal deduplication, this is the threshold score for deleting records.  Secondly, when a Find Overlap is carried out, this is the threshold for purging records (usually in preparation for merging them into the master file).

With the master file, this score only refers to the deletion threshold for internal duplicates.

If you do not wish records to be deleted, set this value to zero (0) or "NONE".

Other Job Script Options

Merge Databases  

If you check this option, each file after the first one will be merged with the first file.  Records that have been found to overlap with the master file (via the Find Overlap steps) will not be added to the master file.

If you set this option off, then the overlapping records in the "merge files" will be purged according to the deletion score in each file, but the file will not be added to the master file.  Note that if the deletion score is set to zero or NONE, then records will not be purged either.

Note that the Multiple File Wizard is an exception for use of this option, as it always merges the files into the master file irrespective of whether this option is set on.

Run Another Job At End?

Sometimes, you may wish to link more than one job together.  This is especially the case when running small program files, or when you want to merge files into multiple master files.  In all cases, the first job will be totally complete before any part of the next job is started.

To enable this option, click on the check box.  You will be presented with a job script picker screen like this:

To run another job at the end of the current job, just choose the new job from the list available.  Once you have chosen the job you want, click on the Continue button to confirm the choice.

If you don't want to run a job at the end, click on the Clear button.  This will tell the job script processing NOT to run a job at the end of the current one.

Click Cancel if you are happy with the current state of this option, and don't wish to change it.

 

 

Was this article helpful?
0 out of 0 found this helpful

have a question or not finding what you're looking for?

Submit a ticket to get some help