|
PIN |
  
|
| Show/Hide Hidden Text |
dBASE Index Maintenance
PIN file opcode [pinno|loadfile]
The utility program PIN is provided to allow you to maintain any standalone .NDX files that the CopiaFacts system uses.
Please note that releases of PIN with version numbers 7.2xx and 7.300 to 7.304 had a serious bug (but fortunately a rare one) affecting key deletions. We recommend upgrading immediately to PIN version 7.305 or later if you still have an earlier release.
We strongly recommend rebuilding your NDX file indexes as soon as you migrate to the 7.3xx release of PIN. If they were created to permit duplicate keys (which was the default in release 7.2xx), then you must rebuild the index before it can be updated. We also recommend closing down COPIAFACTS nodes that are using the index while you perform and check the re-index operation, but this is not essential if you have a lot of small files to re-index. If you want to leave the index file type and key length unchanged, the simplest course of action is to re-index the file with the 7.3xx release of PIN: PIN filename i The above command will backup the file, re-index it, and convert alphanumeric keys to lower case. Duplicate keys will be removed and the file will become a 'unique-keys' index if it was not before. If you want to change the key type of a phone lookup file from character to numeric or vice-versa, first make a text-file copy of your index using a command: PIN filename L >temp.txt Then backup and delete your original NDX file, and create a new file using: COPY filename.ndx filename.ndx.backup Use the List command to check at least the start of each file after completing the migration. |
The original use of the PIN program ('PIN' in the sense of Personal Identification Number) was to manage the .NDX index files used by a number of CopiaFacts commands to validate a PIN or password. The PIN utility only operates on the .NDX file and not on any associated dBASE .DBF files, so should not be used on other than standalone NDX files.
PIN is now more often used as a simple means of maintaining DNS.NDX and DNS_MAIL.NDX 'do not send' files. For telephone numbers, a numeric key index is normally used but a character (alphanumeric) index is also supported. The lower-case opcode values should be used for e-mail addresses, of course with alphanumeric key files. Matching alphanumeric keys for do-not-send lookup is always case-sensitive, so the use of lower-case letters when loading and maintaining the index with PIN ensures that the case is always consistent.
Telephone and fax numbers can also be 'normalized' in North America Numbering Plan (NANP) areas. This process is described in the topic for Do Not Send files. Unlike earlier releases, the current 7.3xx PIN release will read the FAXFACTS.CFG file (if available) to find the definition of an environment variable NORMALIZE_DNS, as well as looking in the Windows environment for this variable (which will override the CFG file). Currently this is a global setting and applies to all NDX files. Normalizing is recommended because it avoids the need to duplicate long-distance numbers with and without a leading '1'. The 7.3xx releases of PIN no longer include special opcode values to select normalized numbers.
If normalization is not used, numeric key values in both numeric and alphanumeric key files are stripped of all non-numeric characters before use. An entered key is treated as numeric if it contains no letters and no '@' sign.
It is very important that you do not turn normalization on or off on an active system without careful planning. For example, if you create an index with normalization enabled and then turn normalization off, no ten digit NANP phone number (with no leading '1') will ever be found in the index because the leading '1' will not be added.
PIN is designed to interact with the COPIAFACTS engine to prevent simultaneous lookup and maintenance operations. However if you have a lot of work to do with bulk operations in PIN, it is recommended that this is done with a copy of the NDX file or while COPIAFACTS nodes are not using the file.
From release 7.307, PIN also supports a positive integer data value which can be associated with the index entry. This value, shown as m in the descriptions below, can be used for example to record the source of an entry in the file. If no value is provided, the data value is set to 1 in all index entries.
Command-Line Arguments:
| file | the full pathname (optionally without .NDX extension) of the .NDX file to be created or processed. Only .NDX extensions will be processed. You may use the standard system variables such as @FFBASE and @FFLOG at the start of an NDX file pathname (but not for the redirection pathname, if used). |
| opcode | the operation to be performed, as follows: |
| C | Create the file with a numeric index. This command should normally be used for phone numbers because the file is a bit smaller and the lookup is a bit quicker. |
| Cn | Create the file with an alphanumeric index. Choose a value of n which is long enough for international phone numbers, if used. For e-mail a value of 64 is usually long enough. The maximum value is 100. |
| Am,am | Add a single key. Duplicate keys will be rejected with a special exit code, so that you can report if a key was already present in the file. If the lower-case opcode is used, the value entered for the key will be converted to lower case. The key should not be surrounded by quotes or double-quotes. Keys longer than the alphanumeric key length are truncated. The value m, if present, is used to set a data value for the key. An addition is rejected if the key is present and the value is different from that specified by the m parameter or implied (1) by its absence. Otherwise this operation will update the data value of an existing entry. |
| Fm,fm | Add each key in the named file to the index. Double-quotes surrounding the key are removed. If the lower-case opcode is used, each key will be converted to lower case before it is added. Keys longer than the alphanumeric key length are truncated. Processing continues if a single key is duplicated or invalid. The quoted or unquoted key value in the list may be followed by a tab character and a data value to be applied to that key entry, or to update the data value if different. The value m, if present and positive, is used to force a data value for all the added keys; if the value m is negative its positive value will be applied only to those entries in the list which have no data value. |
| D,d | Delete a key. If the lower-case opcode is used, the value entered for the key will be converted to lower case (and will therefore not succeed in deleting any key containing upper-case characters). Deletions are now done by internally setting the data value of the entry to -1, so after a lot of individual deletions you may wish to re-index using the I command. |
| B,b | Bulk delete keys from a list. If the lower-case opcode is used, each key will be converted to lower case (and will therefore not succeed in deleting any key containing upper-case characters). Deletions are now done by internally setting the data value of the entry to -1, so after a lot of deletions you may wish to re-index using the I command. Alternatively, consider using the J command to delete using re-index. |
| Im,im | Re-Index. The keys are saved internally to a list, the index is cleared, and the keys are then re-loaded. A backup is automatically made before the operation starts, using file extensions .001, .002 etc. Because all NDX files are maintained with unique keys, files created with non-unique keys using earlier releases of PIN, or with other applications, will become unique and the number of duplicates removed will be reported at the end of the operation. Using a lower-case 'i' operation code will also convert alphanumeric keys to lower case. If normalization is enabled, keys will be normalized before they are added back. The index file is held locked during the re-index operation, so that access from the COPIAFACTS engine is prevented while the index may be empty or incomplete. The value m, if present and positive, is used to force a data value for all the keys; if the value m is negative its positive value will be applied only to those entries in the list which do not have an explicit (greater than 1) data value. This operation code can be followed by a second code letter to perform a conditional re-index as described below. |
| J, j | Bulk Delete using Re-Index. For large bulk deletions it is more efficient to delete using a re-index operation, which will also reduce the index size. The J command will save all active keys, remove those in the supplied list, and then re-load the remaining keys. |
| E,e | Process a list of keys from the named file to standard output, eliminating from the list any keys that are matched in the index. No changes are made to the index. If the lower-case opcode is used, the values in the list will be converted to lower case before use. The key is extracted from the list line up to the first white space, but the entire line is copied to the output for lines that are not eliminated. Double-quotes surrounding the whole line, or double-quotes surrounding the extracted key in the list are removed if present before the line is tested. |
| S,s,R,r | These opcode values report whether the key is matched in the index or not. The S codes report status by means of the errorlevel (exit code) returned by the PIN program. The 'R' codes also report by means of an output message on the console. If the lower-case opcode is used, the value entered for the key will be converted to lower case before it is tested. |
| L,l | The entire contents of the index file are listed to standard output, in ascending key sequence. No changes are made to the index. If the lower-case opcode is used, the values will be converted to lower case before output. Unlike earlier releases of PIN, only the keys are output, without double-quotes or padding. |
| M,m | The entire contents of the index file are listed to standard output, in ascending key sequence. If the data value of an item is greater than 1, the value is output following the index entry and a tab character. No changes are made to the index. If the lower-case opcode is used, the values will be converted to lower case before output. Unlike earlier releases of PIN, only the keys are output, without double-quotes or padding. |
| Q,q | The entire contents of the index file are listed to standard output, in ascending key sequence. No changes are made to the index. If the lower-case opcode is used, the values will be converted to lower case before output. To match earlier releases of PIN, alphanumeric keys are padded on the right with blanks to the full key length, and the output is enclosed in double-quotes. |
| U,u | The entire contents of the index file are listed to standard output, in ascending key sequence, with any eleven-digit numeric keys starting with a '1' un-normalized by deletion of the first digit. No changes are made to the index. If the lower-case opcode is used, values will be converted to lower case before output. Keys are output without double-quotes or padding. |
| pinno | the number or string to be Added or Deleted. Values longer than the defined key size are truncated on the right. |
| loadfile | a file containing keys to be loaded (or deleted), one per line. Standard Copia variables such as @FFBASE can be used. The special filename "-" (without the quotes) indicates that the file is to be read from 'standard input', which allows a command shell pipe to be used to supply the keys to be loaded. |
Conditional Re-Index
In special cases, when upgrading a system which automates the use of PIN, it may be necessary to convert NDX file to unique-key format while the index is in use. To do this, change an existing "PIN dnsfile a" or "PIN dnsfile f" command in your batch file to read "PIN dnsfile ia" or "PIN dnsfile if" respectively. This special command syntax will rebuild the index, if and only if necessary, then continue to perform the operation specified by the second letter. A data value m may follow the a or f command letter.
The re-index will be deemed necessary if the existing index file header indicates that duplicate keys are permitted in the file. The re-index operation sets the flag that indicates that keys must be unique. This syntax may be used with any combination of an upper or lower case i combined with one of upper- or lower-case a or f.
Exit Codes:
The S, s, R and r operation codes cause the program to return an errorlevel which can be tested in a batch file. The value is 0 if the key is matched or -5, -6 or -7 if the key is not matched; any other value (listed in PIN Errors) indicates that an error occurred in the lookup.
Creation of an index or addition or deletion of a single key value returns 0 in the errorlevel on success and a non-zero result on error.
List addition returns -1 if duplicates were encountered and list deletion returns -1 if missing items were encountered; otherwise 0 is returned if the whole list was successfully processed or another non-zero value if an error occurred.
All operations return -2 if the index cannot be locked after the specified number of retries.
All operations which modify the file fail and return -3 if the index file permits duplicate keys. Re-indexing the file will convert it to a unique key file.
On a re-index, if an error occurs during the backup or list phase, the reset of the index and the reload are skipped.
Additional trace information on the operation of PIN can be obtained by specifying this program name in FFTRACE.
Examples:
Create a numeric index, for example for a do-not-send phone number list:
PIN DNS C :Create DNS.NDX
Create an alphanumeric index, for example for a do-not-send email address list:
PIN DNS_MAIL C40
PIN @FFLOG\DNS_MAIL C64
Add and delete values:
PIN DNS A 1234567890
PIN DNS_DOMN a7 copia.com
PIN DNS_MAIL d steve@copia.com
List index values:
PIN DNS_MAIL L > emails.lst
Load index values:
PIN DNS F faxdns.lst
Clean a list by reference to the index:
PIN DNS E key.lst > newkeylist.lst
Report a lookup result:
c:\copia\faxfacts\log> PIN DNS_MAIL r steve@copia.com
Key steve@copia.com IS matched in file DNS_MAIL.NDX
Normalize a list containing numbers with and without a leading '1':
c:\copia\faxfacts\log> SET NORMALIZE_DNS=1 <- or set in FAXFACTS.CFG
c:\copia\faxfacts\log> PIN DNS i
Added back 185626 records from 249998 (64372 duplicates)
Topic url: http://www.copia.com/support/refmanual/index.html?pin.htm
