You can script TextPipe's powerful text-processing functions from any
external programming language using TextPipe's COM interface. You can also use the
File Menu\Export functions to generate VBScript or JScript code to create and run a given filter. A sample script is included in the VBScript\sample.vbs file. You can also see
our Web page sample.
The same interface is supported by both TextPipe.Application (the interactive
application) and TextPipeEngine.Application (the ultra-high speed component,
found in the separately purchased TextPipe Engine). The icon XX is used to
indicate methods or functions that do nothing in TextPipeEngine.Application -
most of these functions relate to GUI functions.
| Open and
closing files |
|
| openFilter( filename : string ) |
Opens a filter with the
specified filename |
| saveFilter( filename : string ) |
Saves the current filter with
the given filename |
| close() |
Closes the filter window |
| closeWithoutSave() |
Same as close but closes the
window without prompting even if the filter has been changed |
| property filterChanged : boolean |
Read/Write. True if the current
filter has been saved, False if it has been changed |
| property
scratchPad : string |
Read/Write. The scratch pad text |
| property
trialInput : string |
Read/Write. The trial input text |
| property comments : string |
Read/Write. The comments tab
text. |
| property
isRunning : boolean |
Read Only. True if the filter is
currently running. |
| Filters |
|
Before any filters
are added or any Logging, Input or Output settings are changed, you must
call
startFilters()
first, and then call
endFilters()
when you have finished adding or changing. These function
turn off the GUI, making adding filters much faster. They also prevent the
GUI settings from overwriting the settings you make.
To create sub
filters, you must surround each set of sub filters like this:
startSubfilters()
...define filters here...
endSubfilters()
|
| clearAllFilters() |
Removes all filters from the
filter list |
| moveCursorToEnd() |
Moves the cursor in the treeview
to the end of the list. By default, when a filter is loaded the cursor is on
the Input Filter. If you want to add additional filters you generally will
want to move the cursor to the end of the filter list first. |
| makeSubfilterOf( parent :
treenode, child : treenode ) |
This function has been
superseded by the startSubfilters and endSubfilters commands.
Moves the child filter so that it becomes the last child sub
filters of the parent filter. You must determine if this action is legal or
not yourself - TextPipe does no error checking. |
| addSelectionFilter( type :
integer, locateMethod : integer, param1 : integer, param2 : integer,
moveTo : integer, processIndividually : boolean,
excludeDelimiter : boolean, excludeQuotes : boolean ) : treenode |
Adds a filter from the Restriction
menu. Note that Restriction filters almost always require sub filters in order
to have any effect.
Type - the type of filter to add
0 Delete column
1 Restrict lines
2 Restrict columns
3 Restrict to bytes
4 Restrict CSV fields
5 Restrict tab-delimited fields
6 Remove lines
7 Remove CSV fields
8 Remove Tab-delimited fields
9 Move columns
10 Move CSV columns
11 Move Tab-delimited columns
12 Copy columns
13 Copy CSV columns
14 Copy Tab-delimited columns
15 Restrict CSV fields with column header
16 Restrict tab fields with column header
17 Remove Byte Range
Locate Method - how to determine which areas to affect
0 Restrict %d .. %d
1 Restrict %1:d starting at %0:d
2 Restrict %1:d starting at END - %0:d
3 Restrict %d .. END - %d
4 Restrict END - %d .. END - %d
param1, param2 - the required integer values for the
locate method moveTo : integer - where to move or copy the columns
or fields to. Default 1. processIndividually - whether or
not to apply sub filters to each CSV or Tab field individually, or to the
fields as one string value. Default false. excludeDelimiter - whether or not to include the comma or Tab field
delimiter when passing the field to the sub filter. Default true. excludeQuotes - whether or not to include the CSV quotes that may surround
the field when passing the field to the sub filter. Default true. |
| addGrepFilter( type : integer,
pattern : string, includeLineNumbers : boolean, includeFilename : boolean,
ignoreCase : boolean, countMatches : boolean, patternType : integer ) :
treenode |
Adds a Grep type line based
filter.
type
0 Restrict lines matching
1 Restrict lines NOT matching
2 Extract
3 Retain matching lines
4 Retain non-matching lines
5 Remove matching lines
6 Remove non-matching lines
pattern - the pattern to match
includeLineNumbers - true to include the line number where
the pattern was found, default False
includeFilename - true to include the filename where the
pattern was found, default False
ignoreCase - ignore case when matching the pattern,
default True
countMatches - true to only output a count of the number
of matches, default False
patternType
0 Perl pattern
1 Egrep pattern
2 Brief pattern
3 MS Word pattern
4 EasyPattern
|
| addSplitFilter( type : integer,
splitSize : integer, splitChar : string, splitCharPos : integer,
splitCharCount : integer, splitLines : integer, splitFilename : string ) : treenode |
Adds a split type filter
type
0 Split at a given size
1 Split at a given character
2 Split at a given number of lines
splitSize - the size file to split at
splitChar - the character to split at
splitCharPos -
0 Split before the character (it goes into the next
file)
1 Split after the character (it remains in the first file)
2 Split on top of the character (remove it)
SplitCharCount - the number of times to see SplitChar
before splitting
SplitLines - split after a given number of lines
SplitFilename - the name to give to each output split file |
| addMergeFilter( type : integer,
filename : string ) : treenode |
Adds a merge type filter
type
0 Merge into filename
1 Retain lines found in filename
2 Remove lines found in filename
3 Link filter filename
filename - the filename to use |
| addHeadTailFilter(
includeOrExclude : boolean, linesOrBytes : integer, startOrEnd : boolean,
count : boolean, ) : treeNode |
Adds a Head/Tail type filter
includeOrExclude - whether to include or exclude the text,
False for Include, True for Exclude
linesOrBytes - measure in terms of lines or bytes, False
for Lines, True for Bytes
startOrEnd - measure from the start or end of the file,
False for Start, True for End
Count - the count of lines or bytes to include or exclude |
| addRunFilter( commandLine :
string, inputFilename : string, outputFilename : string ) : treeNode |
Adds a Run External Program
filter commandLine - the command line of the program
to run. Should include double quotes around long filenames, and also usually
includes the input and output filename.
inputFilename - the filename that TextPipe should read
from after the External Program writes to it.
outputFilename - the filename that TextPipe should
write to for the External Program to read in. |
| addSortFilter( type : integer,
reverse : boolean, removeDuplicates : boolean, startColumn : integer, length
: integer ) : treenode |
Adds a sort type filter
type - the sort type
0 ANSI sort (case insensitive)
1 ANSI sort (case sensitive)
2 ASCII sort (case insensitive)
3 ASCII sort (case sensitive)
4 Numeric sort
5 Sort by length of line
6 Sort by date and time
7 Sort by date
8 Sort by time
reverse - whether to reverse the normal sort order, when
set to True the order is Descending, default false
removeDuplicates - whether to remove duplicate values or
not, default true
startColumn - the column to start comparisons from
length - the length of the comparison |
| addDuplicatesFilter( type :
integer, ignoreCase : boolean, startColumn : integer, length : integer ) :
treenode |
Adds a Show or Remove Duplicates
filter
type -
0 Remove duplicate lines
1 Show duplicate lines
ignoreCase - whether to ignore case during comparisons or
not
startColumn - the starting column for the comparison
length - the length of the comparison |
| addReplaceFilter( searchStr :
string, replaceStr : string, type : integer, matchCase : boolean, wholeWord
: boolean, caseSensReplace : boolean, promptOnReplace : boolean, extract :
boolean, firstOnly : boolean, skipPromptIdentical : boolean ) : treenode |
Adds a search and replace (find
and replace) filter. See also the setPerl, setEditDistance and setBufferSize
functions (below) for setting additional parameters.
searchStr - the string to search for
replaceStr - the string to replace it with
type - the find type:
0 Replace
1 Pattern (old style)
2 Sounds like
3 Edit distance
4 Perl pattern
5 Brief pattern
6 Word pattern
7 EasyPattern
matchCase - Matches case when set to True, ignores case
when set to false (default)
wholeWord - Matches whole words only when set to True,
default false
caseSensReplace - Replaces with matching case when set to
True, default false
promptOnReplace - Prompts before replacing when set to
True, default false
extract - when True, all non-matching text is discarded,
default false
firstOnly - when True, only replace the first occurrence,
default False
skipPromptIdentical - when True, don't bother prompting if
the replacement text is identical to the original. |
| addReplaceListFilter( filename :
string, type : integer, matchCase : boolean, wholeWord
: boolean, caseSensReplace : boolean, promptOnReplace : boolean, firstOnly : boolean, skipPromptIdentical : boolean ) : treenode |
Adds a search and replace list filter. See also the setPerl, setEditDistance and setBufferSize
functions (below) for setting additional parameters.
filename - the file to load search/replace pairs from
Other options - as above.
|
| setEditDistance( threshold :
integer ) : boolean |
Sets the edit distance threshold
for the immediately preceding search/replace filter. Default 2. |
| setBufferSize( bufferSize :
integer ) : boolean |
Sets the buffer size for the
immediately preceding search/replace filter. Default 4096. |
| setPerl( bufferSize : integer,
greedy : boolean, allowComments : boolean, dotMatchesNewLines : boolean ) :
boolean |
Sets the perl matching options
for the immediately preceding search/replace filter.
bufferSize - the maximum buffer size to use for matches.
Any match must fit into this buffer, so if you want to match larger pieces
of text, increase the size of this buffer to suit. Default 4096.
greedy - if the pattern finds the longest match (greedy)
or the shortest match. Default false.
allowComments - allow comments in the perl pattern.
Default false.
dotMatchesNewLines - allow the '.' operator to match all
characters, including new lines. Default true. |
| addNumberFilter( type : integer,
value : integer ) : treenode |
Adds a Number-type filter.
type - the type of filter to add
0 Convert Tabs to Spaces
1 Convert Spaces to Tabs
2 Word wrap (column width %d)
3 Pad to width of %d
4 Centre in width of %d
5 Right justify in width of %d
6 Restrict CSV field %d
7 Restrict tab-delimited field %d
8 Truncate to width %d
9 Force to width %d
10 Repeat file %d times
11 Restrict to blocks of length
12 Expand packed decimal
13 Expand zoned decimal
Value - the numeric value to use |
| addInsertFilter( insertPos :
integer, insertString : string ) : treenode |
Adds an insert column-type
filter.
insertPos - the position to insert the string
insertString - the string to insert |
| addDatabaseFilter( connectionStr
: string, mode : integer, fieldDelimiter : string, textQualifier : string,
generateHeader : boolean, insertTable : string, timeout : integer ) :
treenode |
Adds a database-type filter.
connectionStr - the database connection string.
mode
0 Delimited output
1 Fixed width
2 XML
3 Insert script
fieldDelimiter - the string to use between columns
textQualifier - the string to use around string column
values
generateHeader - generates header information when True
insertTable - the name of the insert table
timeout - SQL command timeout in seconds, default 30 |
| addMapFilter( filename : string
) : treenode |
Adds a single byte map-type filter
from a file filename - the filename containing the map
See also prepareMapFilter |
| prepareMapFilter() : treenode
setMap( characterIndex : integer, outputString : string )
setMap( characterIndex : integer, outputString : string )
setMap( characterIndex : integer, outputString : string )
... |
Adds a single byte map-type filter and
allows it to be configured by the following statements
See also addMapFilter |
| prepareUnicodeMapFilter(
unenteredValue : integer, replaceWith : string ) : treenode setUnicodeMap2(
startRange, endRange, outputString : string )
setUnicodeMap( characterIndex : integer,
outputString : string )
setUnicodeMap( characterIndex : integer, outputString : string )
setUnicodeMap( characterIndex : integer, outputString : string )
... |
Adds a Unicode map-type filter
and allows it to be configured by the following setUnicodeMap or
setUnicodeMap2 statements. unenteredValue:
- 0 - Pass through unchanged
- 1 - Pass to subfilter
- 2 - Replace with value (use next option to specify it)
- 3 - Remove
Replace with:
The string to replace it with, if the Replace Type is 2.
|
| addEOLfilter( inputLineFeeds :
integer, fixedLength : integer, outputLineFeeds : integer, newLineFeed :
string, removeBadEOLs : boolean ) : treenode |
Adds an EOL (end of line)
conversion filter
inputLineFeeds - the type of incoming line feeds
0 Unix
1 Mac
2 DOS
3 Auto
4 Fixed (use the fixedLength parameter to specify the length)
fixedLength - the length of fixedLength input files when
option #4 is chosen above.
outputLineFeeds - the type of outgoing line feeds
0 Unix
1 Mac
2 DOS
3 None
4 String (use the newLineFeed parameter to specify the new string)
newLineFeed - the new line feed string on output when
option #4 is chosen.
removeBadEOLs - whether to remove bad EOLs or not |
| addSimpleFilter( type : integer
) : treenode |
Adds a simple filter type,
requiring no special parameters.
type - the type of filter to add.
1 Convert ASCII to EBCDIC
2 Convert EBCDIC to ASCII
3 Convert ANSI to OEM
4 Convert OEM to ANSI
5 Convert to UPPERCASE
6 Convert to lowercase
7 Convert to Title Case
8 Convert to Sentence Case
9 Convert to tOGGLE cASE
10 Remove blank lines
11 Remove blanks from End of Line
12 Remove blanks from Start of Line
13 Remove binary characters
14 Remove ANSI codes
15 Convert IBM drawing characters
16 Remove HTML and SGML
17 Remove backspaces
18 Resolve backspaces
19 Remove multiple whitespace
20 UUEncode
21 Hex Encode
22 Hex Decode
23 MIME Encode (Base 64)
24 MIME Decode (Base 64)
25 MIME Encode (Quoted printable)
26 MIME Decode (Quoted printable)
27 UUDecode
28 Extract email addresses
29 Unscramble (ROT13)
30 Hex dump
32 XXEncode
33 XXDecode
34 Reverse line order
35 Remove email headers
36 Decimal dump
37 HTTP Encode
38 HTTP Decode
39 Randomize lines
40 Create word list
41 Reverse each line
42 Convert to RanDOm case
43 Extract URLs
44 ANSI to Unicode
45 Unicode to ANSI
46 Display debug window
47 Word concordance
48 Delete all
49 Restrict to each line in turn
50 Convert CSV to Tab-delimited
51 Convert CSV to XML' )
52 Convert Tab-delimited to CSV
53 Convert Tab-delimited to XML
54 Convert CSV (with column headers) to XML
55 Convert Tab-delimited (with column headers) to XML
56 Convert CSV (with column headers) to Tab-delimited
57 Convert Tab-delimited (with column headers) to CSV
58 Restrict to file name
59 Convert Word documents to text
60 Swap UTF-16 word order
61 Swap UTF-32 word order
62 Remove BOM (Byte Order Mark)
63 Make Big Endian
64 Make Little Endian
|
| addStringFilter( type : integer,
str : string ) : treenode addStringFilterFile( type
: integer, filename : string ) : treenode |
Adds a string-type filter.
type - the type of filter to add
0 Add left margin
1 Add header
2 Add footer
3 Add right margin
4 Remove lines
5 Retain lines
6 Remove lines matching perl pattern
7 Retain lines matching perl pattern
8 Add text side by side
9 Add repeating text side by side
10 Not Used
11 Not Used
str - the string to use
filename - the filename to use |
| addLineNumberFilter( startNumber
: integer, increment : integer, skipBlankIncrement : boolean,
dontNumberBlank : boolean, numberFormat : string ) : treenode |
Adds a Line Number filter
startNumber - the starting line number
increment - the amount to add for each new line number
skipBlankIncrement - don't increase the line number
for blank lines
dontNumberBlank - don't put a line number on blank lines
numberFormat - the format to use for the line number |
| addScriptFilter( language :
string, script : string, timeout : integer ) : treenode |
Adds an ActiveX script filter
language - the language of the script
script - the code
timeout - the command timeout in seconds, default 30 |
| addCommentFilter( comment :
string ) : treenode |
Adds a comment type filter
comment - the comment. |
| addMathsFilter( operation :
integer; operand : integer ) : treenode |
Adds a maths type filter
operation - the operation to perform
0 +
1 -
2 *
3 div (the remainder is ignored)
4 mod (the remainder after division)
5 xor
6 and
7 or
8 not
9 shift left (0 inserted)
10 shift right (0 inserted)
11 rotate left
12 rotate right
operand - the operand to use |
| addXMLFilter( findType :
integer; tag : string; attribute : string; endTag : string; includeText :
boolean; includeQuotes : boolean; matchCase : boolean ) : treenode |
Adds an HTML/XML filter
findType - the operation to perform
0 restrict to an element
1 restrict to an attribute
2 restrict to between tags
tag - the element or start tag to find attribute
- the attribute to find endTag - the endTag to find includeText -
whether to include the find string in the restriction result (default false)
includeQuotes - whether to include surrounding quotes in the attribute
result or not (default false) matchCase - match case exactly or not
(default false) |
| addUnicodeConversionFilter( toUnicode : boolean;
conversion : integer) : treenode |
Adds a Unicode conversion filter
toUnicode - true to convert TO Unicode, false to convert
FROM Unicode
conversion - the conversion to perform, from 1 to 144 or so. You can
determine the correct value by exporting a filter that contains the
conversion filter you require. |
| Output Filter |
|
| Note - Before
changing these settings you must call startFilters first, and call
endFilters when you have finished. |
| property outputMode : integer |
Read/Write. Sets the output mode:
0 Output to clipboard (all files are joined together)
1 Output to files
2 Output merged (joined) to a single file
|
| property outputTestMode :
integer |
Read/Write. Sets the output
filter test mode.
0 Normal mode
1 test mode
2 Backup mode. |
| property outputFolder : string |
Read/Write. Sets the output
filter folder. |
| property outputRetainDate :
boolean |
Read/Write. Sets the output
filter retain date setting. True turns on the retain date setting, default
False. |
| property outputMergeFile :
string |
Read/Write. Sets the output
filter to merge data to a single file, using the given filename. |
| property outputExtension :
string |
Read/Write. Sets the output
filter extension. |
| property outputAppend : boolean |
Read/Write. Sets append mode
when True, default False. |
| property
outputOnlyOutputChangedFiles : boolean |
Read/Write. Only outputs files
that have actually changed, default True. |
| property outputKeepStructure :
boolean |
Read/Write. When used in
conjunction with outputFolder, determines if the original folder structure
is retained, default False. |
| addSecondaryOutputFilter( mode :
integer, testMode : integer, retainDate : boolean, append : boolean,
onlyOutputChangedFiles : boolean, folder : string, extension : string,
keepStructure : boolean, mergeFilename : string ) : treenode |
Adds a secondary output filter,
usually used for splitting some lines or records to a new file. All
parameters are as above for a normal Output filter. |
| Input Filter |
|
| Note - Before
changing these settings you must call startFilters first, and call
endFilters when you have finished. |
| property InputMode : integer |
Read/Write. Sets the input mode:
0 Input from clipboard
1 Input from files (default). You can add files using the addFile
function.
2 Input from Trial Run area. You can set the Trial Run text using the
trialInput property
3 Input from random file generator. You can set the
|
| property inputBinaryFiles :
integer |
Read/Write. Controls how binary
files are handled
value (default 0)
0 Binary files are processed
1 Binary files are skipped
2 Binary files are confirmed before processing
|
| property InputBinarySampleSize :
integer |
Read/Write. Default 100.
Controls the binary sample size for Input files |
| property InputPromptOnEach :
boolean |
Read/Write. Default False.
Controls whether to prompt before processing each file. |
| property InputPromptOnReadOnly :
boolean |
Read/Write. Default False.
Controls whether to prompt before processing each read only file. |
| property InputDeleteFiles :
boolean |
Read/Write. Default False.
Controls if input files are deleted after processing. Use with caution! |
| property inputRandom : integer |
Read/Write. Sets the input to
come from a randomly generated file of size size, default 1024 |
| Logging
functions |
|
| Note - Before
changing these settings you must call startFilters first, and call
endFilters when you have finished. |
| property logEnabled : boolean |
Read/Write. When True, logging
is on, default False |
| property logFilename : string |
Read/Write. The file to log data
to |
| property logAppend : boolean |
Read/Write. When True, logging
appends to the existing file |
| property logThreshold : integer |
Read/Write. When more than
threshold lines are present, all get written to the log file (this is
for interactive mode only, for non-interactive mode ALL lines get written to
the log file) |
| Running
Filters |
|
| execute() : boolean |
Runs the current filter (same as
the 'Go' button). This function does not return until the thread completes.
Returns true on success, False on failure (with error message in errorText) |
| executeClipboard() : boolean |
Runs the current filter with
input from and output to the Clipboard (same as the 'Clipboard' button).
This function does not return until the thread completes.
Returns
true on success, False on failure (with error message in errorText) |
| processString( inputText :
string ) : string |
Processes the string inputText
and returns the result string, just as though the entire string was in a
file. If an error occurs, errorText will be set to the error
message, or blank if there is no error. |
| compileFilter() : string |
Compiles the filter into an
internal form and saves it so that the compiling doesn't have to be repeated
each time text is processed. This function is ONLY used in conjunction with
processString(s)WithCompiledFilter - it does not speed up any other process.
Any changes made to the filter list after the filter was compiled
will not be reflected. Also, the normal Input and Output filters are
ignored - a string input and output filter is used instead.
Returns a blank string on success, error message on failure. errorText is
also cleared if there is no error, or set if there is an error. |
| processStringWithCompiledFilter(
inputText : string ) : string |
Processes the string inputText
using the pre-compiled filter. If the filter list has not been
compiled with compileFilter this function will return an empty string. This
function calls startJob and endJob at the beginning and end of processing.
Note the singular - String |
compiledFilterStartJob()
processStringsWithCompiledFilter(
inputText : string ) : string
...
(usually called multiple times)
...
compiledFilterEndJob() |
Processes the string inputText
using the pre-compiled filter. If the filter list has not been
compiled with compileFilter this function will return an empty string. This
function DOES NOT call startJob and endJob at the beginning and end of
processing - they must be called separately. This function is normally
called a number of times. Note the plural - Strings compiledFilterStartJob() - this function prepares a compiled filter for
processing.
compiledFilterEndJob() - this function completes a
compiled filter's processing. |
| isRunning() : boolean |
Returns true if a processing
thread is running |
| property errorText :
string |
Returns the text of the last
error, or blank if there was no last error. |
| Adding Files |
|
| clearAllFiles() |
Clears all files from the file
list |
| addFile( fileSpec : string,
subfolders : integer, action : integer ) |
Adds a file to the file list.
fileSpec - the filename, folder or wildcard
subfolders - how many subfolders to include (default 255)
0 No subfolders
1..9 1..9 Subfolder(s)
255 All subfolders
action - the action column (default 1)
1 Include the files
2 Exclude the files
3 Ignore the files
|
| addFileList( filename : string ) |
Adds a list of files to the file
list.
filename - the list of files is contained in this file. |