Documentation for Anduril Bash API functions
Thu 26 Oct 2023 02:52:04 AM UTC
functions.sh: Anduril related functions.
These functions make it easy to create BASH components in the Anduril framework.
addarray
Writes an entry in the array index
Usage:
addarray outputportname key file
createarrayindex
Appends index in the array output port with keys matching
the basenames of th files
Usage:
createarrayindex outputportname
export_command
Exports the _command file as environment variables.
Ports and parameters are prefixed with their type,
i.e. input_ output_ parameter_ metadata_
Usage:
export_command
exportarraykeys
Exports array values into variables named after the corresponding key
The only parameter is an array input
all non a-zA-Z0-9_ characters are replaced with _.
If the key starts with a number, it is prefixed with "key"
Usage:
exportarraykeys inputportname
forarray
Run a script for each file in an array, writing the output to
another array. This function replaces the Anduril function
ArrayBashEvaluate. The function uses BashEvaluate input and output
array ports named "array1" and "arrayOut1", unless arguments inArray
and outArray are set. If outArray is a plain folder output port, no index is generated.
The first argument is the script sourced for each array file. Its stdout
is used for the content of the output array file. Refer to the
key and file in the loop with "$key" and "$file". Running index: $i
The second (optional) argument states how the output filenames
are generated: either "keys" or "files". Defaults to "keys".
The "files" mode may result in non-unique file names!
Usage:
forarray scriptfile [keys] [inArray] [outArray]
getarrayfiles
Return a newline separated list of array files
Usage:
files=( $( getarrayfiles inputportname ) )
getarrayfile
Return a file for a key
Usage:
file=$( getarrayfile inputportname keyname )
getarraykeys
Returns a newline separated list of array keys
Usage:
keys=( $( getarraykeys inputportname ) )
getarraykeyindex
Gets an index number for a specific key
Usage:
index=( $( getarraykeyindex inputportname key ) )
getbundlepath
Returns bundle path, if found in ANDURIL_BUNDLES
Usage:
tools_path=$( getbundlepath tools )
getinput
Returns input port from the _command file.
Usage:
var=$( getinput input_name )
getmetadata
Returns metadata from the _command file.
Usage:
var=$( getmetadata "metadata_name" )
getoutput
Returns output port from the _command file.
Usage:
var=$( getoutput output_name )
getparameter
Returns parameter from the _command file.
Usage:
var=$( getparameter "parameter_name" )
gettempdir
Create and get the temporary dir path:
Usage:
tmpdir=$( gettempdir )
gettempdir-local
Create and get a local disk temporary dir path:
Usage:
tmpdir=$( gettempdir-local )
writeerror
Writes a string in the error file, and on the stdErr
Usage:
writeerror "Something went wrong"
writelog
Writes a string in the log file, and on the stdOut
Usage:
writelog "This is a message"
generic.sh: Non-Anduril, generic functions
These functions may come handy, even in your interactive shell
calc
Simple calculator using bc.
Second argument is the scale (number of decimals)
- 0 for integer
- defaults to 4
Usage:
calc "(4+5)/4"
Usage: calc "4%5" 0
Usage: calc 'b=2; for (i=1;i<=5;i++) { (b+b)^b-i }'
csvcoln
Gets a column number from a csv file header
If column not found, echos 0
Usage:
coln=$( csvcoln path/to/file.csv "ColName" )
(to extract that column: cut -f $coln path/to/file.csv)
filesize
Return a human readable size from integer of bytes
Usage:
filesize 10000
iscmd
Checks whether an executable is available,
if not, prints error message, and returns 1
Usage:
iscmd wget || exit 1
stringtomap
Exports a record converted in to a string.
The reverse function of std.recordToString
Usage:
stringtomap "key1=value1,key2=value2"; echo $key1
tsvkit.sh: Tab sebarated file functions, wrappers for csvkit
To use the tsv- commands, install csvkit:
sudo apt-get install python-pip; sudo pip install csvkit
c2t
Convert comma separated stream in to tab separated stream
Usage:
echo "foo,bar" | c2t
header
Print only the first line of input
Usage:
header file.csv
Usage: cat file.csv | header
noheader
Strip first row of input
Usage:
noheader file.csv
Usage: cat file.csv | noheader
tsvecho
Echo with tab separated values, quoted
Usage:
tsvecho value1 value2 "some value" > header.csv
Usage: echo value1 value2 | tsvecho
tsvstrip
Strip tsv of quotes
Usage:
cat file.csv | tsvstrip
tsvtranspose
Transpose a tsv file
Usage:
cat file.csv | tsvtranspose
tsvhead
Head a file, but print also the header. header not counted in line numbers
Usage:
cat file | tsvhead -n 30
tsvtail
Tail a file, but print also the header. header not counted in line numbers
Usage:
cat file | tsvtail -n 30
tsvcut
csvcut with tab-delimited dialect, see original script for options
Usage:
tsvcut -c Col1,Col3 input1.tsv
tsvformat
csvformat with tab-delimited dialect, see original script for options
Usage:
tsvformat -c Col2 -m searchString input1.tsv
tsvgrep
csvgrep with tab-delimited dialect, see original script for options
Usage:
tsvgrep -c Col2 -m searchString input1.tsv
tsvjoin
csvjoin with tab-delimited dialect, see original script for options
Usage:
tsvjoin -c 1,1 input1.tsv input2.tsv
tsvlook
csvlook with tab-delimited dialect, see original script for options
Usage:
tsvlook file1.tsv
tsvquery
Simple SQL query for tab-delimited files. The tables are named tsv1,tsv2 ...,
unless named with NAME=FILE syntax. If defining database with -d
it is kept for later queries. A query is not optional(!), but it can be
an empty string.
Usage:
tsvquery mytable=file1.tsv file2.tsv "SELECT * FROM tsv1,mytable WHERE C4 not like '%NA%' ORDER BY C4 DESC"
Usage: tsquery -d DB.sqlite tsv1=file1.tsv "SELECT * FROM tsv1;" # Will keep the database file, reusable later
tsvsort
csvsort with tab-delimited dialect, see original script for options
Usage:
tsvsort -c Col3 input.tsv
tsvstack
csvstack with tab-delimited dialect, see original script for options
Usage:
tsvstack file1.tsv file2.tsv
tsvfold
Folds a sequence of line separated arguments into a TSV table with a header you specify,
which means it may easily cause aliasing if you have the wrong number of them.
Any loop which uses "print" to produce field values should ideally work.
Usage:
cat data | tsv Column1 Column2 ...
Usage: seq 100 | tsvfold First Second Third Fourth
tsvdims
Print dimensions of a TSV
Usage:
tsvdims file.txt
Usage: cat file.txt | tsvdims
Generated: Thu 26 Oct 2023 02:52:04 AM UTC