Documentation for Anduril Bash API functions

Thu 26 Oct 2023 02:52:04 AM UTC

functions.sh: Anduril related functions.

These functions make it easy to create BASH components in the Anduril framework.

addarray

Writes an entry in the array index
Usage:

 addarray outputportname key file

createarrayindex

Appends index in the array output port with keys matching
the basenames of th files
Usage:

 createarrayindex outputportname

export_command

Exports the _command file as environment variables.
Ports and parameters are prefixed with their type,
i.e. input_ output_ parameter_ metadata_
Usage:

 export_command

exportarraykeys

Exports array values into variables named after the corresponding key
The only parameter is an array input
all non a-zA-Z0-9_ characters are replaced with _.
If the key starts with a number, it is prefixed with "key"
Usage:

 exportarraykeys inputportname

forarray

Run a script for each file in an array, writing the output to
another array. This function replaces the Anduril function
ArrayBashEvaluate. The function uses BashEvaluate input and output
array ports named "array1" and "arrayOut1", unless arguments inArray
and outArray are set. If outArray is a plain folder output port, no index is generated.
The first argument is the script sourced for each array file. Its stdout
is used for the content of the output array file. Refer to the
key and file in the loop with "$key" and "$file". Running index: $i
The second (optional) argument states how the output filenames
are generated: either "keys" or "files". Defaults to "keys".
The "files" mode may result in non-unique file names!
Usage:

 forarray scriptfile [keys] [inArray] [outArray]

getarrayfiles

Return a newline separated list of array files
Usage:

 files=( $( getarrayfiles inputportname ) )

getarrayfile

Return a file for a key
Usage:

 file=$( getarrayfile inputportname keyname )

getarraykeys

Returns a newline separated list of array keys
Usage:

 keys=( $( getarraykeys inputportname ) )

getarraykeyindex

Gets an index number for a specific key
Usage:

 index=( $( getarraykeyindex inputportname key ) )

getbundlepath

Returns bundle path, if found in ANDURIL_BUNDLES
Usage:

 tools_path=$( getbundlepath tools )

getinput

Returns input port from the _command file.
Usage:

 var=$( getinput input_name )

getmetadata

Returns metadata from the _command file.
Usage:

 var=$( getmetadata "metadata_name" )

getoutput

Returns output port from the _command file.
Usage:

 var=$( getoutput output_name )

getparameter

Returns parameter from the _command file.
Usage:

 var=$( getparameter "parameter_name" )

gettempdir

Create and get the temporary dir path:
Usage:

 tmpdir=$( gettempdir )

gettempdir-local

Create and get a local disk temporary dir path:
Usage:

 tmpdir=$( gettempdir-local )

writeerror

Writes a string in the error file, and on the stdErr
Usage:

  writeerror "Something went wrong"

writelog

Writes a string in the log file, and on the stdOut
Usage:

 writelog "This is a message"

generic.sh: Non-Anduril, generic functions

These functions may come handy, even in your interactive shell

calc

Simple calculator using bc.
Second argument is the scale (number of decimals)
- 0 for integer
- defaults to 4
Usage:

 calc "(4+5)/4"


Usage:

 calc "4%5" 0


Usage:

 calc 'b=2; for (i=1;i<=5;i++) { (b+b)^b-i }'

csvcoln

Gets a column number from a csv file header
If column not found, echos 0
Usage:

 coln=$( csvcoln path/to/file.csv "ColName" )


(to extract that column: cut -f $coln path/to/file.csv)

filesize

Return a human readable size from integer of bytes
Usage:

  filesize 10000

iscmd

Checks whether an executable is available,
if not, prints error message, and returns 1
Usage:

 iscmd wget || exit 1

stringtomap

Exports a record converted in to a string.
The reverse function of std.recordToString
Usage:

 stringtomap "key1=value1,key2=value2";  echo $key1

tsvkit.sh: Tab sebarated file functions, wrappers for csvkit

To use the tsv- commands, install csvkit:

sudo apt-get install python-pip; sudo pip install csvkit

c2t

Convert comma separated stream in to tab separated stream
Usage:

 echo "foo,bar" | c2t

header

Print only the first line of input
Usage:

  header file.csv


Usage:

  cat file.csv | header 

noheader

Strip first row of input
Usage:

  noheader file.csv


Usage:

  cat file.csv | noheader

tsvecho

Echo with tab separated values, quoted
Usage:

 tsvecho value1 value2 "some value" > header.csv


Usage:

 echo value1 value2 | tsvecho 

tsvstrip

Strip tsv of quotes
Usage:

 cat file.csv | tsvstrip

tsvtranspose

Transpose a tsv file
Usage:

 cat file.csv | tsvtranspose

tsvhead

Head a file, but print also the header. header not counted in line numbers
Usage:

 cat file | tsvhead -n 30

tsvtail

Tail a file, but print also the header. header not counted in line numbers
Usage:

 cat file | tsvtail -n 30

tsvcut

csvcut with tab-delimited dialect, see original script for options
Usage:

 tsvcut -c Col1,Col3 input1.tsv

tsvformat

csvformat with tab-delimited dialect, see original script for options
Usage:

 tsvformat -c Col2 -m searchString input1.tsv

tsvgrep

csvgrep with tab-delimited dialect, see original script for options
Usage:

 tsvgrep -c Col2 -m searchString input1.tsv

tsvjoin

csvjoin with tab-delimited dialect, see original script for options
Usage:

 tsvjoin -c 1,1 input1.tsv input2.tsv

tsvlook

csvlook with tab-delimited dialect, see original script for options
Usage:

 tsvlook file1.tsv

tsvquery

Simple SQL query for tab-delimited files. The tables are named tsv1,tsv2 ...,
unless named with NAME=FILE syntax. If defining database with -d
it is kept for later queries. A query is not optional(!), but it can be
an empty string.
Usage:

 tsvquery mytable=file1.tsv file2.tsv "SELECT * FROM tsv1,mytable WHERE C4 not like '%NA%' ORDER BY C4 DESC"


Usage:

 tsquery -d DB.sqlite tsv1=file1.tsv "SELECT * FROM tsv1;" # Will keep the database file, reusable later

tsvsort

csvsort with tab-delimited dialect, see original script for options
Usage:

 tsvsort -c Col3 input.tsv

tsvstack

csvstack with tab-delimited dialect, see original script for options
Usage:

 tsvstack file1.tsv file2.tsv

tsvfold

Folds a sequence of line separated arguments into a TSV table with a header you specify,
which means it may easily cause aliasing if you have the wrong number of them.
Any loop which uses "print" to produce field values should ideally work.
Usage:

 cat data | tsv Column1 Column2 ...


Usage:

 seq 100 | tsvfold First Second Third Fourth

tsvdims

Print dimensions of a TSV
Usage:

 tsvdims file.txt


Usage:

 cat file.txt | tsvdims