bash - How can I iterate over .log files, process them through awk, and replace with output files with different extensions? -

let's have multiple .log files on prod unix machine(sunos) in directory: example:

ls -tlr                                                                                                                                                                                                                      total 0                                                                                                                                                                                                                                -rw-r--r-- 1 21922 21922 0 sep 10 13:15 file2017-01.log                                                                                                                                                                                -rw-r--r-- 1 21922 21922 0 sep 10 13:15 file2016-02.log                                                                                                                                                                                -rw-r--r-- 1 21922 21922 0 sep 10 13:15 todo2015-01.log                                                                                                                                                                                -rw-r--r-- 1 21922 21922 0 sep 10 13:15 fix20150223.log  

the purpose here via nawk extract specific info logs( parse logs ) , "transform" them .csv files in order load them oracle tables afterwards. although nawk has been tested , works charm, how automate bash script following:

1) list of given files in path

2) nawk (to extraction of specific data/info log file)

3) output separately each file unique .csv directory

4) remove .log files path

what concern me loadstamp/timestamp on each file ending different. have implemented script works latest date. (eg. last month). want load historical data , bit stuck.

to visualize, desired/target output this:

bash-4.4$ ls -tlr                                                                                                                                                                                                                      total 0                                                                                                                                                                                                                                -rw-r--r-- 1 21922 21922 0 sep 10 13:15 file2017-01.csv                                                                                                                                                                                -rw-r--r-- 1 21922 21922 0 sep 10 13:15 file2016-02.csv                                                                                                                                                                                -rw-r--r-- 1 21922 21922 0 sep 10 13:15 todo2015-01.csv                                                                                                                                                                                -rw-r--r-- 1 21922 21922 0 sep 10 13:15 fix20150223.csv  

how bash script please achieved? loading takes 1 time, it's historical mentioned. extremely useful.

an implementation written readability rather terseness might like:

#!/usr/bin/env bash infile in *.log;   outfile=${infile%.log}.csv   if awk -f yourscript <"$infile" >"$outfile";     rm -f -- "$infile"   else     echo "processing of $infile failed" >&2     rm -f -- "$outfile"   fi done 

---

to understand how works, see:

• globbing -- mechanism *.log replaced list of files extension.
• the classic for loop -- for infile in syntax, used iterate on results of glob above.
• parameter expansion -- ${infile%.log} syntax, used expand contents of infile variable .log suffix pruned.
• redirection -- syntax used in <"$infile" , >"$outfile", opening stdin , stdout attached named files; or >&2, redirecting logs stderr. (thus, when run awk, stdin connected .log file, , stdout connected .csv file).