awk - How to process fields between matching patterns? -
related post: how select lines between 2 patterns?
@fedorqui, providing these different options awk. have been using parse through var log messages when troubleshooting ooms , works great. want extend further have not been able figure out how proceed. trying do:
print lines between rss , out of memory. have done example
order sections between each match rss field. have not been able figure out
add column own header , perform mathematical operation. have been able running formatting issues. not sure how skip first , last line when adding column lose lines. not able keep spacing original if operations other print.
here's command using right now:
less /var/log/messages'|awk '/swapents/{x=1; print "=================="};/out of memory/{x=0} x'|sed 's/[]\[]//g' here's source data:
sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.617265 pid uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.622250 1828 0 1828 4331 116 14 3 0 -1000 udevd sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.627310 2664 0 2664 28002 53 23 3 0 -1000 auditd sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.633181 2680 0 2680 62032 1181 24 4 0 0 rsyslogd sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.638888 2694 0 2694 3444 61 11 3 0 0 irqbalance sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.644912 2710 81 2710 5430 56 14 3 0 0 dbus-daemon sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.651108 2779 0 2779 19958 203 42 3 0 -1000 sshd sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.656670 2789 0 2789 5622 56 17 3 0 0 xinetd sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.653452 out of memory: kill process 43390 (mysql) score 1000 or sacrifice child blah blah blah sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.617265 pid uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.622250 1828 0 1828 4331 116 14 3 0 -1000 udevd sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.627310 2664 0 2664 28002 53 23 3 0 -1000 auditd sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.633181 2680 0 2680 62032 1181 24 4 0 0 rsyslogd sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.638888 2694 0 2694 3444 61 11 3 0 0 irqbalance sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.644912 2710 81 2710 5430 56 14 3 0 0 dbus-daemon sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.651108 2779 0 2779 19958 203 42 3 0 -1000 sshd sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.656670 2789 0 2789 5622 56 17 3 0 0 xinetd sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.653452 out of memory: kill process 43390 (mysql) score 1000 or sacrifice child here's output looks like:
================== 0mb sep 8 11:35:15 pid 0mb name <---- should header (pid virt rss etc) sep 8 11:35:15 1828 0mb udevd sep 8 11:35:15 2664 0mb auditd sep 8 11:35:15 2680 4mb rsyslogd sep 8 11:35:15 2694 0mb irqbalance sep 8 11:35:15 2710 0mb dbus-daemon sep 8 11:35:15 2779 0mb sshd sep 8 11:35:15 2789 0mb xinetd sep 8 11:35:15 2822 0mb crond sep 8 11:35:15 out 0mb or <---- should footer (out of memory etc) ================== 0mb sep 8 11:35:15 pid 0mb name <---- should header (pid virt rss etc) sep 8 11:35:15 1828 0mb udevd sep 8 11:35:15 2664 0mb auditd sep 8 11:35:15 2680 4mb rsyslogd sep 8 11:35:15 2694 0mb irqbalance sep 8 11:35:15 2710 0mb dbus-daemon sep 8 11:35:15 2779 0mb sshd sep 8 11:35:15 2789 0mb xinetd sep 8 11:35:15 2822 0mb crond sep 8 11:35:15 out 0mb or <---- should footer (out of memory etc) ================== 0mb you can see output separator added each oom field, awk tries calculate values it, love avoid if possible. header , footer getting chopped off , nice avoid too.
here's like:
======================== sep 8 11:35:15 pid rss memused_mb oom_score_adj name sep 8 11:35:15 2664 53 {rss*4/1024} -1000 auditd sep 8 11:35:15 2789 56 {rss*4/1024} 0 xinetd sep 8 11:35:15 2710 56 {rss*4/1024} 0 dbus-dae sep 8 11:35:15 2694 61 {rss*4/1024} 0 irqbalan sep 8 11:35:15 1828 116 {rss*4/1024} -1000 udevd sep 8 11:35:15 2680 181 {rss*4/1024} 0 rsyslogd sep 8 11:35:15 2779 203 {rss*4/1024} -1000 sshd sep 8 11:35:15 out of memory: kill process 43390 (mysql) score 1000 or sacrifice child ======================== sep 8 11:35:15 pid rss memused_mb oom_score_adj name sep 8 11:35:15 2664 53 {rss*4/1024} -1000 auditd sep 8 11:35:15 2789 56 {rss*4/1024} 0 xinetd sep 8 11:35:15 2710 56 {rss*4/1024} 0 dbus-dae sep 8 11:35:15 2694 61 {rss*4/1024} 0 irqbalan sep 8 11:35:15 1828 116 {rss*4/1024} -1000 udevd sep 8 11:35:15 2680 181 {rss*4/1024} 0 rsyslogd sep 8 11:35:15 2779 203 {rss*4/1024} -1000 sshd sep 8 11:35:15 out of memory: kill process 43390 (mysql) score 1000 or sacrifice child ========================
awk solution:
$ cat tst.awk /swapents/ { x=1; print "==================" printf( "%s %s %s pid\t%4s\tmemused_mb\toom_score_adj\tname\n", $1, $2, $3, "rss"); next } /out of memory/ { printf( "%s %s %s %s\n", $1, $2, $3, substr($0,index($0,$7))); x=0 } x { printf( "%s %s %s %s\t%4d\t%10.5f\t%13d\t%s\n", $1, $2, $3, $7, $11, ($11*4)/1024, $15, $16 ) } you can play around formatting, precision in column 6, using specifiers in printf function. call with:
$ awk -f tst.awk /var/log/messages edit: with sorting
op asked sort output rss columns. using standard sort wouldn't work here because want sort in between starting , ending matches. can solve saving intermediate result in array , sort self-defined function. this:
$ cat tst2.awk /swapents/ { x=1; print "==================" printf( "%s %s %s pid\t%4s\tmemused_mb\toom_score_adj\tname\n", $1, $2, $3, "rss"); next } /out of memory/ { n=asort(a, sorted, "cmp_rss") (i=1; i<=n; i++) { print sorted[i] } delete a; printf( "%s %s %s %s\n", $1, $2, $3, substr($0,index($0,$7))); x=0 } x { a[i++] = sprintf( "%s %s %s %s\t%4d\t%10.5f\t%13d\t%s", $1, $2, $3, $7, $11, ($11*4)/1024, $15, $16 ); } function cmp_rss(i1, v1, i2, v2) { split(v1, a1, " ") split(v2, a2, " ") rss1=a1[5]; rss2=a2[5]; return (rss1 - rss2) } which leads to:
$ awk -f tst2.awk input.txt ================== sep 8 11:35:15 pid rss memused_mb oom_score_adj name sep 8 11:35:15 2664 53 0.20703 -1000 auditd sep 8 11:35:15 2710 56 0.21875 0 dbus-daemon sep 8 11:35:15 2789 56 0.21875 0 xinetd sep 8 11:35:15 2694 61 0.23828 0 irqbalance sep 8 11:35:15 1828 116 0.45312 -1000 udevd sep 8 11:35:15 2779 203 0.79297 -1000 sshd sep 8 11:35:15 2680 1181 4.61328 0 rsyslogd sep 8 11:35:15 out of memory: kill process 43390 (mysql) score 1000 or sacrifice child ================== sep 8 11:35:15 pid rss memused_mb oom_score_adj name sep 8 11:35:15 2664 53 0.20703 -1000 auditd sep 8 11:35:15 2710 56 0.21875 0 dbus-daemon sep 8 11:35:15 2789 56 0.21875 0 xinetd sep 8 11:35:15 2694 61 0.23828 0 irqbalance sep 8 11:35:15 1828 116 0.45312 -1000 udevd sep 8 11:35:15 2779 203 0.79297 -1000 sshd sep 8 11:35:15 2680 1181 4.61328 0 rsyslogd sep 8 11:35:15 out of memory: kill process 43390 (mysql) score 1000 or sacrifice child 
Comments
Post a Comment