Thread by @devops_tech

Mastering Linux AWK command!

A Powerful Text-Processing Tool!

A Thread with 15 important examples of awk for Linux admins or DevOps Engineers👇

1/4 - What is Awk? 🧵

awk is a powerful text-processing tool and programming language. It's designed for working with structured text data like CSV files, log files, and more.

2/4 - How Does it Work?

Awk operates on a line-by-line basis, reading input data and applying user-defined actions to lines that match specified patterns. This makes it great for filtering, transforming, and extracting data.

3/4 - Key Features 🧵

Pattern-Action Model:
Awk uses a pattern-action model. When a line matches a pattern, a specified action is executed. This allows for conditional processing.

Fields:
Awk treats each line as a collection of fields, which can be accessed and manipulated easily. Fields are often separated by whitespace or custom delimiters.

Variables:
Awk supports variables and provides built-in variables like NR (line number), NF (number of fields), and more for data manipulation.

Functions:
It has a rich set of built-in functions for text manipulation, arithmetic, and string handling.

4/4 - Use Cases

Awk is incredibly versatile and handy for various tasks:

- Data extraction and reporting
- Data cleaning and transformation
- Log file analysis
- CSV/TSV processing
- Text formatting and reformatting

Here are 15 important examples of awk commands with proper explanation 👇

1/15 - Disk Space Usage 🧵
Use df to list disk usage, and awk to extract the percentage of used space:

df -h | awk '$NF == "/" {print "Usage:", $5}'
# Extracts disk usage percentage for the root file system

2/15 - List Running Processes 🧵
List top CPU-consuming processes using ps and awk:

ps aux | awk '{if ($3 >= 10) print $0}'
# Lists processes using more than 10% CPU

3/15 - Users and Login Times 🧵
Check who is logged in and their login times with who and awk:

who | awk '{print "User:", $1, "Logged in at:", $3, $4}'
# Displays logged-in users and login times

4/15 - Find Large Files 🧵
Find large files in a directory with find and awk:

find /path/to/dir -type f -exec ls -lh {} + | awk '$5 >= 100M {print $5, $9}'
# Lists files larger than 100MB

5/15 - Process Monitoring 🧵
Monitor CPU and memory usage of a specific process with top and awk:

top -p PID -n 1 | awk '/PID/{getline; print "CPU:", $9, "MEM:", $10}'
# Monitors CPU and memory usage of a process by PID

6/15 - User Account Summary 🧵
Generate a summary of user accounts with awk and /etc/passwd:

awk -F':' '{ print "User:", $1, "Home:", $6 }' /etc/passwd
# Lists user accounts and their home directories

7/15 - Log File Analysis 🧵
Analyze log files and extract specific information:

awk '/error/ {print $0}' /var/log/syslog
# Extracts lines containing "error" from syslog

8/15 - Network Interface Info 🧵
Get network interface information using ifconfig and awk:

ifconfig | awk '/inet addr/{print "Interface:", $1, "IP:", $2}'
# Displays network interfaces and their IP addresses

9/15 - Service Status 🧵
Check the status of a service with systemctl and awk:

systemctl is-active service-name | awk '{print "Service Status:", $0}'
# Checks if a service is active or not

10/15 - System Uptime 🧵
Display system uptime using uptime and awk:

uptime | awk '{print "Uptime:", $3, $4}'
# Shows system uptime

11/15 - User Resource Usage 🧵
View resource usage of a specific user with top and awk:

top -b -n 1 -u username | awk '/username/{print "CPU:", $9, "MEM:", $10}'
# Monitors CPU and memory usage of a specific user

12/15 - Check Memory Usage 🧵
Check memory usage using free and awk:

free -m | awk '/Mem/{print "Total Memory:", $2 "MB", "Used Memory:", $3 "MB"}'
# Displays total and used memory

13/15 - List Open Ports 🧵
List open ports using netstat and awk:

netstat -tuln | awk '/LISTEN/{print "Port:", $4}'
# Lists open ports in listening state

14/15 - Find Largest Directories 🧵
Find the largest directories using du, sort, and awk:

du -h /path | sort -rh | awk 'NR<=5{print $2, $1}'
# Lists top 5 largest directories

15/15 - Backup and Rename Files 🧵
Create backups of files and append a timestamp using cp, awk, and date:

cp filename.txt filename_$(date +%Y%m%d%H%M%S).bak
# Creates a backup file with a timestamp

Time for Bonus 💰😍

16/20 - Check CPU Information 🧵
Retrieve detailed CPU information from /proc/cpuinfo with awk:

awk -F: '/model name/ {print "CPU Model:", $2}' /proc/cpuinfo
# Displays CPU model information

17/20 - Monitor Disk I/O 🧵
Monitor disk I/O activity in real-time using iostat and awk:

iostat -x 1 | awk '$1=="sda" {print "Read:", $6 "KB/s, Write:", $7 "KB/s"}'
# Monitors disk I/O for a specific disk (e.g., sda)

18/20 - List Active Network Connections 🧵
List active network connections and their states with ss and awk:

ss -tuln | awk 'NR>1 {print "Protocol:", $1, "State:", $2, "Local Address:", $4}'
# Lists active network connections and their states

19/20 - Check System Load 🧵
Monitor system load averages using uptime and awk:

uptime | awk -F'[a-z]:' '{print "Load Average (1/5/15 min):", $2, $3, $4}'
# Displays system load averages

20/20 - System Performance Snapshot 🧵
Take a snapshot of CPU and memory usage with top and awk:

top -b -n 1 | awk '/Cpu/{print "CPU Usage:", $2 "%"}; /KiB Mem/{print "Memory Usage:", $8}'
# Captures CPU and memory usage snapshot

These awk examples are essential for Linux system administration tasks and can help you efficiently manage and monitor your system. 🐧🔍 #Linux #SystemAdmin #AwkCommands

Repost the thread if you find it useful. Thanks!

https://twitter.com/devops_tech/status/1697885597426434089?s=20