Skip to main content

Shell Scripting

Status

Master Plan - Comprehensive init.md complete, ready for SKILL.md implementation

Shell scripting remains indispensable for system automation. This skill covers robust bash/sh scripting with proper error handling, argument parsing, portability (POSIX sh vs Bash), testing, and integration with common utilities.

Scope​

This skill teaches:

  • Error Handling - set -euo pipefail, trap handlers, defensive programming
  • Argument Parsing - getopts, long options, usage messages
  • Parameter Expansion - String manipulation, default values, arrays
  • Portability - POSIX sh vs Bash, cross-platform scripting
  • Utility Integration - jq, yq, awk, sed, grep, find
  • Testing and Linting - shellcheck, bats testing framework

Key Components​

Error Handling Patterns​

Fail-Fast with set Options:

#!/usr/bin/env bash
set -euo pipefail  # Exit on error, undefined vars, pipe failures
  • -e Exit immediately if command fails
  • -u Treat unset variables as errors
  • -o pipefail Return exit code from failed pipe command

Trap Handlers for Cleanup:

cleanup() {
    rm -f "$temp_file"
    echo "Cleanup completed"
}
trap cleanup EXIT  # Always run cleanup
trap 'echo "Error on line $LINENO"' ERR

Explicit Error Checking:

if ! command_that_might_fail; then
    echo "Error: command failed" >&2
    exit 1
fi

Argument Parsing​

Basic getopts:

while getopts "f:v" opt; do
    case $opt in
        f) file="$OPTARG" ;;
        v) verbose=true ;;
        *) usage; exit 1 ;;
    esac
done

Long Options (Bash):

while [[ $# -gt 0 ]]; do
    case $1 in
        --file) file="$2"; shift 2 ;;
        --verbose) verbose=true; shift ;;
        *) echo "Unknown option: $1"; exit 1 ;;
    esac
done

Parameter Expansion​

String Manipulation:

${var#pattern}     # Remove shortest match from beginning
${var##pattern}    # Remove longest match from beginning
${var%pattern}     # Remove shortest match from end
${var%%pattern}    # Remove longest match from end
${var/old/new}     # Replace first occurrence
${var//old/new}    # Replace all occurrences

Default Values:

${var:-default}    # Use default if var is unset
${var:=default}    # Set var to default if unset
${var:?error}      # Exit with error if var is unset
${var:+value}      # Use value if var is set

Utility Integration​

jq - JSON Processing:

curl -s api.example.com/data | jq '.results[] | .name'

yq - YAML Processing:

yq eval '.services.webapp.image' docker-compose.yml

awk - Text Processing:

ps aux | awk '$3 > 50 { print $2, $11 }'  # High CPU processes

sed - Stream Editing:

sed -i 's/old/new/g' file.txt  # Replace in-place

Decision Framework​

POSIX sh vs Bash?

Need portability (Alpine, embedded)?
  YES → Use POSIX sh (#!/bin/sh)
        Avoid: arrays, [[, ${var//}, process substitution

Need advanced features?
  YES → Use Bash (#!/usr/bin/env bash)
        Available: arrays, [[, ${var//}, <()

CI/CD environment controlled?
  YES → Bash usually available
        → Use Bash for richer features

Docker/container context?
  Alpine base? → POSIX sh (no bash by default)
  Debian/Ubuntu base? → Bash available

When to Use Shell vs Python/Go?

Use Shell when:
  - System commands are 80%+ of logic
  - Rapid prototyping/throwaway scripts
  - Zero dependencies required
  - Pipeline composition (cmd1 | cmd2)

Use Python/Go when:
  - Complex data structures needed
  - Network/API heavy operations
  - Long-lived daemon processes
  - Unit testing critical
  - Team unfamiliar with shell

Tool Recommendations​

Linting and Testing​

shellcheck - Static analysis for shell scripts

shellcheck script.sh  # Find bugs and portability issues

Common issues detected:

  • Unquoted variables
  • Useless uses of cat
  • Incorrect test operators
  • Portability warnings

bats - Bash Automated Testing System

# test/script.bats
@test "addition works" {
    result="$(echo '2 + 2' | bc)"
    [ "$result" -eq 4 ]
}

bats test/script.bats  # Run tests

shfmt - Shell script formatter

shfmt -w script.sh  # Format in-place

Utility Tools​

jq - JSON processor yq - YAML processor fzf - Fuzzy finder for interactive selection parallel - Run commands in parallel envsubst - Environment variable substitution

Integration Points​

With Other Skills:

  • administering-linux - Automate system tasks
  • building-ci-pipelines - CI/CD build scripts
  • managing-configuration - Wrapper scripts for Ansible/Terraform
  • operating-kubernetes - kubectl automation scripts
  • writing-infrastructure-code - IaC helper scripts

Use Cases:

  • Docker entrypoint scripts
  • CI/CD pipeline steps (GitHub Actions, GitLab CI)
  • Cron jobs and scheduled tasks
  • System initialization scripts
  • Deployment automation

Learn More​