Bash String Manipulation

1. Why and How: The Core of Parameter Expansion

At its simplest, you've been using parameter expansion every time you've written $MY_VAR. The more explicit form, ${MY_VAR}, is where the real power lies. Using braces {} not only prevents ambiguity (e.g., echo "${MY_VAR}_suffix" vs. $MY_VAR_suffix) but also unlocks a rich set of operators for string manipulation.

Before we dive into the techniques, it's crucial to understand why this is a preferred method in many cases.

As you've just read, the performance gain from avoiding subprocesses can be significant, especially inside loops that process many files or data points.

2. Slicing and Dicing: Substring Extraction

One of the most common string operations is extracting a piece of a string, or a "slice." Parameter expansion provides a clean syntax for this.

• String Length: ${#variable} returns the number of characters in the variable.
• Substring Extraction (Slicing): ${variable:offset:length} extracts a substring of length characters starting from offset (zero-indexed).

Let's look at a concrete example:

Notice how var:0:4 extracts "Bash". If you omit the length, it extracts everything from the offset to the end of the string.

Now, let's see these concepts demonstrated in a live terminal session.

3. Pattern-Based Manipulation

Where parameter expansion truly begins to replace tools like sed, cut, basename, and dirname is in its pattern-matching operators.

Prefix and Suffix Removal

This is incredibly useful for tasks like manipulating file paths.

• Remove from Front (Prefix):
  • ${variable#pattern}: Deletes the shortest match of pattern from the beginning.
  • ${variable##pattern}: Deletes the longest (greedy) match of pattern from the beginning.
• Remove from End (Suffix):
  • ${variable%pattern}: Deletes the shortest match of pattern from the end.
  • ${variable%%pattern}: Deletes the longest (greedy) match of pattern from the end.

The distinction between "shortest" and "longest" is key. For example, given file="report.tar.gz", ${file%.*} removes .gz (shortest match), while ${file%%.*} removes .tar.gz (longest match).

Search and Replace

You can also perform search-and-replace operations directly in the shell.

• ${variable/pattern/string}: Replaces the first match of pattern with string.
• ${variable//pattern/string}: Replaces all matches of pattern with string.

If you omit /string, the matched pattern is deleted.

You can also "anchor" the replacement to the beginning or end of the string:

• ${variable/#pattern/string}: Replaces only if pattern matches at the beginning.
• ${variable/%pattern/string}: Replaces only if pattern matches at the end.

4. Case Modification

Since Bash 4.0, you can also change the case of characters in a string, which is very useful for normalizing input.

• ${variable^}: Uppercase the first character.
• ${variable^^}: Uppercase all characters.
• ${variable,}: Lowercase the first character.
• ${variable,,}: Lowercase all characters.

env="PROD"
echo "Deploying to ${env,,} environment."



# Output: Deploying to prod environment.

This is a much cleaner and more efficient way to handle case changes than piping to tr or awk.

5. Putting It All Together: A Practical Scenario

Let's combine these techniques to solve a common problem. Imagine you have a script that processes files, and you need to normalize the filenames for logging.

Input: filepath="/data/RAW_SALES_2026-Q1.CSV"
Desired Output: raw_sales_2026-q1

How would you achieve this? Think about it for a moment. You'll need to chain a few operations.

...

Here's the sequence:

1. Extract the filename from the path. We need to remove the longest match of */ from the beginning.

   filename="${filepath##*/}" 
   
   
   
   # filename is now "RAW_SALES_2026-Q1.CSV"
   

2. Remove the file extension. We need to remove the shortest match of .* from the end.

   base="${filename%.*}"
   
   
   
   # base is now "RAW_SALES_2026-Q1"
   

3. Convert the result to lowercase.

   normalized="${base,,}"
   
   
   
   # normalized is now "raw_sales_2026-q1"
   

Putting it all on one line can be less readable, but it is possible by nesting temporary variables:

filepath="/data/RAW_SALES_2026-Q1.CSV"
tmp="${filepath##*/}"
echo "${tmp%.*}" | tr '[:upper:]' '[:lower:]' # Using tr for older bash



# or with bash 4+
normalized_base="${${filepath##*/}%.*}" # This syntax can be tricky
normalized="${normalized_base,,}"
echo "$normalized"

The step-by-step approach is generally clearer and easier to debug.

Conclusion

Today you've added a powerful set of native Bash tools to your scripting arsenal. By mastering parameter expansion, you can write more efficient, readable, and self-reliant scripts that don't depend on external commands for basic string manipulation.

Key Takeaways:

• Always use "${variable}" with braces for clarity and to enable operators.
• Slicing (${var:offset:length}) allows you to extract substrings without cut.
• Pattern Removal (#, ##, %, %%) is a fast replacement for basename, dirname, and simple sed commands for stripping prefixes and suffixes.
• Search and Replace (/, //) provides sed-like substitution capabilities directly within the shell.
• Case Modification (^, ^^, ,, ,,) is the modern, efficient way to normalize string case, replacing tr.

Preview of the Next Lesson:
We've mastered operating on single string values. But what happens when you need to manage a list of items, like a set of server hostnames or a list of files to process? In the next lesson, we will explore how to use indexed and associative arrays to manage collections of data in Bash scripts, moving from single variables to structured data management.