How to Merge Thousands of Small Files Quickly in Unix and Perl

Method 1: The Unix cat Command (Fastest & Simplest)

The cat command (short for concatenate) is specifically designed for this purpose and works incredibly efficiently.

Bash

# Merge all files matching the glob pattern into a single output file
cat *someglob* > output.txt

Method 2: The Perl One-Liner

While cat is generally faster and simpler, here is a Perl version that accomplishes the same task if you specifically need to use Perl:

Perl

# Expands the glob pattern to read all matching .txt files 
# and prints their contents into all.csv
perl -we '@ARGV = glob("@ARGV"); print while (<>);' *.txt > all.csv

---