How can I split a file into line ranges, e.g. lines 1-10, 11-20, 21-30?
Some Unix systems provide the split utility for this purpose:
split --lines 10 --numeric-suffixes input.txt output-
For more flexibility you can use sed. The sed command can print e.g. the line number range 1-10:
sed -n -e '1,10p' -e '10q'
This stops sed from printing each line (-n). Instead it only processes the lines in the range 1-10 ("1,10"), and prints them ("p"). The command will quit after reading line 10 ("10q").
We can now use this to print an arbitrary range of a file (specified by line number):
# POSIX shell
file=/etc/passwd
range=10
cur=1
last=$(wc -l < "$file") # count number of lines
chunk=1
while [ $cur -lt $last ]
do
endofchunk=$(($cur + $range - 1))
sed -n -e "$cur,${endofchunk}p" -e "${endofchunk}q" "$file" > chunk.$(printf %04d $chunk)
chunk=$(($chunk + 1))
cur=$(($cur + $range))
doneThe previous example uses POSIX arithmetic, which older Bourne shells do not have. In that case the following example should be used instead:
# legacy Bourne shell; assume no printf either
file=/etc/passwd
range=10
cur=1
last=`wc -l < "$file"` # count number of lines
chunk=1
while test $cur -lt $last
do
endofchunk=`expr $cur + $range - 1`
sed -n -e "$cur,${endofchunk}p" -e "${endofchunk}q" "$file" > chunk.$chunk
chunk=`expr $chunk + 1`
cur=`expr $cur + $range`
doneAwk can also be used to produce a more or less equivalent result:
awk -v range=10 '{print > FILENAME "." (int((NR -1)/ range)+1)}' file