Getting to the middle of a large file
I've been dealing with some fairly big log files recently, most of them over 100mb and text editors don't really like loading up files that large. Fortunately from I found a way to move forward from http://www.fastechws.com/tricks/unix/head_tail_mid_files.php I created a script called mid:
</div> <div>#!/bin/sh</div> <div>if test $# -lt 2; then</div> <div> echo "$0: insufficient arguments on the command line." >&2</div> <div> echo "usage: $0 startlinenum numlines [filename]" >&2</div> <div> exit 1</div> <div>fi</div> <div>tail -n +$1 $3 | head -n $2</div> <div>exit $?</div> <div>
Then I started a sort of binary search, starting with:
mid 200000 20 logfile.log
and visually checking if the information I wanted was there, this worked ok because I was looking for a specific date range, if I was looking for strings I'd have used
grep -ni "search-terms" logfile.log
-n prints line numbers for every match.
Once I'd found the section I was looking for
mid 453000 4000 logfile.log > logsection.log
Got me a new file that a text editor can handle.
