Posterous theme by Cory Watilo

quiet software blog

Short notes from a busy programmer

Getting to the middle of a large file

I've been dealing with some fairly big log files recently, most of them over 100mb and text editors don't really like loading up files that large. Fortunately from I found a way to move forward from http://www.fastechws.com/tricks/unix/head_tail_mid_files.php  I created a script called mid:

</div>
<div>#!/bin/sh</div>
<div>if test $# -lt 2; then</div>
<div>    echo "$0: insufficient arguments on the command line." >&2</div>
<div>    echo "usage: $0 startlinenum numlines [filename]" >&2</div>
<div>    exit 1</div>
<div>fi</div>
<div>tail -n +$1 $3 | head -n $2</div>
<div>exit $?</div>
<div>

Then I started a sort of binary search, starting with:
mid 200000 20 logfile.log
and visually checking if the information I wanted was there, this worked ok because I was looking for a specific date range, if I was looking for strings I'd have used
grep -ni "search-terms" logfile.log
-n prints line numbers for every match.
Once I'd found the section I was looking for
mid 453000 4000 logfile.log > logsection.log
Got me a new file that a text editor can handle.

Drop down terminals

I have an awful habit of opening new terminals every 2 minutes because I've left something running on the open one or I've left it behind a window on another screen.

As a way to keep them all in once place I've started using a lovely drop down terminal called Guake on Ubuntu. Named after the classic drop down terminal in the game Quake it has tabs, a fullscreen mode, and it toggles on F12.

$ sudo apt-get install guake

My Mac-wielding office cohorts seem to think that Visor is a decent alternative for that platform so if you're stuck using one of those pretty shiny things give it a go.

Emacs - browsing the kill ring

The browse-kill-ring in debian/ubuntu emacs-goodies can be set to run when you use M-y without having yanked anything first. It displays a list of everything in the kill ring in an extra buffer allowing you to choose what to yank from a list rather than yank something then cycle through the rest with M-y.

Great if you have a bad memory like me. Set it up with:

sudo apt-get install emacs-goodies-el

Then stick this in your .emacs

 (when (require 'browse-kill-ring nil 'noerror)
   (browse-kill-ring-default-keybindings))

Emacs replace with newline

I've always had trouble replacing things in emacs with newlines, for example here's how to spread out an xml file with no whitespace

M-shift-%
><
>C-q C-j<

We're effectively just putting a line break between the > and  < so 

<xml_tag attr="random_stuff"><more_xml /></xml_tag>

becomes

<xml_tag attr="random_stuff">
<more_xml />
</xml_tag>

Not clever but it makes large bodies on text easier to break up sometimes.

Piping django commands into mysql

Some of the management commands in Django (sqlflush and sqlclear spring to mind) rather understandably don't act straight on the database as they delete large amounts of data. They instead print out the sql commands for you to use at your leisure.

Being a lazy sort I rarely feel like copying and pasting things about so here's how to pipe sqlclear straight into mysql:

python manage.py sqlclear [list of app names] | mysql -u [username] -p[password] -D [database name]

sqlflush doesn't take any arguments so:

python manage.py sqlflush [list of app names] | mysql -u [username] -p[password] -D [database name]

So, now we've sorted out how to get round what the Django guys were trying to stop please don't delete any of your data, and if you do tell your wife, kids or dog about it rather than me.

The plan

Hi, if you're reading this I probably don't know you. I'm a programmer and this is the beginnings of my blog.

Quiet Software aims to write quality applications to help small businesses become a bit more efficient. I'll post bits of the internet up here that interest me, things I've learned, and news about projects in the pipeline.

Check back soon for more.

​