maelstrom

bash prompts for the common man

Setting your bash prompt is one of those geek machismo things that usually culminates in something like

export PS1='\[\e]0;\w\a\]\n\[\e[32m\]\u@\h \[\e[33m\]\w\n\[\e[0m\]$ '

the idea being that lots of escape sequences = eliteness. (After a while you only see blondes, brunettes, and redheads.) Though, I'd guess most people just copy someone else's bash prompt and foist it off as their own, rather than learn ansi / xterm / bash escape sequences. Like me initially. :)

However, you can easily make your prompt setup readable by breaking it down. I've started doing this in my prompt file.

My motivation initially was to avoid beeping console prompts. The xterm escape sequence to set the window title contains a bell character, which was of course interpreted by xterm and friends, but not when I'd sit down at system consoles (where usually TERM=cons25). I needed to set PS1 according to TERM.

In the course of things, I discovered the \t bash escape sequence, which gives you the current time in hh:mm:ss form. Nice. By incorporating this into the prompt you can now tell by inspection how long you've been sitting with your jaw open trying to remember what you were about to do. Or, how severe one's random spastic ls-ing has gotten.

For emergencies, there's also the no-color prompt.

For nostalgia (or out of masochism) there's the old dos prompt.

jeri ellsworth

If this doesn't make your heart go pitter-pat there's no hope for you.

i hate mail, but here's how

Email is one of those crufty old technologies that you're glad you can usually take for granted. The devil is definitely in the details: MTAs, MUAs, spool files, Maildir format, mbox format, SMTP, POP, IMAP, aliases, sendmail's cryptic M4 config files, queues, relaying, fetchmail...who honestly has the time to learn all this crap--just so they can send and receive mail?

Thankfully, in addition to being a crufty technology, it's also a ubiquitous technology. So there's tons of software that spares you the details, and plenty of resources out there if you ever have to learn about them.

I used to be a Thunderbird user, so for a period of time I was happily oblivious to the details. A couple things bothered me though:

• Thunderbird is huge. Typically, it would consume between 30M and 40M, which if you think about it is ridiculous for a piece of software that performs such a simple task. I also wasn't pleased with the time it took during startup / shutdown / context switch. Switching back and forth between my mail user agent and my development environment was a pretty frequent thing. Did I want to wait 5 seconds every time I switched for Thunderbird to process WM_PAINT? I don't think so.
  
  
• Thunderbird is primarily mouse-driven. Speed freaks like me want all functionality exposed through a keyboard interface, and not one that's terribly hard to memorize, or involves lots of CTL+ALT+SHIFT hand acrobatics.
  
  
• I needed to check mail locally too. Sometimes, I would sit down at the mail server itself, or I'd be on a public terminal or a friend's machine. In these cases I would be without my beloved Thunderbird. The best thing I could hope for was PuTTY so I could ssh into the mail server. And on foreign machines, I wouldn't want to leave local traces of all the email I'd read.

So I started to look at mutt. As a vim user, I was psyched about all the vimisms that would carry over--for instance, roguelike movement keys when mutt is in index mode. Less key mappings to learn. mutt is also small and speedy: right now it's only taking up a mere 3.6M.

The problem was that mutt was not going to insulate me from all the cruftiness of mail technology through a slick graphical interface. So, I put off migrating to mutt for a while, at least until I could learn enough of the details to competently configure it.

Now that my configuration has somewhat stabilized I'll run through it, so that possibly someone else with similar needs can benefit. Briefly, here's my setup:

• the mail server is running FreeBSD 4.9 with sendmail
• ssh with pubkey authentication to the mail server is available
• the mail server has imap-uw installed but not running as a network daemon
• my laptop environment is sometimes cygwin with X, sometimes FreeBSD 5.3 with fluxbox

I need to

• send and receive mail locally on the mail server itself, or remotely from my laptop
• keep configuration portable so I don't have to maintain two different similar sets of configuration files
• filter mail into folders as it arrives
• occasionally browse mail offline on my laptop--e.g. when I'm on a trip, or in a meeting without a network connection
• do all this securely

Here's my muttrc file. Since I have more than just one mutt configuration file, I've put them all under ~/.mutt.

As it turns out, my need to browse mail offline under cygwin imposes a significant constraint: I can't use Maildir. Maildir is a mailbox format in which each message is stored in a separate uniquely named file. This has some significant advantages over the mbox format, which stores all messages from a particular folder in one big file. However, the Maildir file naming convention uses characters that are not legal for Win32 filenames.

So I'm stuck with mbox, one for each mail folder. I store my mboxes under ~/mailbox on both the mail server and my laptop. For offline browsing, I periodically rsync my local ~/mailbox directory with the server's.

Also, I create a folder named "spool" on both ends. On the mail server, this is actually a symlink to /var/mail/agrow, my system mail spool. On my laptop it's just a normal file. This way, regardless of whether I'm reading local mail on the server, browsing remote mail via imap, or reading local offline mail on my laptop, I can always open the spool folder to read new unfiltered mail.

To read mail remotely from my laptop, I create an ssh tunnel to the imapd binary using mutt's tunnel command:

I then set the folder variable to imap://localhost/mailbox. When reading mail locally on the server, the folder variable is set to ~/mailbox. In both cases, all subsequent references to mailboxes are prefixed by "+", which tells mutt to make them relative to the folder variable--so they're portable.

To send mail remotely from my laptop, which runs cygwin most of the time, I could have installed a full-blown MTA like exim that would relay to my mail server. Some people install ssmtp instead, which is just a wrapper that knows how to connect to a real remote mail server and forward local smtp traffic to it. I took an even simpler approach and used mutt's sendmail command to make an ssh tunnel to the mail server's sendmail binary:

With ssh-agent running, no user intervention is required. mutt is thoroughly fooled into thinking it's talking directly to sendmail on the mail server.

Portability of all this configuration junk is a snap. Almost all of it carries over from my laptop to the mail server, and for those settings that don't, my muttrc sources a muttrc.hostname file which can vary from host to host.

My mail filtering occurs on the server side. I wrote a Perl script that uses Simon Cozens' Mail::Audit to run regexps against the subject, to, and from fields of an incoming message and redirect the message to one of the mbox files under ~/mailbox. With a few modifications and some tools from procmail, I also used this same filter program to migrate all my existing mail--which was in a hodgepodge of mboxes and Maildirs--to the appropriate ~/mailbox file.

Finally, to make mutt more visually appealing, here's my colors.muttrc file.

Resources:

• the mutt manual
• my first mutt
• mail filtering with Mail::Audit

comment security codes

Well, after 70 comment spams over the last two weeks, I've had to put comment security codes back in place. Sorry for the inconvenience--let this not deter you real people from posting.

tortoisesvn and putty client install

A step-by-step on getting started with the subversion repositories here at thegotonerd.com. You can of course use cygwin ssh and the command line svn client for cygwin (that's what I use), but I assume if you're going that route, there's probably nothing I can tell you that you don't already know.

Download and install the PuTTY suite. Make sure you download the Windows-style installer for the whole suite.

Download and install TortoiseSVN.

The first thing we're going to do is generate a private / public key pair with PuTTYgen. Find PuTTYgen in the Start Menu.

At the bottom you have some parameters relating to what encryption method you're going to use, and how large the key will be. I recommend a 2048 bit SSH2 RSA key. 4096 is for the uber paranoid.

Hit Generate. Wiggle your mouse over the blank area to let the program siphon off some of your wonderful human nondeterminism. Avoid jerky, predictable motions.

Just kidding.

Now you need to choose a passphrase for the key. This is just to password protect the key itself: if anyone ever obtains your private key, they still need to figure out your passphrase to use the thing. If you leave it blank though, you risk giving them free logins to any number of boxes you login to down the road.

Choose something you'll remember, maybe using a mnemonic, and make sure it has some non-alpha characters in it.

Now save both the public and private keys in a safe place, ie not a shared network drive. You'll also need to save off the public key in an OpenSSH compatible format. Copy the contents of the box that says "Public key for pasting into OpenSSH authorized_keys file." Use notepad to save it to a file named authorized_keys (no extension).

Next, we need to initiate a login to the server thegotonerd.com using PuTTY, so we can obtain the server's public key. Run PuTTY from the Start Menu. Type in the hostname. From the tree view on the left, select "Connection," and specify your username as the "Auto-login username." Go back to "Session" by selecting it from the tree view. Probably, you'll want to save these settings for future reference, so type thegotonerd.com into the Saved Sessions box and hit Save.

Now click Open. A dialog box like the following should pop up. Say yes to save the server's public key. At this point, you can just close the PuTTY terminal window without logging in--we got what we came for.

The next task is to tell the server about our public key. It's kind of an I'll-show-you-mine-if-you-show-me-yours situation here. We will actually need to upload the key using PuTTY's psftp utility. Find psftp in the Start Menu. Do an "openthegotonerd.com" and supply your password when it asks you.

Use lcd to change the local directory to where you stored your authorized_keys file. Make a remote directory .ssh and upload the authorized_keys file into it. When you're finished, set permissions on the .ssh directory so that only you can read and write it.

Now, the goal of all of this is to have passwordless logins to thegotonerd.com using pubkey authentication. To do this, you need an ssh agent running in the background with your private key loaded and ready to use. In the PuTTY suite this guy is called Pageant. Find Pageant in the Start Menu and run it. A little computer icon with a hat will appear in your system tray. Double click the icon and you'll see the list of keys you currently have loaded--initially this list should be empty.

Click Add Key and browse to your private key. To unlock it, you'll need to enter the passphrase from before.

If successful you'll see something like this. You can close Pageant now--it will continue to run in your system tray. Here's a tip: to be be prompted to load your private key at startup time, locate your private key in File Explorer and drag a copy of it to the Startup menu. Or better yet make a shortcut to it from there.

Alright, time to test all this. Run PuTTY again and load the session for thegotonerd.com. Click Open. If all goes well, you'll be dropped right to a shell prompt without asking you for a password.

If all does not go well, here's how to track down the problem. Open a command prompt and cd to the folder where you installed PuTTY. Run "plink.exe -load thegotonerd.com -v" and have a look at the output. As you can see here, the server is trying ssh2 authentication but I forgot to load my key with Pageant, so it's falling back on password authentication.

If you're still with me it's time to move on to the TortoiseSVN setup. Compared to what we've just been through, this is a breeze. Create a sandbox somewhere. This is just a folder where you'll check out local copies of subversion repositories and work on them. I recommend C:\sandbox. If you're a developer you'll probably be frequently opening command prompts here, and you want something easy to cd to, and certainly something without spaces in the path name.

TortoiseSVN installs itself as a context menu extension in File Explorer, so you access it by right clicking in a File Explorer window. Do so, and select TortoiseSVN -> Settings. The only thing that concerns us right now is the Network tab, where you'll tell Tortoise to tunnel all its traffic over ssh.

In the ssh client box, browse to plink.exe, which is wherever you installed PuTTY. You need to put quotes around this path, and then add the argument " -l yourusername" after it. Otherwise, plink will be run but won't know what username it should be logging in as. You'll get error messages or worse yet black terminal windows that just hang there.

Now, let's try to check out a copy of a repository. I've created a repository called dungheap on the server that I use for general messing around, and getting familiarized with subversion. First time subversion users can test some basic operations like checkout, checkin, revert, etc. in here.

Right-click and do Checkout. Type in the URL of the repository. For the dungheap, it's

svn+ssh://thegotonerd.com/svn/dungheap/trunk

Notice we're using the svn tunneled over ssh protocol, the host is thegotonerd.com, the repository root is /svn, and we're checking out the trunk, which is the main line of development. (You'll understand this if you're familiar with branching. If not, don't worry about it.)

Now I am telling it to check it out as C:\sandbox\dungheap. If you don't, it will create a folder "trunk" for you, which isn't too descriptive.

Hit Ok. It will ask you if you want to create the top-level folder. Say yes. If all goes well, you'll see some log messages scroll by. Two black plink windows should open up during this process. If they stick there after the checkout, you can just close them.

You're on your way, and I'll hand you off to the excellent subversion book.

sql server: restricting connections to localhost

Here's something wacky I bet you didn't know about SQL Server: if you enable TCP/IP it will bind to every network interface it finds. As far as I know there is no way to only bind it to the loopback device so it just listens for connections from localhost. Which I've come to rely on more and more...tunnel all traffic to the server over ssh, which authenticates the client against the system, and then proceed to access the service as a local one. Don't mess with application-specific authentication or encryption methods. Don't maintain any more account / ACL information than you have to.

Of course, the "workaround" is to just turn on the windows firewall on all external network interfaces and block port 1433. Seems kind of stupid to have a server listening on a blocked port though.

bad ide cable: now i've seen it all

I'll say it again: crap. Here's the story. About a month ago, this server's IDE cable went out. Of course I never suspected an IDE cable--especially since one drive seemed to be working ok--so I plopped in a new hard drive as primary and tried to repair the old one. After there was still a lot of weirdness (like the volume name showing up garbled at boot time) I got wise and switched out the cable.

The old drive reads fine now, but because of my fsck repair attempt with a bad IDE cable in place, half of the /usr slice is now in lost+found. I briefly tried to sift through the thousands of files and directories here to see if I could salvage the important stuff, like the berkeley dbs that movable type uses. Gave up though after repeated grep realloc() errors; the files appear seriously hosed.

So I've lost all my blog entries from about Sept. 2004 to the present. Alas, this blog was most affected.

The good news: this gives me a chance to make a fresh start. Starting with nightly backups. In the process of reinstalling MT I decided to use a PostgreSQL backend instead of BerkeleyDB, having had weird endian problems with the latter after migrating off the CS department's Solaris machines. Some sort of pg_dump / rsync solution probably.

Coming soon to this server: subversion and trac.