maelstrom

more on cvsup and the ports tree

A couple other things to know about cvsup and the freebsd ports collection. First, always cvsup the entire ports tree, except for ports collections that nothing you use will depend on. (Like the russian collection if you don't speak Russian, or x11* if this is a server which won't run X.) If you don't cvsup the entire thing dependencies between ports will get way out of whack.

You specify which collections not to update via a refuse file. This typically is /usr/local/etc/cvsup/sup/refuse. My refuse file for our web and development server currently looks like

So cvsup won't waste it's time--and yours--trying to keep these unneeded things up to date.

The first thing you do after you get a freebsd version up and running is to update the ports tree via cvsup. Since the ports evolve continuously, it is likely that the current ports tree has departed significantly from your out-of-the box /usr/ports. So this first update will take quite a while--three or four hours perhaps.

After that, successive cvsup's will run much faster. Put cvsup in your crontab so that your ports are updated every night, or every week, automatically. Make sure -L 2 is passed to cvsup and capture the output. Grok this output periodically to see if anything radical has changed.

-r: not found ports error

While trying to install the mail/mutt port on a freebsd 4.9 release system, I got this message:

Turns out problem was that ports/Mk was out of date. You cvsup this by adding ports-base to your cvsup targets in your supfile. Evidently, you should always specify ports-base.

a sql-like mini-language for an oo database

This is my development journal proper so I might as well use it as such. By that I mean start talking to myself.

So, I am finally using Tangram. I'm developing an issue tracking system (yes, the world needs another) and decided to make it my Tangram pilot project.

FYI, the idea behind an issue tracking system is to let developers, tech support staff, and end users interact in a much more structured and visible way. Issues can be potential bugs, support requests, feature requests, general usage questions, etc. The system tracks each issue over its lifetime, and publishes all the interaction so the same issue doesn't have to be dealt with again & again.

As with any serious program I undertake, I began with a simple command line interface, modularizing non-interface code as I go along. The problem is that it quickly became not so simple. I wanted to handle objects gracefully, and the original flat-file input format I had in mind proved to not work well with hierarchies. Also I wanted to specify really granular permissions on objects and their attributes. Mostly, this was so I could set up different roles (system admin, tech support, end user, etc.).

So I designed a mini-language. It is akin to SQL, except it is much better at dealing with hierarchical data, and not as baroque. The grammar specification is compact, with only five basic commands.

• add: insert a new object into the database
• get: query objects from the database into a working set
• list: display objects in the working set
• update: commit working set objects to the database
• delete: delete working set objects in the database

I guess now that I think about it, the idea of a working set is akin to ADO's RecordSet idea. Except that it is a working set of objects, not just name / value pairs.

Anyway the nice thing about this working set idea is that it keeps you from having to specify which objects list, update, and delete should act on. You first specify which objects via a get command. Then you perform some actions on them.

SQL tries to kill two birds with one stone--for once-obvious performance reasons--by saying things like "update foo where bar = 'baz'". However with client-side cursors and Tangram's ability to transparently pull fields onto the client side on demand, you don't really need this ability, it just makes life more complex.

So now it has gotten down to parsing and interpreting this mini-language. For parsing I'm using Parser::RecDescent, which takes a BNF-style grammar specification and fits input text to its syntax structure, or complains. Each time a production (so the lingo goes--this just means a syntactical element like a boolean expression) is matched, you get the chance to perform an action using the matching tokens.

Historically, most parsers are bottom-up. Parser::RecDescent is top-down. Despite the term "top-down," sub-rules still get matched before their parent rules. E.g. the rule for matching an integer would get matched before the rule for constructing an algebraic expression, because algebraic expressions are built from numbers (and other things of course).

A consequence of this is that any actions you specify for dealing with numbers get taken before actions that deal with expressions. This is an issue for me, because of the way I usually implement interpreters.

Typically I have two stacks sitting around in my interpreter. One is for symbols, another is for operators. As expressions are parsed successfully, operators get pushed onto the operator stack, and operands get pushed onto the symbol stack. When it comes time to interpret things you pop off an operator, pop off one or two symbols and apply the operator to them, push the result back on the symbol stack. Then lather, rinse, repeat, until the operator stack runs out.

However with post-order traversal of the parse tree (this is what Parse::RecDescent does by invoking sub-rule actions first) I get into problems with function calls. A function call needs to set up brand new operator and symbol stacks--otherwise its lather, rinse, repeat steps will clean out the callee's stacks. We really don't want this to happen. So I bundle up the two stacks together and call it a frame. There is then an overall frame stack. The callee's frame gets pushed onto the frame stack before every function call and life is good.

Life isn't so good though with post-order traversal. The rules for matching expressions are sub-rules of the rules that match function calls, so these expressions don't have a new frame to push onto. With the ability to pre-order traverse the function call node, we can do this.

I got excited when I saw the autotree directive of Parse::RecDescent. This returns a parse tree which I could then feed into my interpreter, doing both pre- and post-order actions. However the autotree output of Parse::RecDescent is nigh gibberish. So I will have to implement my own parse tree building code, a minor bummer.

passwordless ssh on win2k server?

If anyone has gotten cygwin sshd passwordless logins (via pubkey) working on windows 2000 advanced server, lemme know. I've tried a lot of crap including granting the sshd user all the silly winnt access privileges I thought he needed. But I always get a password prompt.

xml resume, gcj

Recently I decided to go with XML Resume for my resume. That way, I keep one all-inclusive and up-to-date version of it in xml which is fed into a pipleline that spits out html, txt, rtf, ps, dvi, pdf, etc. Also it has filtering so I can tweak it to target specific employers or positions.

Problem is, much of XML Resume is implemented in Java, which I really don't care for. Well I like the language--it's C++ without all the powerful but crazy gibberish of templates, operator overloading, and multiple inheritance--but hate installing a gigantic runtime and sdk, and then using programs that run slow as snot.

Still in order to produce that all-important pdf I need Apache FOP. Which is written in Java. So currently I'm looking at GCJ, the Java frontend to the gcc compiler. What's kind of cool is that you can either (a) compile to bytecode, the traditional way, or (b) compile it to native code, linking to runtime libraries, like any other compiled language. Here's an article on GCJ that explains some of this.

Well, we'll see how this works out.

activesync remote display

Stop paying for silly pocket pc screen capture utilities and just use ActiveSync Remote Display. It's free and part of MS's Windows Mobile Power Toys package.

gnu uninstall?

Okay here is my cool program idea for the day. Modify the gnu install file utility to optionally report everything it is changing--creating missing parent directories, changing file attributes, copying a file--so that you can write a corresponding uninstall that takes this output and exactly undoes the action of install. Currently install doesn't support this, and as a result make uninstall is a do-it-by-hand thing.

user has not been granted requested logon type

God, now I know why Raymond beats on the virtue of orthogonality so much in the Art of Unix Programming.

I had to set up samba again between my FreeBSD box and my new XP laptop. Added a user and set up a network share on my laptop that they had access to. I kept getting this message when I tried to access the share from another windows box: "logon failure: the user has not been granted the requested logon type."

Turns out you have to (1) fool with several settings under local security policy and (2) add the user a second time under the security tab of the folder properties dialog box.

(1) Run gpedit.msc. Expand Local Computer Policy -> Computer Configuration -> Windows Settings -> Security Settings -> Local Policies -> Security Options. Here you will find a bunch of non-orthogonal settings. For the policy "Access this computer from the network" add your user. For the policy "Log on Locally" add your user.

(2) Right click the folder you're gonna share, go to properties, sharing tab, make sure you've selected "share this folder." Under the security tab, add the user and check read & execute, list folder contents, and read. Again, file permissions under windows is a collection of non-orthogonal settings (how is Modify different from Write, and what does Full Control add to the picture?).

mangband 0.7.2a on cygwin

Mangband 0.7.2a builds & runs on cygwin just fine, with a few minor modifications. I've only tested it on cygwin 1.5.9. You will need to install libncurses-devel, libncurses7, and all of the X packages via cygwin setup. Then apply the following patch.

cool cygwin xterms

Pieces of my bashrc file. By default do colored directory listings, and use a prompt that consists of username@host, cwd, and cursor on the next line.

Create 4 tiled xterms that look nice with cygwin's XWin server on a 1024x768 screen. Each one is a different readable color on a black background, so you can tell them apart quickly.

Here's what it looks like (kind of):

the art of unix programming

Picked up Raymond's The Art of Unix Programming in B&N the other day, and couldn't put it down. Preachy at times but exciting nonetheless, articulates what I love so much about Unix. Since I've never put it into words...mostly it's just a feeling. Weird.

docbook and xmlto

Cool tools: DocBook is a system for authoring docs in xml, and xmlto is a tool that converts them to every format known to man. Well...html, dvi, htmlhelp, man, pdf, ps, txt, etc. In particular all the FreeBSD documentation is written in DocBook. For those of us who are tired of plowing precious effort into maintaining 10 copies of the same thing by hand (c.f. my resume), groan audibly when asked "can you send it to me in format x?", or find the latex toolchain to be a big hairy mess when writing anything but math papers.

sql diff tools

Sweet. Recent thoughts about a schema transformation framework for Tangram have led me to MySQL::Diff for MySQL and pgdiff for postgreSQL. With something like this you could transparently upgrade a live database to reflect a new class structure. Ie, once again forget about the fact you even have a database behind it all & just code away happily. Now, I wonder how hard it would be to do a more general SQL schema diff that would work sql server 2k, oracle, and maybe firebird as well.