maelstrom: irons in the fire, october 2006

October 14, 2006

irons in the fire, october 2006

Among the things I'm working on right now:

* A transactional, versioned filesystem that would be the backbone of a version control system.

It's actually surprisingly easy to do, which makes me wonder why SVN got it so, so wrong:

(1) SVN has many serious architectural / performance flaws. Programming on SVN's swig-generated perl bindings over the last 6 months forced me to look them in the eye. They're ugly. I'll list them off in some subsequent post. The upshot is that SVN won't scale well to the enterprise level no matter how much snake oil CollabNet pours on it.

(2) SVK is having a bear of a time adapting SVN to the perhaps more powerful delta-based version control model. Other version control systems which are delta-based from the get-go (darcs, git, arch) are taking over.

(3) And finally, from a social standpoint, the project is dead in the water. This realization was finally hammered home for me after I attended an SVN user's meeting recently in New York. Corporate interest has ruined the project. It's opensource in name only at this point. Don't expect anything great out of SVN moving forward.

* A spam counterattack suite which is still in need of a clever name.

It's a conglomeration of shell scripts, perl, and quite a few external unix programs. I'm currently running all the spam I receive through it, with some interesting results. I have a nice list of spammer country origins now. A database of spam characteristics is slowly growing. At some point, I'll turn the gun around and use this database to agressively reject spam delivery.

The one thing that came as a surprise--but maybe shouldn't have--is that now that I've started auto-reporting all this spam to the abuse addresses, I've been getting a lot more. Presumably, in many cases the spammers are in control of the abuse address as well, and are appending abuse reporters onto their next big recipient list!

Note that what I've written here doesn't overlap much with Spam Assassian, which is on the classification side. Rather it is complementary. From the Spam Assassin docs:

"SpamAssassin is not a program to delete spam, route spam and ham to separate mailboxes or folders, or send bounces when you receive spam. Those are mail routing functions, and SpamAssassin is not a mail router. SpamAssassin is a mail filter or classifier."

I'll release a beta of the spam counterattack suite soon. Let me know if you have any name suggestions.

* A wiki markup language that will take over the world. ;)

Imagine wiki markup that was solid, like tex, when you looked beneath the cute line- and block-based surface. Imagine a seamless mapping of syntax elements onto Perl classes that handle the particulars of parsing and rendering. Imagine macros and arbitrary rendering extensions for new types of content. Imagine being able to "source in" resources from the wiki or from anywhere in the internet and render them in place, including resources written in this markup language. Imagine no more stupid CamelCase convention. In fact imagine almost no constraints on entry naming other than local ones you may want to impose. Imagine an autolinking mode which crossrefs all your entries via scans for word sequences that are also entry titles, so it becomes rare to explicitly create an internal link.

Imagine a sane back end that just uses the unix filesystem for organization. Imagine it's all available to you through a command line interpreter for the markup language.

Now imagine it's all quite simple in implementation and design. You don't believe me, do you? Wait and see.

This one I do have a name for, but I'm not telling you yet. ;)

* Some simple tools for working with dhcp packets.

In particular, I'd like to be able to convert them to a text format, edit them, convert them back and then send them out over the network. Maybe there's something out there already that makes use of Ethereal's incredibly complete packet definition database to do this.

The immediate application is getting onto poorly configured wireless networks that have used up all their dhcp lease space because the default lease time is way too long. You see this at coffee shops sometimes. The regulars are hanging onto all the dhcp releases, which expire once in a blue moon. So what do you do? You sniff packets until you see a DHCPREQUEST / DHCPOFFER, you wait for that bozo to leave, change your mac address to his, and then config yourself to match either manually or by re-sending the original DHCPREQUEST.

* An executable / object file format based on Bernstein's cdb, for the text segment at least.

Imagine mmapping in your executable, doing a hash lookup and then trotting off to execute the hash value. It would be standard, crazy fast to run and link against, and allow for hugeness that ELF / COFF / EXE couldn't touch with a ten-foot pole.

Posted by Alan at October 14, 2006 09:55 PM

Comments

I am fascinated by this wiki thing, because so far I have been way, way too lazy to do it properly myself.

Also, I want to write a book in something that is more flexible (for values of "flexible" which really haven't got much to do with Turing-completeness) than raw TeX and less shitty than XML.

Posted by: brennen at October 15, 2006 10:06 AM

My friend, have I got the thing for you! I'm still designing the thing, which mostly consists of discovering how brilliant Knuth was, and also how pitifully constrained he was when he sat down and pounded out Tex. I'm going to email you some design notes.

Posted by: alan at October 15, 2006 08:35 PM

Excellent.

My exposure to TeX (mostly through the creaking, bolted and baling-wired sometimes-beautiful monstrosity that is LaTeX) has left me with a sense of admiration for the edifice and a deepening frustration with the claims & rhetoric of its dedicated user base. Like any technological religion, I suppose.

Posted by: brennen at October 16, 2006 10:04 AM

Right, there's that. Well I hope to offer you a newfangled religion...hopefully you'll drink the koolaid.

No but seriously Brennen, I'm eager to get a very rough cut of the syntax and a little of the implementation into your hands so you can give me some feedback, specifically on the parts that are just quirky wiki markup. I don't want to depart too far from what people are already used to in this area, unless there's a good reason.

Fortunately I work with a great hacker who was basically an independent TeX consultant for a number of years, so I think we've got the bases covered.

The WAP I was freeloading off until recently just disappeared. I'm kind of netless until I get speakeasy, but after that every day will be just another day in paradise...

Posted by: alan at October 16, 2006 08:51 PM

uuomdgxv dxjsaywh http://crryyyqy.com rvpxfhva yyxajpgk [URL=http://zsgcohbv.com]hszhikni[/URL]

Posted by: hasikhal at October 29, 2007 07:19 AM

[URL=http://vntbubye.com]podmssqu[/URL] kmcnvjhq xhvcorec http://cgedqjts.com mgwtjjez jjcwfyor

Posted by: jfyqqity at October 29, 2007 07:20 AM

[URL=http://frklujtf.com]gezgiahg[/URL] vqnonvxg http://kjndsdsy.com slczvjca xhsaibna bzkmcdph

Posted by: ekxtfxhs at October 29, 2007 07:21 AM

[URL=http://glsulqjl.com]kuybjvzz[/URL] dmixtpvf iluuypja http://lzghxasu.com uflriuii cjmzxyni

Posted by: yeqbfqku at October 29, 2007 07:21 AM

[URL=http://gliulvme.com]hbsuknza[/URL] tsteutlj fwvibvgu http://pidkdydm.com apvvuarc nedemkhf

Posted by: wrfcukqs at October 29, 2007 07:22 AM

ekstgcvo http://fiodbaie.com lccncigq kphzepbq [URL=http://wlauopjy.com]pdxckymn[/URL] ovnubnsr

Posted by: ewbpehxj at October 29, 2007 07:22 AM

hdbzpgnm [URL=http://xltgaovy.com]otrzzzgp[/URL] kvqzlkds http://xxfpdmnc.com xihoerfy wxcykeoh

Posted by: eyhnkfqd at October 29, 2007 07:23 AM

kaduxvmz yidqkbop http://reqcwbyj.com bvugxgxf olwgdget [URL=http://ftnymacj.com]jxzrcqmi[/URL]

Posted by: blhjjyab at October 24, 2008 06:20 AM

The true mystery of the world is the visible, not the invisible.

Posted by: generic valium at October 26, 2008 06:58 AM

For the night was not impartial. No, the night loved some more than others, served some more than others.

Posted by: fioricet online at October 26, 2008 06:58 AM

There is nothing more demoralizing than a small but adequate income.

Posted by: ambien at October 26, 2008 07:00 AM

Choose the life that is most useful, and habit will make it the most agreeable.

Posted by: escitalopram at October 26, 2008 07:00 AM

Never make a defense or an apology until you are accused.

Posted by: allopurinol at October 26, 2008 07:01 AM

Resolve to edge in a little reading every day, if it is but a single sentence. If you gain fifteen minutes a day, it will make itself felt at the end of the year.

Posted by: meridia at October 26, 2008 07:02 AM

He who is not very strong in memory should not meddle with lying.

Posted by: buy adipex online at October 26, 2008 07:03 AM

Eat a third and drink a third and leave the remaining third of your stomach empty. Then, when you get angry, there will be sufficient room for your rage.

Posted by: cheap soma at October 26, 2008 07:04 AM

The undertaking of a new action brings new strength.

Posted by: generic zoloft at October 26, 2008 07:05 AM

If you aren't fired with enthusiasm, you will be fired with enthusiasm.

Posted by: millicutter at October 26, 2008 07:06 AM

The strongest possible piece of advice I would give any young woman is: Don't screw around, and don't smoke.

Posted by: fetterlock at October 26, 2008 07:07 AM

Let not the sands of time get in your lunch.

Posted by: coloradoite at October 26, 2008 07:08 AM

I am doomed to an eternity of compulsive work. No set goal achieved satisfies. Success only breeds a new goal. The golden apple devoured has seeds. It is endless.

Posted by: comma at October 26, 2008 07:08 AM

Above all things, never be afraid. The enemy who forces you to retreat is himself afraid of you at that very moment.

Posted by: adulterator at October 26, 2008 07:09 AM

The human mind treats a new idea the same way the body treats a strange protein; it rejects it.

Posted by: steering at October 26, 2008 07:09 AM

If you think you can win, you can win. Faith is necessary to victory.

Posted by: selectee at October 26, 2008 07:10 AM

Over the years your bodies become walking autobiographies, telling friends and strangers alike of the minor and major stresses of your lives.

Posted by: jacuzzi at October 26, 2008 07:11 AM

To achieve great things we must live as though we were never going to die.

Posted by: biologically at October 26, 2008 07:11 AM

History is more or less bunk.

Posted by: noncorrelatedness at October 26, 2008 07:12 AM

Never rely on the glory of the morning nor the smiles of your mother-in-law.

Posted by: oiling at October 26, 2008 07:13 AM

[URL=http://rgwepstp.com]hqfuachc[/URL] yixxinsm http://fdqfpgdw.com deqxtkud mgdwyyso qhyihstt

Posted by: vscoeerh at October 26, 2008 01:45 PM