Jonas Kölker's blog

Hi all (both of you).

Today, I'm going to muse about learning and understanding, particularly regarding when it comes to computer science and programming.

My main claim is this: if you can't implement it, you haven't really understood it.

It's generally known (or at least I have heard it said many times) that you understand something only if you can express it in your own words. When it comes to computer science, what you'll learn a lot of is algorithms, and one of the kind of words they can be expressed in is real code, not just pseudo-code.

So, am I really suggesting that you should flip through your textbook on automata theory and implement Brzozowski's algorithm for minimizing DFAs, and pick up that algorithms textbook and implement splay trees, and pick up the course notes on NP-completeness and implement the reduction from graph coloring to circuit-sat?

Why, yes I am. Not, you know, today, and not necessarily in the language you know the best. But if you got plans about learning a new language (which you should always have), you should have some project to work on. Check out if the language has a library for manipulating finite automata; if not, implement one and give it back to the community. Or implement a SAT solver, or a library of whacky data structures.

"But in implementing all these things, I'll be dealing with not only essential complexity (which is good because it'll strengthen my understanding of the essentials of the thing I'm implementing) but also a lot of accidental complexity!" Yep, that's true. But I've found that what looks like accidental complexity is often essential complexity.

I remember implementing an I/O-efficient heap (each node is a big sorted array; each last element is greater than all the first elements of the many children). When you restructure the heap you generally want to push elements upwards; but there's a limit to how much you can push stuff up which turned out to be really tricky to discover. Doing the pain-staking debugging taught me something more about I/O heaps I probably wouldn't have known just by reading about them.

You should always be learning a new programming language.

I hope you take a little offense to that; that means you're the kind of programmer I'm writing this for :-)

Why am I saying that? No wait; first of all, why isn't it as shocking as you might think? As you may well know (or have heard stated many times, which of course makes it true), the software business is moving fast. You know, really fast. I've heard that half your knowledge is made useless every five years. So just to keep a good job you will want to learn something new all of the time.

Two days ago, it was all C++, yesterday Java, today C#. Or JavaScript, then PHP, then ActionScript, then Ruby. Or take your favorite pick.

What I advocate is not just learning the languages that your bosses want you to write code in. By all means do so, it'll make you keep earning the same amount of money.

But if you're writing code because you love writing code, not (just) because it pays the bills, I recommend veering off the beaten path.

Dijkstra has said the following:

A most important, but also most elusive, aspect of any tool is its influence on the habits of those who train themselves in its use. If the tool is a programming language this influence is, whether we like it or not, an influence on our thinking habits.

What I'm really saying is that you should learn new languages that will stretch your way of thinking, because that way you'll become a better programmer. The more ways you can attack a given problem, the more likely it is that you'll come up with the best way of tackling it. If you hadn't done any functional programming, would you have an intuitive understanding of Google's MapReduce?

So go pick up Real World Haskell, or Structure and Interpretation of Computer Programs, or Learn Prolog Now!, or something as old-fashioned as The C Programming Language. Or do some non-web programming in JavaScript.

Today's entry is about merging the two linux clipboards. When I say "Linux clipboards", I mean the clipboards you'll see on Linux, but they are really a function of x11 and the ways that X clients interacts with certain buffers.

X11 has three buffers, called cutbuffer, primary and clipboard. The old way of copy-pasting was for the copying client to put data in the cutbuffer and the pasting client to grab that data and use it however the client felt appropriate.

The two new ways is doing the exact same thing with the primary buffer, and the clipboard buffer. Except not really; instead of copying data to the buffer, you just raise your hand and say "I have that data". Then people come to you for it, and you can have a little chat about whether they should get the data as plain text, or text with html markup, or as the URL of the copied html document, or some other way.

But the main thing is that there are two handles to pull instead of one. When you select something with your mouse, you use the primary buffer, just like you do when you middle-click to paste. When you use the C-c and C-v hotkeys (or Edit->Copy and Edit->Paste menu items), you use the clipboard buffer.

I want the two to act as one; that is, I can select something with the mouse in my xterm and paste it with C-v in my firefox. Or copy text from firefox with C-c and paste it in my xterm with a middle click or S-insert.

Here's how you do it: install autocutsel, and run

   $ autocutsel -fork -selection CLIPBOARD
   $ autocutsel -fork -selection PRIMARY

What autocutsel does is simple: it links the cutbuffer with one of the two other buffers, such that when one of the two change, autocutsel propagates that change to the other one. The result of running autocutsel twice is then clearly this: when you update the clipboard, one autocutsel will update the cutbuffer; the other autocutsel will then update the primary. Similarly in the other direction.

Have fun with your new clipboard superpowers until next time ;)

Today's lecture: how to enable CPU frequency scaling on a Thinkpad t43p (which happens to be the laptop I own), with Ubuntu installed on it (which happens to be the OS I run on my laptop).

First of all, you need the kernel module speedstep-centrino to be loaded:

# modprobe speedstep-centrino

You'll probably want the module to be automatically loaded after future reboots, so add it to that particular list:

# echo speedstep-centrino >> /etc/modules

Next, you want to install some piece of software that tells the kernel to tell the CPU to slow down at opportune moments. I've chosen cpufreqd:

# aptitude install cpufreqd

The postinstallation script should start up the frequency scaling daemon. You may want to review the configuration, or trust me when I say that it's a reasonable default:

$ emacs /etc/cpufreqd.conf

Finally, you may want to test that cpufreqd actually does something. For the default policy, you should see the frequency drop a little when you unplug the cable, so try that out. I know of two ways to observe the current frequency; one is via sysfs:

$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq

The other is via conky, using the ${freq} widget. I use conky for other system monitoring purposes, so I went with that.

Now you have frequency scaling, which will help you run longer on a full battery.

Here's my brief guide to setting up distcc on an untrusted host.

For those of you who don't know distcc, it's a program which will ssh into a bunch of other machines (or connect via unencrypted tcp) and run your compile jobs on them, the point being increased compile speeds due to the extra horsepower.

By "untrusted host", I mean that you're not root and you trust root but no other user.

So, here's the guide:

1. install distcc on your machine (apt-get install distcc)
2. copy /usr/bin/distccd to you@untrusted:~/bin/distccd
3. put you@untrusted:~/bin/distccd in ~/.distcc/hosts
4. repeat 2 and 3 for all hosts
5. ln -s $(which distcc) ~/bin/cc
6. repeat 5 for c++, gcc, g++
7. run make -j4, where 4 is twice the number of hosts in ~/.distcc/hosts

I've heard the Ruby on Rails buzz. Being a curious programmer with a severe case of Not Invented Here, I decided to write my own blog in Ruby on Rails. Here's the story of how I failed.

I remember toying (not seriously) with Rails before, in particular watching a fifteen minute video on how to set up a simple blog with rails. So it should be simple: watch and imitate, right?

Not right. As it turns out, the version of Rails I use is 2.0.2 (-1ubuntu1), and the tutorial is made for an earlier version which is incompatible on some points.

Okay, so I can just find a newer tutorial, right? Not much luck there; the ones on the RoR site are horribly out of date. I did manage to find one, which was very brief and pretty much left me with a bunch of scaffolds (in non-RoR lingo, default templates).

With more persistence, I might have found a guide that taught me what I wanted to know. But I decided to wait until the documentation catches up with the code; that may be forever, if the code is updated on a rapid schedule. But I still wanted to have some kind of blogging software for my self-expression needs.

So, I use Chronicle. It's pretty cool: it generates static HTML and RSS from markdown sources with a quite reasonable defualt template.

Conclusion: RoR is not for me, but that may change in the future.

Welcome to my new blog. I'm Jonas Kölker, and I'm going to be your host whenever I feel like blogging. What I intend to write about will be mainly things related to programming and system administration, including network administration.

You won't believe how much lack of fun you can have with networks until you try it. But the end result is cool.

Here's a brief tutorial on setting up a bond between a wired and a wireless interface.

What's bonding? Simply put, it's creating a virtual interface which uses one or more other interfaces to transmit its traffic, for the purposes of reliability and/or increased throughput. Why is that useful? You can get the full speed of your wired interface when the cable is plugged in, and have your connections (e.g. file transfers and ssh sessions) not die when you unplug it.

So here's the short version of how, if you do not encrypt your wireless network:

# pkill dhclient3 # you don't want dhcp for eth0 and eth1
# modprobe bonding mode=active-backup miimon=100 primary=eth0
# ifconfig bond0 up
# ifenslave bond0 eth0 eth1
# dhclient3 bond0

Note that mode=active-backup means that only one of the enslaved interfaces will be used at any given time. If the primary interface is available, that will be used; otherwise one of the others (in my case there's only the wireless eth1 left).

If you use WPA, you need eth1 to be the active interface when associating; this happens when you first connect, and also during use. If you use wpa_supplicant (which network manager uses behind the scenes as far as I'm aware), it needs to be launched with -b bond0. How to make network manager give arguments to wpa_supplicant is left as an exercise for its users.

But how to make eth1 the active interface? I've written a shell script which does, as far as I can tell, the right thing:

#!/bin/sh
cd /
bond=/sys/class/net/bond0/bonding/
while sleep 1s; do
    wpa_cli status | grep -q 'COMPLETED'
    primary=eth$?
    echo $primary | diff - $bond/active_slave > /dev/null || \
        echo $primary > $bond/primary
done

Here's my complete /etc/network/interfaces stanza for bond0:

auto bond0
iface bond0 inet dhcp
    pre-up ifconfig bond0 up
    pre-up ifenslave bond0 eth0 eth1
    pre-up wpa_supplicant -i eth1 -b bond0 -c /root/wpa.conf &
    pre-up /root/the-above-script &
    down wpa_cli terminate
    down pkill the-above-scr
    post-down ifconfig bond0 up
    post-down ifenslave -d bond0 eth0 eth1

I've tried setting up a bridge on top of a bond, and virtual NICs (with veth) on top of that bridge. That tends to make the bonding fail and (I think) screw up some kernel networking internals, so only do this if don't mind spending time on figuring out what goes wrong and what works.