Hackszine.com: Linux Server Archives

Archive: Linux Server

May 14, 2008

Debian/Ubuntu users: update your SSL keys and certs

It was announced yesterday that sometime back in September 2006 a line of code was removed from the Debian distributed OpenSSL package. That one line of code was responsible for causing an uninitialized data warning in Valgrind. It also seeded the random number generator used by OpenSSL. Without it, the error went away, but the keyspace used by affected systems went from 2^1024 to about 2^15. Oh noes!

A large majority of Debian and Ubuntu systems are affected. To correct the problem, you'll need to not only update OpenSSL, but also revoke and replace any cryptographic keys and certificates that were generated on the affected systems. From the Debian security advisory:

Affected keys include SSH keys, OpenVPN keys, DNSSEC keys, and key material for use in X.509 certificates and session keys used in SSL/TLS connections. Keys generated with GnuPG or GNUTLS are not affected, though.

For most people, this boils down to your ssh server's host key and any public key pairs used for remote ssh authentication. Any keys or certificates generated on the affected machines for SSL/https use also need to be revoked and regenerated. It's pretty ugly, really.

As far as teachable moments go, there's probably a lot to think about here. Software developers have this weird natural tendency to want to fix and reengineer things that aren't even broken. I'd go so far as to say that the desire to reengineer is inversely proportional to a programmer's familiarity and understanding of the code. I think it comes from our intense desire to make sense of things. It's the guru who's able to channel that hacker urge into solving new problems instead of creating new bugs out of old solutions.

DSA-1571-1 openssl -- predictable random number generator
OpenSSL PRNG Debian Toys (more discussion of the problem here)

October 18, 2007

Remote snapshot backups with rsync and Samba

Thanassis Tsiodras writes:

What would you do if you had to automatically backup a remote Linux box (e.g. your web server), and all you had locally was Windows machines? How about this:
automatically expanding local storage space
transmissions of differences only
automatic scheduling
local storage of differences only
secure and compressed transfer of remote data and
instant filesystem navigation inside daily snapshot images

I covered all these requirements using open source tools, and I now locally backup our 3GB remote server in less than 2min!

We've all used Samba and rsync before, but Thanassis has really put all the pieces together into a complete backup system that's superior to a lot of commercial products I've seen.

The really impressive bit is how he's easily doing snapshot images using filesystem hardlinks. You can save several days worth of snapshots at very little cost because additional space is only taken up by files that have changed. Using hardlinks, identical files from different snapshots all point to the same inode.

root# mount /dev/loop0 /mnt/backup root# cd /mnt/backup root# rm -rf OneBeforeLast root# cp -al LastBackup OneBeforeLast root# cd LastBackup root# rsync -avz --delete root@hosting.machine.in.US:/ ./
The "cp -al" creates a zero-cost copy of the data (using hardlinks, the only price paid is the one of the directory entries, and ReiserFS is well known for its ability to store these extremely efficiently). Then, rsync is executed with the --delete option: meaning that it must remove from our local mirror all the files that were removed on the server - and thus creating an accurate image of the current state.

And here's the icing on the cake: The data inside these files are not lost! They are still accessible from the OneBeforeLast/ directory, since hard links (the old directory entries) are pointing to them!

In plain terms, simple navigation inside OneBeforeLast can be used to examine the exact contents of the server as they were BEFORE the last mirroring.

Just imagine the data recovery headaches you could solve by adapting that to a cron job that shuffles a months worth of nightly backups.

Optimal remote Linux backups with rsync over Samba - Link