jd:/dev/blog - Naquadah Network

Boarding the Prometheus

jd — Sun, 18 May 2008 10:26:00 +0200

As I said a month ago, my main server Delmak was dying. Well it still runs (proof: you could read this blog some days ago).

Thanks to friends I host for free, they've kindly given enough money to buy a brand new server (C2D E8400, 4 GB RAM, 2x500 GB RAID 1) in order to replace the good old Delmak. This new box has been named Prometheus after the only BC-303 class battleship ever built.

Delmak was used to mainly run as a Web, mail and databases server. I decided to do use this server switch to change the server software I use.

The first mail server I setup was based on Exim 3, courier-{imap,pop}{-ssl,} with userdb files. That was... rough. Later I switched to Exim 4, using vexim, and MySQL as a back-end. That was something like 3 years ago I guess. Since then I never really touched that back. I added spamassassin and clamav filtering some months after, because some users asked for it. That's all.

So this week, I decided to switch away from this configuration. I do not understand Exim anymore anyway, so I decided to use Postfix which I often use and administrate at work. Obviously, I also now use PostgreSQL as database back-end, since it rocks, and since Postfixadmin supports it. By the way, be aware that the Debian package of postfixadmin is crappy (the configuration file is readable by anyone by default, with the database password in it). I also set up postgrey which is quite nice and efficient.

Well, then was time for amavisd-new installation, but I did not do it. Seriously, amavisd-new configuration is a bloody mess, as the language it is written in (yes, Perl).

So I switched to dspam which I heard is nice. Well, it seems to be for now, since it even supports clamav daemon usage directly, which is very very nice because that means I do not have to set up another thing for that.

I also switched from courier to dovecot, mainly because the latter seems to be faster and lighter. I then changed the default virtual_transport to Dovecot LDA. The main advantage of this is that the LDA updates the Dovecot index while delivering. It also supports quota, which I do not use and plug-ins, like the Sieve language for mail filtering.

So I decided to change my procmailrc to a new Sieve filter. My procmailrc is quite small since I only use regex to match lists and some mail address, so it has only something like 12 rules. And well, I did not do it since I discovered after some googling that Dovecot implementation of Sieve is grabbed from Cyrus which does not support variables for now. That means that the following procmailrc code:

:0:
* ^X-Mailing-List: <debian-.+@lists.debian.org>
* ^X-Mailing-List: <debian-\/[^@]+
list-debian-$MATCH/

which will translate to:

require [ "regex", "variables", "fileinto" ]
if header :regex "X-Mailing-List" "<debian-(.+)@"
{
    fileinto "lists.debian.${1}";
    stop;
}

But that won't work since Dovecot Sieve implementation does not support "variables". Well, since I'm not ready to list all the lists I'm subscribed to, Sieve is a no-go for now. I'll stick with procmail.

We've been almost down

jd — Tue, 04 Mar 2008 18:02:00 +0100

I had a big fright this afternoon. My friend ludo asked me to shut down my main server (delmak.naquadah.org) for racking one of my new equipement.

Delmak is my oldest server, hosting almost everything since 5 years. I knew I will have to shut it down some days, but after 280 days of uptime I was pretty confident everything will be allright.

But obviously, it refused to boot again. Fortunately, only the power was burned, and after changing it with a new one, we manage to get delmak booting again.

Now I'm really thinking about some better solution to not having everything on a server that can explose any day now.

abydos's dead

jd — Sun, 22 Jul 2007 13:55:00 +0200

My workstation is dead yesterday at 22:01. It did not suffer, the screens simply turned off, and now it does not boot anymore. It seems that the processor is dead.

Four years of good services. Now have to buy a new workstation, sic.

Main consequence for now is that I cannot read my mails anymore for now. Anyway, I'm on holidays for a week.

Creating a FLAN

jd — Tue, 16 Jan 2007 12:48:13 +0000

I have several servers around the Internet, and one of my need is to have VPN between them to access various data and services, like SNMP or NFS. I used to build VPN with OpenVPN between some of them.

My main problem was that I had to set up IP interfaces at each end, build multiple tunnels or make some routing, and... that was not what I wanted, too complicated.

I took a deeper look in OpenVPN and found a solution: I created a FLAN… No, not a cake, a Faked LAN! This is so easy and powerful that I beat myself to not have though about that before.

I did it with 3 servers for now, this is how it works: on each server I create an interface named if-remote hostname which is a tap (Ethernet) interfaces connected to the remote host. On server1, I have two interfaces, if-server2 and if-server3 which are like direct wired connection to the remote host, and I can do Ethernet on them.

When each server has its link up, I create an Ethernet bridge. In this bridge, I put the two interfaces connected to the remote servers. This finally build something like that for each server:

server1 -- eth0
  \ 
 if-bridge 192.168.4.88
    \  \_ if-server2 -- VPN -- if-server1 -- if-bridge 192.168.4.89 -- server2
     \_ if-server3 -- VPN -- if-server-1 -- if-bridge 192.168.4.90 -- server3

Obviously, you will have a loop in your faked Ethernet LAN, so you'll have to active STP in order to have a working LAN. And it works.

The main advantage with this solution is that each server has only one IP on this virtual network, and there's no connection problem if one of your host is down: STP will rebuild the network in a transparent way. The main problem is that you may have to set up a lot of link if you want a full redundancy. I think I'll setup only two links between each server, because it maybe too painfull otherwise.

You can also do some optimization if you need, because the current bridge implementation in Linux is so powerful that you can put cost on interfaces for your bridge, or define which bridge should be the root of your tree. Have fun!

The man who did not know he had an amd64

jd — Tue, 07 Nov 2006 21:13:50 +0000

On sunday, I was looking around at the /proc/cpuinfo on one of my last server. I saw that this Pentium 4 had a lot more of cpu flags that the one on my workstation. I discovered the nx flags and its purpose some days before, but I did not know what the lm flags was for...

Oh my god, that's the 64 bits support. This box is an amd64 and it was installed as an i386. That's like using a knife to kill a kitten when you have an axe!

So, even if the box was 800 km away from me, I decided to reinstall it from scratch, with the help of a serial cable connected on it.

That was so easy. I just love Debian for such things.

Step one: recompile linux-2.6-2.6.18 with support for 64 bits processors, that was easy, Goswin Brederlow made a patch I used and adapted in #379090.
Step two: scratch your swap and debootstrap an amd64 sarge in it. Copy blindly your /lib/modules/2.6.18-1-amd64 inside.
Step three: reboot with your 64 bits kernel on your brand new 64 bits Debian system and launch sshd.
Step four: ask everyone on IRC WHY THE HELL you get sshd killed with a fucking kernel backtrace on your serial console each time you try to ssh to your box
Step five: listen everyone advices and dist-upgrade to etch
Step six: backup all the old data and scratch all your partitions, because you want LVM now. And migrating to 64 bits and LVM at the same time is more dangerous, so more exciting, so more fun.
Step seven: move your brand new amd64 etch into your old root partition. Believe in you and that you did not forget anything to backup.
Step eight: create your logical volumes and move your stuff in here, like /var, /usr and then blindly reboot. Thanks god you have a serial console.
Step nine: apt-get install everything back and upgrade your old sarge conffiles to etch.
Step ten: wonder why slapd is segfaulting again and again, and then yell after this fucking Berkeley DB files that are not architecture independant. Flame yourself because you don't have a LDIF backup of your LDAP tree.
Step eleven: install a i386 sarge with LDAP to slapcat your old LDAP tree and restore it.
Step twelve: Take a break. Have a Kit^Wbeer.

In the end, I'm happy, even if everyone is wondering why I killed a server during 10 hours just because it's better.

Dust in the box

jd — Fri, 15 Sep 2006 11:26:19 +0000

I really should clean my workstation case more often.

abydos kernel: CPU0: Temperature above threshold

I agree that 50 C idle / 68 C full load is too hot.

I cleaned it (arrgh so dusty), change the CPU fan. Back to 30 C idle!
But I did not plug back the case fan.

abydos kernel: end_request: I/O error, dev hdc, sector 43940535
[...]

Err, 58 C for an hard disk is too hot God bless RAID 1.

* jd is going to buy some hard drives

Xen 3

jd — Sun, 26 Feb 2006 00:32:15 +0000

Finally, I managed to switch to Xen 3 on a box!

My hardware problems with Ethernet devices was solved by adding acpi=off to the Xen hypervizor boot parameters, and turning off tx checksumming (ethtool -K eth0 tx off).

Upgrading to Xen 3... aborted :(

jd — Thu, 09 Feb 2006 22:00:44 +0000

It seems that after two hours stuck in front of my minicom, trying to upgrade my server/gateway from Xen 2 to Xen 3, Sarge version of udev and hotplug are too old. I will have to try with a backport tomorrow... What a pity...

And now...

jd — Mon, 26 Dec 2005 11:46:40 +0000

My laptop hard drive just died. I just hate hardware.

Merry xmas.

Kheb is not anymore

jd — Fri, 23 Dec 2005 15:23:27 +0000

Kheb, my old Pentium machine hosted in my parents' house, just died.
I think the CPU is dead. I will replace it with another Pentium II box I have in my flat. Because they are going to get a DSL access in the next days (yeah, we finally got DSL connection in this 700 inhabitans village).

I just updated my history page, just for fun!

My new server

jd — Fri, 12 Aug 2005 12:14:02 +0000

After 2 years of services, my main server hosted by Lost Oasis, called Netu, is now... Delmak!
That's the same machine, but it is now running Xen (see my last entry about how Xen is fantastic and can improve your sexual performance).

Delmak is the dom0 and Netu is a domU. Netu is now managing FTP and shell access to my users, and services like mail, http, etc, will be migrated to Delmak.

Why Delmak? Delmak is a planet and Netu is its moon. Can you feel the Xen spirit?

Keyboard

jd — Tue, 12 Apr 2005 14:41:25 +0000

Again, my keyboard is dead. Why can't I keep a keyboard more than one year ?

One year ago, my Keytronic was killed by a friend with a whisky-coke...

New laptop

jd — Sun, 06 Mar 2005 17:31:08 +0000

Yesterday I received my new laptop, it's a Dell Inspiron 510m.

Hardware:

Intel Centrino M 1.3 GHz
Intel 855GM graphic card
256 MB RAM
15" display XGA (1024x768)
Intel PRO/100 Ethernet controller
Intel PRO/Wireless LAN 2100 controller
CD-RW/DVD
40 GB hard drive
Windows XP Home Edition (ahahaha !)

I installed a Debian Sid on it (of course), and everythings works fine.

Its name is shifu.

NB: A good point to Dell: they have not removed the serial port ! I see too many laptop today with no more serial port, and that really suck.

About my shell config, part 2

jd — Sat, 05 Mar 2005 17:10:47 +0000

In response to Tollef, my shell config is already managed by Subversion. However, I cannot handle to install svn client on each machine I connect to. Some are production servers and have minimal required packages installed only.

About my shell config

jd — Thu, 03 Mar 2005 17:53:16 +0000

Since several weeks I wonder something about my zsh configuration and I cannot find a good solution. Maybe you, my faithful reader, you have an idea !

Let me explain.

I have a function called scpzshconf which copy my zsh configuration files on a remote host using tar c | ssh remotehost tar x -. So when I change my configuration on my workstation, I have to do scpzshconf to many hosts. I would like to have something (a zsh function for example) which would be able to check which zsh configuration files are running on remote host and compare it with the version I am running localy. If remote version is older than local one, then it should scpzshconf from local host to remote host. (Just remember that sometimes I am using connections that are NATed).

Example:

local % cat ~/.zsh/configversion
2
local % ssh remote
remote % cat ~/.zsh/configversion
1

And then it should copy zsh configuration from local to remote.

Any idea how to handle this ?

Ashrak won

jd — Sun, 27 Feb 2005 13:42:09 +0000

Ok... You know, sometimes, you wake up and you feel it inside: it's a bad day.

So I was playing supertux on my laptop jolinar when... it powered itself off. Pfioouuu. Nothing more. And I cannot power it on again, the power button seems to do nothing.

Fortunately, my home is managed by Subversion and I made a svn commit 10 minutes before the crash. And I have a 10 days old backup on another machine, so I won't be bothered if I can't get my data back.

But I don't have a laptop anymore.

Ashrak are killers from Goa'uld, aiming at killing Tok'Ra rebels. Jolinar of Malkshur was one of them.

Laptop problem

jd — Sat, 15 Jan 2005 12:45:40 +0000

Since several month, I have a problem with my laptop. When I use the integrated network card, my computer freezes. Let me explain.

If I play a sound file with xmms while I'm surfing, it's ok. But if I stop typing at keyboard, 20-60s after the computer stops. The clock is blocked and the sound does not play anymore. I can't ssh to my laptop neither. When I come back I have to run ntpdate to set the clock back to the good time...

If I use my PCMCIA wifi card or if I unplug my network cable, I do not have any problem anymore.

I tried several driver and several 2.6 kernel for my Intel network card (e100 and eepro100) but it does not change anything. I use alsa for my sound card (snd-es1968).

I think it's an hardware issue, but I am not sure and I don't know why.

What you can do with VPN ? This !

jd — Wed, 22 Dec 2004 19:15:47 +0000

You can play with routing:

Host                                    Loss%   Snt   Last   Avg  Best  Wrst StDev
1. kheb.queton.naquadah.org              0.0%    47    1.1   2.1   0.3  36.1   5.9
2. netu.vpn.queton.naquadah.org          0.0%    46   74.7 230.8  66.7 2160. 387.6
3. nasya.vpn.dmz.naquadah.org            0.0%    46  129.6 354.1 121.4 1979. 408.9
4. gw.dmz.naquadah.org                   0.0%    46  117.7 390.4 115.6 2064. 453.1
5. abydos.adm.naquadah.org               0.0%    46  122.7 348.9 115.5 1958. 402.3

For people knowing my network arch:

jolinar -> kheb -> netu -> nasya -> othala -> abydos

This means:

laptop (parents home) -> queton gw -> Main VPN server -> DMZ VPN server -> dmz/adm gw -> workstation (home)

Netu is the central VPN server. queton.n.o and {adm,dmz}.n.o are two networks on 2 differents sites. Quite fun !

I use vtun for this.

New hard disk for my laptop

jd — Wed, 15 Dec 2004 18:18:39 +0000

Hey, it's Christmas time ! I bought a new hard drive for my laptop (the old was a slow Toshiba 10 GB). It's a Hitachi 40 GB 5400 RPM 8 MB !

I ran a quick-and-dirty installation of Sarge in order to restore a full / backup.

Restore requested to host jolinar, backup #46, by acid from 192.168.2.13

Wait & see.

Exim4 !

jd — Tue, 14 Dec 2004 15:28:56 +0000

Yeah, it's finally done: I have migrated my primary MX to exim4 ! With only a 15 minutes downtime.

Since several month, I said that I will do it, and I did it ! I rewrote my exim3 conf for exim4. It was not too hard, but took me some hours to test it since my box delivers about 1k mails/days. It seems that everything is ok, but I am still tailling -f paniclog...