I have several servers around the Internet, and one of my need is to have VPN between them to access various data and services, like SNMP or NFS. I used to build VPN with OpenVPN between some of them.
My main problem was that I had to set up IP interfaces at each end, build multiple tunnels or make some routing, and... that was not what I wanted, too complicated.
I took a deeper look in OpenVPN and found a solution: I created a FLAN… No, not a cake, a Faked LAN! This is so easy and powerful that I beat myself to not have though about that before.
I did it with 3 servers for now, this is how it works: on each server I create an interface named if-remote hostname which is a tap (Ethernet) interfaces connected to the remote host. On server1, I have two interfaces, if-server2 and if-server3 which are like direct wired connection to the remote host, and I can do Ethernet on them.
When each server has its link up, I create an Ethernet bridge. In this bridge, I put the two interfaces connected to the remote servers. This finally build something like that for each server:
server1 -- eth0
\
if-bridge 192.168.4.88
\ \_ if-server2 -- VPN -- if-server1 -- if-bridge 192.168.4.89 -- server2
\_ if-server3 -- VPN -- if-server-1 -- if-bridge 192.168.4.90 -- server3
Obviously, you will have a loop in your faked Ethernet LAN, so you'll have to active STP in order to have a working LAN. And it works.
The main advantage with this solution is that each server has only one IP on this virtual network, and there's no connection problem if one of your host is down: STP will rebuild the network in a transparent way. The main problem is that you may have to set up a lot of link if you want a full redundancy. I think I'll setup only two links between each server, because it maybe too painfull otherwise.
You can also do some optimization if you need, because the current bridge implementation in Linux is so powerful that you can put cost on interfaces for your bridge, or define which bridge should be the root of your tree. Have fun!