OpenWRT: Setup IPsec Racoon

From mUTU
Jump to: navigation, search


IPsec Basics For an overview over all existing Virtual private network (VPN)-related articles in the OpenWrt wiki, please visit vpn.overview

!: This page is about racoon. The new strongSwan documentation can be found here.

A quick starters quide based on Backfire 10.03.1-rc6. Maybe it will save you and me time if one has to setup an IPsec VPN in the future. Hopefully it will ecourage other people to use Openwrt as an IPsec VPN router. We cannot provide a graphical user interface at the moment but at least it is a solid alternative to commercial IPsec appliances. If you came here for informations about Openswan on OpenWrt you may be disappointed. This guide is only about racoon, see

–UPDATE– Openswan documentation is being put together here IPsec Site To Site Using Openswan Packages

If not already installed on your router you need the at least those packages

   ipsec-tools: racoon, setkey, and kernel encryption modules
   kmod-crypto-authenc: Module for block cipher modes (AEAD) (automatically installed with ipsec-tools in latest trunk)
   kmod-ipsec: Basic security module (automatically installed with ipsec-tools in latest trunk)
   kmod-ipsec4: IPv4 security module
   kmod-ipsec6: IPv6 security module
   ip: Required to make scripting easier
   openssl-util: Certificate handling
   iptables-mod-nat-extra: For VPN networks with overlapping IP addresses
   ip6tables: IPv6 firewall support

Altogether those packages will eat up about 1,2 MB of your router's flash memory. The racoon and ip binaries will already be 650KB. Maybe it is time for an extroot installation? Configuration concept

With Linux default IPsec daemon is called racoon. For this one normally edits some files

   /etc/racoon.conf: Central configuration file & endpoint definitions
   /etc/racoon/psk.txt: List of preshared keys
   /etc/racoon/setkey.conf: Security policies for tunnels

The major challenges are handling setkey.conf with dynamic IP addresses and clean integration into the OpenWrt configuration concept. To solve this we will use a hierarchical configuration process. That involves

   /etc/config/racoon: The OpenWrt configuration file for racoon
   /etc/init.d/racoon: The racoon start script. It will generate the required configuration files for racoon
   /var/racoon/racoon.conf: The generated racoon config
   /var/racoon/psk.txt : The generated file with preshared keys
   /var/racoon/cert: The folder with generated certificates

Here a short example of the configuration methodology when having two VPN tunnels to ACME and Yabadoo networks

  1. /etc/config/racoon

config 'tunnel' 'ACME'

 option 'enabled' '1'
 option 'remote' ''
 list   'sainfo' 'acme_lan'

config 'sainfo' 'acme_lan'

 option 'local_address' ''
 option 'remote_address' ''

config 'tunnel' 'Yabadoo'

 option 'enabled' '1'
 option 'remote' ''

Read more about the complete syntax for /etc/config/racoon. IKE Daemon

As already mentioned it can be a little scary to insert security polices into the kernel. Especially when you have a reconnecting interface with a dynamic outside IP. But to be honest all we have to know is already inside our /etc/config/racoon file. So we just have to use the startup script to parse those files and hand them over to the IPSec stack via setkey command. Our solution involves

   automatically determine interfaces and IPs
   allow multiple sainfo sets per tunnel
   if racoon is already running reload policies only
   generate certificates and their hashes

To let racoon run as a background daemon we can place a hook in the init environment. Therefore create the file /etc/init.d/racoon and set the executable bit. Remark: This script is in an advanced beta state. It currently works for site to site or roadwarrior tunnels with either preshared keys or RSA certificates. Feel free to enhance it.

Enhancement: It works also with cisco ASA remote endpoint now :-)

  1. !/bin/sh /etc/rc.common
  2. /etc/init.d/racoon - version 27

NAME=racoon PROG=/usr/sbin/racoon START=60 STOP=60

. /etc/

ConfigFile="/var/racoon/racoon.conf" PSKFile="/var/racoon/psk.txt" UserFile="/var/racoon/xauthuser.txt" CertificatePath="/var/racoon/cert" CallParameters="" RoadWarriorRemote="anonymous" RoadWarriorDNS="" RoadWarriorDomain="" RoadWarriorCerts="" MainConfigDone=0

UserConfig() {

 local enabled
 local xauth
 local name
 local password
 local crt_subject

 config_get_bool enabled $1 enabled 0
 "$enabled" == "0"  && return

 config_get_bool xauth       $1 xauth       0
 config_get      name        $1 name        ""
 config_get      password    $1 password    ""
 config_get      crt_subject $1 crt_subject ""

 if [ $xauth -eq 1 -a "$name" != "" -a "$password" != "" ]; then
   echo "$name $password" >> $UserFile

 if [ "$crt_subject" != "" ]; then
   RoadWarriorCerts="${RoadWarriorCerts}peers_identifier asn1dn \"$crt_subject\";"


CreateSA() {

 local LocalEndpoint=`ip route get $3 | awk -F"src" '/src/{gsub(/ /,"");print $2}' | sed -e 's/metric.*//'`

 echo "spdadd $1 $2 any -P out ipsec \
       esp/tunnel/$LocalEndpoint-$3/unique; \
       spdadd $2 $1 any -P in ipsec \
       esp/tunnel/$3-$LocalEndpoint/unique; \
      " | setkey -c 1>&2


RoadWarriorSubnet() {

 local i

 let "i=255<<(24-${1#*/}+8)&255"
 echo "  netmask4 255.255.255.$i;"

 let "i=${i##*.}+1"
 echo "  network4 ${1%.*}.$i;"


MainConfig() {

 local foreground
 local debug
 local listen

 config_get_bool foreground        $1 foreground 0
 config_get_bool debug             $1 debug      0
 config_get      listen            $1 listen     ""
 config_get      RoadWarriorDNS    $1 dns        ""
 config_get      RoadWarriorDomain $1 domain     ""

 $foreground -ne 0  && CallParameters="-F"
 $debug -ne 0       && CallParameters=$CallParameters" -d"

 echo "# auto generated by /etc/init.d/racoon"
 echo "path pre_shared_key \"$PSKFile\";"
 echo "path certificate \"$CertificatePath\";"
 echo "padding {"
 echo "  maximum_length 20; randomize off;"
 echo "  strict_check off; exclusive_tail off;"
 echo "}"
 echo "timer {"
 echo "  counter 5; interval 20 sec; persend 1;"
 echo "  phase1 30 sec; phase2 15 sec;"
 echo "}"
 echo ""

 if [ "$listen" != "" ]; then
   echo "listen {"
   config_list_foreach "$1" listen AddListenIP
   echo "}"
   echo ""


AddListenIP() {

 local value="$1"
 . /lib/functions/
 local retvalue
 network_get_ipaddr listenIP "$value"
 if [ $? -eq 0 ]; then
   if [ "$listenIP" != "" ]; then
     echo "  isakmp $listenIP;"
     echo "  isakmp_natt $listenIP [4500];"
 network_get_ipaddr6 listenIP "$value"
 if [ $? -eq 0 ]; then
   if [ "$listenIP" != "" ]; then
     echo "  isakmp $listenIP;"
     echo "  isakmp_natt $listenIP [4500];"


TunnelConfig() {

 local enabled
 local remote
 local remote_device
 local preshared_key
 local exchange_mode
 local my_identifier
 local my_identifier_type
 local certificate
 local remote_device

 config_get_bool enabled "$1" enabled 0
 "$enabled" == "0"  && return

 config_get remote             "$1" remote
 config_get remote_device      "$1" remote_device ""
 config_get pre_shared_key     "$1" pre_shared_key ""
 config_get exchange_mode      "$1" exchange_mode
 config_get my_identifier      "$1" my_identifier ""
 config_get my_identifier_type "$1" my_identifier_type "fqdn"
 config_get certificate        "$1" certificate ""
 config_get dpd_delay		"$1" dpd_delay ""

 if [ "$remote" != "$RoadWarriorRemote" ]; then
   if [ -x /usr/bin/dnsip ] ; then
     remote=`/usr/bin/dnsip $remote`
     remote=`nslookup "$remote" | awk 'NR==5 { print $3 }'`
   prg=`racoon -V 2>/dev/null| grep "ipsec-tools 0.8" | wc -l`
   if [ $prg -eq 0 ]; then
     echo "remote $remote {"
     echo "remote \"$1\" {"
     echo "  remote_address $remote;"

   echo "remote anonymous {"
   echo "  generate_policy on;"

 if [ "$pre_shared_key" != "" ]; then
   if [ "$remote" != "$RoadWarriorRemote" ]; then
     echo "$remote $pre_shared_key" >> $PSKFile
     echo "* $pre_shared_key" >> $PSKFile
   if [ "$my_identifier" != "" -a "$my_identifier_type" != "" ]; then
     echo "  my_identifier $my_identifier_type  \"$my_identifier\";"
 elif [ "$certificate" != "" ]; then
   echo "  verify_cert on;"
   echo "  certificate_type x509 \"$certificate.crt\" \"$certificate.key\";"
   echo "  my_identifier asn1dn;"
   if [ "$remote" == "$RoadWarriorRemote" ]; then
     echo "  verify_identifier on;"
     echo "  "$RoadWarriorCerts
     echo "  peers_identifier asn1dn;"

 echo "  exchange_mode $exchange_mode;"
 echo "  proposal_check obey;"
 echo "  nat_traversal on;"
 if [ "$dpd_delay" != "" ]; then
   echo "  dpd_delay $dpd_delay;"

 config_list_foreach "$1" p1_proposal ConfigP1
 echo "}"

 config_list_foreach "$1" sainfo ConfigSA $tunnel $remote
 "$remote" == "$RoadWarriorRemote"  && echo "}"


ConfigP1() {

 local lifetime
 local encryption_algorithm
 local hash_algorithm
 local authentication
 local dh_group

 config_get lifetime              "$1" lifetime 28800
 config_get encryption_algorithm  "$1" encryption_algorithm
 config_get hash_algorithm        "$1" hash_algorithm
 config_get authentication_method "$1" authentication_method
 config_get dh_group              "$1" dh_group

 echo "  proposal {"
 echo "    lifetime time $lifetime sec;"
 echo "    encryption_algorithm $encryption_algorithm;"
 echo "    hash_algorithm $hash_algorithm;"
 echo "    authentication_method $authentication_method;"
 echo "    dh_group $dh_group;"
 echo "  }"


ConfigSA() {

 local tunnel=`echo $2 | cut -d" " -f1`
 local sainfo=$1
 local remote=`echo $2 | cut -d" " -f2`
 local local_subnet
 local local_nat
 local remote_subnet
 local p2_proposal
 local pfs_group
 local lifetime
 local encryption_algorithm
 local local authentication_algorithm

 config_get local_subnet             "$sainfo"      local_subnet
 config_get local_nat                "$sainfo"      local_nat ""
 config_get remote_subnet            "$sainfo"      remote_subnet
 config_get p2_proposal              "$sainfo"      p2_proposal
 config_get pfs_group                "$p2_proposal" pfs_group
 config_get lifetime                 "$p2_proposal" lifetime 3600
 config_get encryption_algorithm     "$p2_proposal" encryption_algorithm
 config_get authentication_algorithm "$p2_proposal" authentication_algorithm

 "$local_nat" != ""  && local_subnet=$local_nat

 if [ "$remote" = "$RoadWarriorRemote" ]; then
   let AnonSA=$AnonSA+1
 $AnonSA -eq 1  && echo "sainfo anonymous {"
 if [ $AnonSA -eq 0 ]; then
   CreateSA $local_subnet $remote_subnet $remote
   echo "sainfo address $local_subnet any address $remote_subnet any {"
 if [ $AnonSA -lt 2 ]; then
   "$remote_device" != "asa"  &&  echo "  pfs_group $pfs_group;"
   echo "  lifetime time $lifetime sec;"
   echo "  encryption_algorithm $encryption_algorithm;"
   echo "  authentication_algorithm $authentication_algorithm;"
   echo "  compression_algorithm deflate;"
   echo "}"
 if [ $AnonSA -eq 1 ]; then
   echo "mode_cfg {"
   echo "  auth_source system;"
   echo "  conf_source local;"
   "$RoadWarriorDNS"    != ""  && echo "  dns4 $RoadWarriorDNS;"
   "$RoadWarriorDomain" != ""  && echo "  default_domain \"$RoadWarriorDomain\";"
   RoadWarriorSubnet $remote_subnet
 if [ $AnonSA -gt 0 ]; then
   echo "  split_network include $local_subnet;"


CertConfig() {

 local val
 local hash

 for opt in key crt; do
   config_get val "$1" "$opt" ""
   if [ "$val" != "" ]; then
     echo $val | sed "s/-\+[A-Z ]\+-\+/\n&\n/g" \
               | sed "s/.\{50,50\}/&\n/g" \
               | sed "/^$/d" > $CertificatePath/$1.$opt
     chmod 600 $CertificatePath/$1.$opt

 hash=`openssl x509 -noout -hash -in $CertificatePath/$1.crt`
 ln -s -f $CertificatePath/$1.crt $CertificatePath/$hash.0


CheckEnvironment() {

 local prg
 for prg in /usr/bin/openssl /usr/sbin/ip; do
   if [ ! -x $prg ]; then
     echo "Error! $prg missing. Exit now."

 mkdir -m 0700 -p /var/racoon
 mkdir -m 0700 -p $CertificatePath


start() {

 local active=`ps | grep /usr/sbin/racoon | grep -v grep | wc -l`


 if [ $active -eq 0 -o "$1" = "force" ]; then
   rm $UserFile 2>/dev/null
   config_load users
   config_foreach UserConfig user

 config_load racoon

 if [ $active -ne 0 -a "$1" != "force" ]; then
   config_foreach CertConfig certificate
   echo "# auto generated by /etc/init.d/racoon" > $PSKFile
   chmod 600 $PSKFile

 config_foreach MainConfig racoon > $ConfigFile
 if [ $MainConfigDone -eq 0 ]; then
   MainConfig XXX > $ConfigFile

 echo "flush; spdflush;" | setkey -c
 config_foreach TunnelConfig tunnel >> $ConfigFile

 if [ $active -eq 0 ]; then
   /usr/sbin/racoon $CallParameters -f $ConfigFile
 elif [ "$1" = "force" ]; then
   racoonctl reload-config


stop() {

 pid=`ps | grep /usr/sbin/racoon | grep -v grep | awk '{ print $1}'`
 "$pid" != ""  && kill $pid
 echo "flush; spdflush;" | setkey -c


restart() {

 start force


Before you start racoon with the web interface you should make a dry run from command line. Enable forground operation in /etc/config/racoon by setting option 'foreground' '1' in section racoon and call /etc/init.d/racoon start. This will show you if there are any errors in your generated configuration file /var/racoon/racoon.conf. Afterwards you can control startup behaviour with LuCI.

An automatic reload of security policies after a router reconnect is very helpful. Luckily pppd will call all scripts in /etc/ppp/ip-up.d after pppoe-wan is up again. Create a small script in this directory that calls the racoon init script. If it detects that racoon is running already it will only set the security policies.

  1. !/bin/sh
  2. /etc/ppp/ip-up.d/racoon

/etc/init.d/racoon start

Another way to reload racoon when the wan IP changes and we are not using ppp, is creating a script like this in /etc/hotplug.d/iface/35-racoon

  1. !/bin/sh

ListenInterface() {

 local iface="$1"
 if [ "$INTERFACE" = "$iface" ]; then
   /etc/init.d/racoon restart


RacoonInstance() {

 config_list_foreach "$1" listen ListenInterface


if [ "$ACTION" = "ifup" ]; then

 config_load racoon
 config_foreach RacoonInstance racoon


Hardware performance

In the times of broadband internet connections encryption and decryption speed of routers can limit throughput of VPN tunnels. CPU utilization maxes out at 100 percent and impacts other services of the device like a web server. If you really want to go with a self made IPsec VPN on a cheap router you should consider some facts

   Older firewall devices with hardware accelerated VPN are sold for a few bucks on Ebay. Juniper Netscreen 5GT for example can easily reach a VPN throughput of 20 MBit/sec. Downside is that firmware updates are only possible with a Juniper support contract. So check twice for a bargain.
   Firewall devices are build to support IPsec out of the box. A convenient web interface helps the administrator to build a tunnel in a few seconds. OpenWrt still lacks a standard LuCI config panel. If you only go with 1-5 VPN tunnels this should be no concern to you.

To find the right OpenWrt hardware for your VPN you should have a look at the following benchmark table. It is build on a simple test without any claim of perfection. Nevertheless the numbers are quite close to what you can expect from an AES 128/256 bit encrypted IPsec Tunnel connection with standard kernel modules. You may notice that those numbers differ from what is written on the OpenSSL wiki page. But simply remember: The tests over there do not include network traffic. If you want to add a new device onto the list check the encrpytion throughput using the following prerequisites

   Logon to a fast Linux machine
   Use a direct LAN connection to the router
   Ensure the router is idle
   Transfer 100 MB of data using ssh
   Calculate the speed from the elapsed time. Throughput = 800 / SecondsElapsed

ssh -2 -c aes128-cbc root@<router> time dd if=/dev/zero bs=500000 count=200 > /dev/null ssh -2 -c aes256-cbc root@<router> time dd if=/dev/zero bs=500000 count=200 > /dev/null

You can have a look at the realtime traffic graph in a dry run afterwards to verify the speed. But do not open it during your test because it invalidates the results. CPU MHz tested device AES128 (s) AES128 (MBit/s) AES256 (s) AES256 (MBit/s) MIPS 24k 680 D-Link DIR-825 Netgear WNDR3700 28.2 28.5 32.4 24.6 MIPS 24k 400 TP-Link TL-WR703N 47.7 16.5 56.1 14.2 MIPS R3000 125 Asus WL-500g 164.8 4.8 183.5 4.3 IPsec Tuning

If you use a default OpenWrt installation you will discover that using the SHA1 hashing function will hit VPN performance. If you go for raw throughput MD5 can be a helpful alternative. One may remark that MD5 is not very secure but for IPsec connections it should be enough as we are talking about hash values of encrypted data with a key that is changed every hour according to phase 2 proposals. A good tradeoff could be to choose AES256/SHA1 for phase 1 and AES128/MD5 for phase 2.

Read on if you have some time and want to enhance your VPN speed. The kernel IPsec architecture relies on different crypto providers. E.g. if you build a tunnel with SHA1 checksums you must hava a module that can calculate those values. A look at /proc/crypto will reveal what modules are loaded and which algorithms they provide. The standard Linux Kernel modules are far from being optimized. At least with kernel 3.2 someone has taken care of SHA1. Those of you that are on MIPS big endian machines can replace the default aes_generic.ko, sha_generic.ko, cbc.ko and md5.ko modules with a single assembler optimized mcespi.ko. Besides of being faster it has some nice characteristcs:

   SHA1 calculation works on registers only
   A lot of memcpy operations have been removed from MD5 and AES.
   AES memory footprint is lowered from 16K to 8,2K
   Avoid little endian byte swapping in AES
   21K module size in contrast to 4 modules with 45K

If you are on AR7161 you should ensure that you already have unaligned access patch 1 from trunk and the not yet implemented unaligned access patch 2. It will free CPU from handling unaligned access expections so that you can reach these results: OpenWrt Device AES256/SHA1 (MBit/s) AES128/SHA1 (MBit/s) AES128/MD5 (MBit/s) trunk + mcespi.ko MIPS 24K @ 680 MHz (AR7161) 37.5 42.4 47.6 10.03.1-rc6 MIPS 24K @ 680 MHz (AR7161) 18.0 19.6 30.3

The module is in early alpha development an the easyiest way to install it includes a few steps.

   create a buildroot environment
   compile an image for your router once
   put the mcespi.c into the the folder build_dir/linux-<arch>/linux-<X.Y.Z>/crypto
   Include the line obj-$(CONFIG_CRYPTO_AES) += mcespi.o into build_dir/linux-<arch>/linux-<X.Y.Z>/crypto/Makefile
   compile the image once again.
   Afterwards you will find build_dir/linux-<arch>/linux-<X.Y.Z>/crypto/mcespi.ko
   Put mcespi.ko to your router into /lib/modules/<X.Y.Z>
   Load the module with insmod
   For automatic loading create a new /etc/modules.d/09-crypto-mcespi with corresponding content.

If you are not on MIPS big endian but you have at least kernel 2.6.39 you can head for SHA1 performance optimization. Download sha_optimized.c and build it as a module. FIXME Ticket #10637 tries to implement that patch into trunk. At least on Atheros AR7161 platform and current trunk this leads to an oops. If you can help and find out why just put your feedback here or over there. Note: On current trunk the default kernel is 3.3 and sha_optimized.c does not have to be backported anymore.

mv 259-crypto_SHA1_3.2_backport.patch target/linux/generic/patches-XXX make target/linux/clean make world

OpenSSL Tuning

While talking about performance optimization there is also room for some improvement for non-VPN encrpytion scenarios. With newer versions of OpenSSL more and more assembler encryption and decryption routines are included. Check-in 21708 provides those for MIPS architecture and the 1.0.1 branch . Porting some of the assembler routines to OpenSSL 0.9.8p for ar71xx (Backfire 10.06.01-rc6) involves the following steps in the buildroot environment:

   Remove build_dir/target_xxx/openssl-0.9.8p/*
   Remove line OPENSSL_OPTIONS += no-perlasm from package/openssl/Makefile
   Replace "${no_asm}" with ":::aes_cbc.o aes-mips.o:::::::" in file package/openssl/patches/110-optimize-for-size.patch. Do not use quotation marks and ensure that you include those 10 colons.
   Start build process for first time with make package/openssl/compile V=99. This process will stop with an error.
   Copy crypto/aes/asm/ from OpenSLL 1.0.1 sources and store it into the newly created build_dir/target_xxx/openssl-0.9.8p/crypto/aes/asm folder.
   Modify build_dir/target_xxx/openssl-0.9.8p/crypto/aes/Makefile and insert two lines (the second one starts with a tab)

aes-mips.s: asm/

       $(PERL) asm/ > $@
   Restart build process with make package/openssl/compile V=99. It will complete without errors this time.
   Copy build_dir/target_xxx/openssl-0.9.8p/ to your router

Comparing benchmark numbers before and afterwards with openssl speed gives a speedup of about 8% for aes-128-cbc and about 5% for aes-256-cbc.