Title : Aggregate internet links with mlvpn
Author: Solène
Date  : 28 March 2020
Tags  : openbsd68

In this article I'll explain how to aggregate internet access bandwidth using
**mlvpn** software. I struggled a lot to set this up so I wanted to share a
how-to.


## Pre-requisites

**mlvpn** is meant to be used with DSL / fiber links, not wireless or 4G links
with variable bandwidth or packet loss.

**mlvpn** requires to be run on a server which will be the public internet
access and on the client on which you want to aggregate the links, this is like
doing multiples VPN to the same remote server with a VPN per link, and
aggregate them.

Multi-wan roundrobin / load balancer doesn't allow to stack bandwidth but
doesn't require a remote server, depend on what you want to do, this may be
enough and mlvpn may not be required.

**mlvpn** should be OS agnostic between client / server but I only tried
between two OpenBSD hosts, your setup may differ.


## Some network diagram

Here is a simple network, the client has access to 2 ISP through two ethernet
interfaces.

em0 and em1 will have to be on different rdomains (it's a feature to separate
routing tables).

Let's say the public ip of the server is 1.2.3.4.


                    [internet]
                        ↑
                        | (public ip on em0)
                 #-------------#
                 |             |
                 |   Server    |
                 |             |
                 #-------------#
                    |       |
                    |       |
                    |       |
                    |       |
        (internet)  |       | (internet)
        #-------------#   #-------------#
        |             |   |             |
        |   ISP 1     |   |  ISP 2      |
        |             |   |             |  (you certainly don't control those)
        #-------------#   #-------------#
                    |       |
                    |       |
      (dsl1 via em0)|       | (dsl1 via em1)
                 #-------------#
                 |             |
                 |   Client    |
                 |             |
                 #-------------#


## Network configuration

As said previously, em0 and em1 must be on different rdomains, it can easily be
done by adding `rdomain 1` and `rdomain 2` to the interfaces configuration.

Example in **/etc/hostname.em0**

    rdomain 1
    dhcp


## mlvpn installation

On OpenBSD the installation is as easy as `pkg_add mlvpn` (should work starting
from 6.7 because it required patching).


## mlvpn configuration

Once the network configuration is done on the client, there are 3 steps to do
to get aggregation working:

1. mlvpn configuration on the server
2. mlvpn configuration on the client
3. activating NAT on the client


### Server configuration

On the server we will use the UDP ports 5080 et 5081.

Connections speed must be defined in bytes to allow **mlvpn** to correctly
balance the traffic over the links, this is really important.

The line `bandwidth_upload = 1468006` is the maximum **download bandwidth of the
client** on the specified link in bytes. If you have a download speed of 1.4 MB/s
then you can choose a value of 1.4\*1024\*1024 => 1468006.

The line `bandwidth_download = 102400` is the maximum **upload bandwidth of the
client** on the specified link in bytes. If you have an upload speed of 100 kB/s
then you can choose a value of 100*1024 => 102400.

The **password** line must be a very long random string, it's a shared secret
between the client and the server.

    # config you don't need to change
    [general]
    statuscommand = "/etc/mlvpn/mlvpn_updown.sh"
    protocol = "tcp"
    loglevel = 4
    mode = "server"
    tuntap = "tun"
    interface_name = "tun0"
    cleartext_data = 0
    ip4 = "10.44.43.2/30"
    ip4_gateway = "10.44.43.1"
    
    # things you need to change
    password = "apoziecxjvpoxkvpzeoirjdskpoezroizepzdlpojfoiezjrzanzaoinzoi"
    
    [dsl1]
    bindhost = "1.2.3.4"
    bindport = 5080
    bandwidth_upload = 1468006
    bandwidth_download = 102400
    
    [dsl2]
    bindhost = "1.2.3.4"
    bindport = 5081
    bandwidth_upload = 1468006
    bandwidth_download = 102400


### Client configuration

The `password` value must match the one on the server, the values of `ip4` and
`ip4_gateway` must be reversed compared to the server configuration (this is so
in the following example).

The `bindfib` lines must correspond to the according rdomain values of your
interfaces.

    # config you don't need to change
    [general]
    statuscommand = "/etc/mlvpn/mlvpn_updown.sh"
    loglevel = 4
    mode = "client"
    tuntap = "tun"
    interface_name = "tun0"
    ip4 = "10.44.43.1/30"
    ip4_gateway = "10.44.43.2"
    timeout = 30
    cleartext_data = 0
    
    password = "apoziecxjvpoxkvpzeoirjdskpoezroizepzdlpojfoiezjrzanzaoinzoi"
    
    [dsl1]
    remotehost = "1.2.3.4"
    remoteport = 5080
    bindfib = 1
    
    [dsl2]
    remotehost = "1.2.3.4"
    remoteport = 5081
    bindfib = 2


### NAT configuration (server side)

As with every VPN you must enable packet forwarding and create a pf rule for
the NAT.


**Enable forwarding**

Add this line in **/etc/sysctl.conf**:

    net.inet.ip.forwarding=1

You can enable it now with `sysctl net.inet.ip.forwarding=1` instead of waiting
for a reboot.

In pf.conf you must allow the UDP ports 5080 and 5081 on the public interface
and enable nat, this can be done with the following lines in pf.conf but you
should obviously adapt to your configuration.

    # allow NAT on VPN
    pass in on tun0
    pass out quick on em0 from 10.44.43.0/30 to any nat-to em0
    
    # allow mlvpn to be reachable
    pass in on egress inet proto udp from any to (egress) port 5080:5081


## Start mlvpn

On both server and client you can run mlvpn with rcctl:

    rcctl enable mlvpn
    rcctl start mlvpn

You should see a new tun0 device on both systems and being able to ping them
through tun0.

Now, on the client **you have to add a default gateway through the mlvpn
tunnel** with the command ` route add -net default 10.44.43.2` (adapt if you
use others addresses). I still didn't find how to automatize it properly.

Your client should now use both WAN links and being visible with the remote
server public IP address.

**mlvpn** can be used for more links, you only need to add new sections.
**mlvpn** also support IPv6 but I didn't take time to find how to make it work,
si if you are comfortable with ipv6 it may be easy to set up IPv6 with the
variables `ip6` and `ip6_gateway` in mlvpn.conf.