Free Range Routing (FRR) is a powerful, open-source routing software suite that provides implementations of various routing protocols, including BGP, OSPF, IS-IS, RIP, PIM, and more.
It’s designed to run on Linux and Unix-like systems, making it a flexible solution for a wide range of network setups—from small labs to large-scale data centers.
Why FRR?
- Scalability: Supports complex network topologies.
- Flexibility: Easily integrates with existing network infrastructures.
- Community-driven: Regular updates and active community support.
A Brief History
FRR originated as a fork of the Quagga project (which is still used for the Looking Glass service) in 2016, aiming to create a more dynamic and community-focused development path. Since then, it has grown into a robust and widely adopted routing platform, used by service providers, enterprises, and research institutions.
Demo Lab Overview
🌐 Topology
This demo lab showcases a Hub-and-Spoke topology using WireGuard for secure tunneling between the nodes.
We use unique ASN (Autonomous System Number) to run eBGP (external Border Gateway Protocol) between the entities.
The OS we use is Ubuntu 24.04.1.
Hub:
- Public IP: Static (known)
- Tunnel IP:
10.5.5.1
Spoke #1:
- Public IP: Ephemeral
- Tunnel IP:
10.5.5.20
Spoke #2:
- Public IP: Ephemeral
- Tunnel IP:
10.5.5.10
The Hub acts as a central point with a fixed public IP, while both Spokes establish dynamic WireGuard connections, enabling BGP peering over the secure tunnels.
Objectives
- Establish WireGuard tunnels between the Hub and Spokes.
- Configure BGP on FRR to route traffic between the nodes.
- Ensure seamless communication between Spokes through the Hub.
In the next sections, we’ll dive into the WireGuard setup, followed by configuring FRR BGP for efficient routing.
Firewall Considerations
- Hub:
- Allow inbound UDP 51820 to accept incoming WireGuard connections from the Spokes.
- Allow inbound UDP 51820 to accept incoming WireGuard connections from the Spokes.
- Spokes:
- Allow outbound UDP 51820 to the Hub’s public IP to establish the WireGuard tunnel.
- BGP (TCP 179) runs inside the WireGuard tunnel and does not require any firewall exceptions.
Why Use WireGuard?
We chose WireGuard for this setup to enhance the privacy, integrity, and security for every bit we transport across the internet.
Wireguard provides:
- End-to-End Encryption: All traffic between Hub and Spokes is encrypted using state-of-the-art cryptographic protocols (ChaCha20 for encryption, Poly1305 for message authentication).
- Simplicity & Performance: WireGuard is lightweight, easy to configure, and offers high performance with low overhead.
- Ephemeral IP Handling: Its ability to handle dynamic public IPs makes it ideal for spokes with changing network addresses.
- Integrity & Authentication: Only peers with the correct public keys can establish connections, ensuring data integrity and preventing unauthorized access.
WireGuard Setup
📦 Prerequisites
Ensure WireGuard is installed on all nodes:
sudo apt update
sudo apt install wireguard🔑 Key Generation
On each node (Hub and Spokes), generate WireGuard key pairs:
cd /etc/wireguard/
wg genkey | tee privatekey | wg pubkey > publickeyprivatekey→ Keep this secure.publickey→Share with peers.
⚙️ Hub Configuration (/etc/wireguard/wg0.conf)
[Interface]
Address = 10.5.5.1/24
ListenPort = 51820
PrivateKey = <Hub_Private_Key>
# Spoke #1
[Peer]
PublicKey = <Spoke1_Public_Key>
AllowedIPs = 10.5.5.20/32
# Spoke #2
[Peer]
PublicKey = <Spoke2_Public_Key>
AllowedIPs = 10.5.5.10/32⚙️ Spoke Configuration (/etc/wireguard/wg0.conf)
Spoke #1:
[Interface]
Address = 10.5.5.20/32
PrivateKey = <Spoke1_Private_Key>
[Peer]
PublicKey = <Hub_Public_Key>
Endpoint = <Hub_Public_IP>:51820
AllowedIPs = 10.5.5.0/24
PersistentKeepalive = 25Spoke #2:
[Interface]
Address = 10.5.5.10/32
PrivateKey = <Spoke2_Private_Key>
[Peer]
PublicKey = <Hub_Public_Key>
Endpoint = <Hub_Public_IP>:51820
AllowedIPs = 10.5.5.0/24
PersistentKeepalive = 25🚀 Start WireGuard
On all nodes, start and enable WireGuard:
sudo wg-quick up wg0
sudo systemctl enable wg-quick@wg0✅ Verify Tunnel
Run on each node to check peer status:
sudo wg showexample-output for Spoke1:
root@spoke1:/home/ugu5ma# wg show
interface: wg0
public key: kRhYptcrypticPublicKeyHFZRg=
private key: (hidden)
listening port: 42119
peer: zqkjHAd3+crypticPublicKeymCU4=
endpoint: <Publix-IP>:51820
allowed ips: 10.5.5.0/24
latest handshake: 1 minute, 41 seconds ago
transfer: 8.50 KiB received, 10.52 KiB sent
persistent keepalive: every 25 seconds
root@spoke1:/home/ugu5ma#Once the tunnels are active, you can ping between the nodes using their Tunnel IPs.
Next, we’ll dive into configuring BGP to enable dynamic routing over the WireGuard tunnels.
Install FRR
Ensure FRR is installed on all nodes, we will stick on the stable release of FRR:
# add GPG key
curl -s https://deb.frrouting.org/frr/keys.gpg | sudo tee /usr/share/keyrings/frrouting.gpg > /dev/null
FRRVER="frr-stable"
echo deb '[signed-by=/usr/share/keyrings/frrouting.gpg]' https://deb.frrouting.org/frr \
$(lsb_release -s -c) $FRRVER | sudo tee -a /etc/apt/sources.list.d/frr.list
# update and install FRR
sudo apt update && sudo apt install frr frr-pythontools
expected output:
root@hub:/home/ugu5ma# # add GPG key
curl -s https://deb.frrouting.org/frr/keys.gpg | sudo tee /usr/share/keyrings/frrouting.gpg > /dev/null
# possible values for FRRVER:
FRRVER="frr-stable"
echo deb '[signed-by=/usr/share/keyrings/frrouting.gpg]' https://deb.frrouting.org/frr \
$(lsb_release -s -c) $FRRVER | sudo tee -a /etc/apt/sources.list.d/frr.list
# update and install FRR
sudo apt update && sudo apt install frr frr-pythontools
deb [signed-by=/usr/share/keyrings/frrouting.gpg]
https://deb.frrouting.org/frr noble frr-stable
Hit:1 http://de.archive.ubuntu.com/ubuntu noble InRelease
Hit:2 http://de.archive.ubuntu.com/ubuntu noble-updates InRelease
Hit:3 http://de.archive.ubuntu.com/ubuntu noble-backports InRelease
Hit:4 http://security.ubuntu.com/ubuntu noble-security InRelease
Get:5 https://deb.frrouting.org/frr noble InRelease [34.3 kB]
Get:6 https://deb.frrouting.org/frr noble/frr-stable amd64 Packages [5,461 B]
Fetched 39.8 kB in 1s (29.2 kB/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
All packages are up to date.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
libcares2 libyang2
Suggested packages:
frr-doc
The following NEW packages will be installed:
frr frr-pythontools libcares2 libyang2
0 upgraded, 4 newly installed, 0 to remove and 0 not upgraded.
Need to get 7,032 kB of archives.
After this operation, 40.9 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://de.archive.ubuntu.com/ubuntu noble/main amd64 libcares2 amd64 1.27.0-1.0ubuntu1 [73.7 kB]
Get:2 https://deb.frrouting.org/frr noble/frr-stable amd64 libyang2 amd64 2.1.128-2~ubuntu24.04u1 [506 kB]
Get:3 https://deb.frrouting.org/frr noble/frr-stable amd64 frr amd64 10.2.1-0~ubuntu24.04.1 [6,414 kB]
Get:4 https://deb.frrouting.org/frr noble/frr-stable amd64 frr-pythontools all 10.2.1-0~ubuntu24.04.1 [38.4 kB]
Fetched 7,032 kB in 7s (966 kB/s)
Selecting previously unselected package libcares2:amd64.
(Reading database ... 86641 files and directories currently installed.)
Preparing to unpack .../libcares2_1.27.0-1.0ubuntu1_amd64.deb ...
Unpacking libcares2:amd64 (1.27.0-1.0ubuntu1) ...
Selecting previously unselected package libyang2:amd64.
Preparing to unpack .../libyang2_2.1.128-2~ubuntu24.04u1_amd64.deb ...
Unpacking libyang2:amd64 (2.1.128-2~ubuntu24.04u1) ...
Selecting previously unselected package frr.
Preparing to unpack .../frr_10.2.1-0~ubuntu24.04.1_amd64.deb ...
Unpacking frr (10.2.1-0~ubuntu24.04.1) ...
Selecting previously unselected package frr-pythontools.
Preparing to unpack .../frr-pythontools_10.2.1-0~ubuntu24.04.1_all.deb ...
Unpacking frr-pythontools (10.2.1-0~ubuntu24.04.1) ...
Setting up libyang2:amd64 (2.1.128-2~ubuntu24.04u1) ...
Setting up libcares2:amd64 (1.27.0-1.0ubuntu1) ...
Setting up frr (10.2.1-0~ubuntu24.04.1) ...
info: Selecting GID from range 100 to 999 ...
info: Adding group `frrvty' (GID 110) ...
info: Selecting GID from range 100 to 999 ...
info: Adding group `frr' (GID 111) ...
info: The home dir /nonexistent you specified can't be accessed: No such file or directory
info: Selecting UID from range 100 to 999 ...
info: Adding system user `frr' (UID 110) ...
info: Adding new user `frr' (UID 110) with group `frr' ...
info: Not creating `/nonexistent'.
Created symlink /etc/systemd/system/multi-user.target.wants/frr.service → /usr/lib/systemd/system/frr.service.
Setting up frr-pythontools (10.2.1-0~ubuntu24.04.1) ...
Processing triggers for rsyslog (8.2312.0-3ubuntu9) ...
Processing triggers for man-db (2.12.0-4build2) ...
Processing triggers for libc-bin (2.39-0ubuntu8.4) ...
Scanning processes...
Scanning candidates...
Scanning linux images...
Running kernel seems to be up-to-date.
Restarting services...
Service restarts being deferred:
/etc/needrestart/restart.d/dbus.service
systemctl restart systemd-logind.service
systemctl restart unattended-upgrades.service
No containers need to be restarted.
User sessions running outdated binaries:
ugu5ma @ session #1: login[1283]
ugu5ma @ user manager service: systemd[1420]
No VM guests are running outdated hypervisor (qemu) binaries on this host.
root@hub:/home/ugu5ma#Check if FRR daemon is up and running with systemctl status frr.service
output:
root@hub:/home/ugu5ma# systemctl status frr.service
● frr.service - FRRouting
Loaded: loaded (/usr/lib/systemd/system/frr.service; enabled; preset: enabled)
Active: active (running) since Thu 2025-02-13 11:29:06 UTC; 4min 54s ago
Docs: https://frrouting.readthedocs.io/en/latest/setup.html
Process: 14391 ExecStart=/usr/lib/frr/frrinit.sh start (code=exited, status=0/SUCCESS)
Main PID: 14401 (watchfrr)
Status: "FRR Operational"
Tasks: 8 (limit: 4554)
Memory: 14.7M (peak: 27.3M)
CPU: 223ms
CGroup: /system.slice/frr.service
├─14401 /usr/lib/frr/watchfrr -d -F traditional zebra mgmtd staticd
├─14411 /usr/lib/frr/zebra -d -F traditional -A 127.0.0.1 -s 90000000
├─14416 /usr/lib/frr/mgmtd -d -F traditional -A 127.0.0.1
└─14418 /usr/lib/frr/staticd -d -F traditional -A 127.0.0.1
Feb 13 11:29:05 hub watchfrr[14401]: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
Feb 13 11:29:05 hub frrinit.sh[14436]: [14436|watchfrr] done
Feb 13 11:29:05 hub staticd[14418]: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
Feb 13 11:29:06 hub frrinit.sh[14438]: [14438|staticd] done
Feb 13 11:29:06 hub watchfrr[14401]: [QDG3Y-BY5TN] zebra state -> up : connect succeeded
Feb 13 11:29:06 hub watchfrr[14401]: [QDG3Y-BY5TN] mgmtd state -> up : connect succeeded
Feb 13 11:29:06 hub watchfrr[14401]: [QDG3Y-BY5TN] staticd state -> up : connect succeeded
Feb 13 11:29:06 hub watchfrr[14401]: [KWE5Q-QNGFC] all daemons up, doing startup-complete notify
Feb 13 11:29:06 hub frrinit.sh[14391]: * Started watchfrr
Feb 13 11:29:06 hub systemd[1]: Started frr.service - FRRouting.
root@hub:/home/ugu5ma#Let’s enable BGPd with vi /etc/frr/daemons
bgpd=yesRestart the daemon with with systemctl restart frr.service
With enabled BGPd FRR uses minimal resources:
Let’s access the virtual-console of the Hub with sudo vtysh and setup the virtual-router. We also log all configuration commands entered via the vtysh shell:
Hub
root@hub:/home/ugu5ma# vtysh
Hello, this is FRRouting (version 10.2.1).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
hub# conf t
hub(config)# log commands
hub(config)# router bgp 65000
hub(config)# bgp router-id 10.5.5.1
hub(config)# no bgp ebgp-requires-policy
hub(config-router)# neighbor 10.5.5.10 remote-as 65010
hub(config-router)# neighbor 10.5.5.10 description Spoke1
hub(config-router)# neighbor 10.5.5.20 remote-as 65020
hub(config-router)# neighbor 10.5.5.20 description Spoke2
hub(config-router)# exit
hub(config)# exit
hub# wr t
Building configuration...
hub# show running-config
Building configuration...
Current configuration:
!
frr version 10.2.1
frr defaults traditional
hostname cipv6lts
log syslog informational
no ipv6 forwarding
service integrated-vtysh-config
!
router bgp 65000
bgp router-id 10.5.5.1
no bgp ebgp-requires-policy
neighbor 10.5.5.10 remote-as 65010
neighbor 10.5.5.10 description Spoke2
neighbor 10.5.5.20 remote-as 65020
neighbor 10.5.5.20 description Spoke1
!
address-family ipv4 unicast
network 10.0.0.0/8
network 10.5.7.1/32
exit-address-family
exit
!
!
end
hub# exitSpoke #2
root@spoke2:/home/ugu5ma# vtysh
Hello, this is FRRouting (version 10.2.1).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
spoke2# conf t
spoke2(config)# log commands
spoke2(config)# router bgp 65010
spoke2(config)# bgp router-id 10.5.5.10
spoke2(config)# no bgp ebgp-requires-policy
spoke2(config-router)# neighbor 10.5.5.1 remote-as 65000
spoke2(config-router)# exit
spoke2(config)# exit
spoke2# wr t
Building configuration...
spoke1# exitSpoke #1
root@spoke1:/home/ugu5ma# vtysh
Hello, this is FRRouting (version 10.2.1).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
spoke1# conf t
spoke1(config)# log commands
spoke1(config)# router bgp 65020
spoke1(config)# bgp router-id 10.5.5.20
spoke1(config)# no bgp ebgp-requires-policy
spoke1(config-router)# neighbor 10.5.5.1 remote-as 65000
spoke1(config-router)# exit
spoke1(config)# exit
spoke1# wr t
Building configuration...
spoke2# exitLet’s see if Spoke#1 can see the Hub as a BGP neighbor:
spoke1# show ip bgp summary
IPv4 Unicast Summary:
BGP router identifier 10.5.5.20, local AS number 65020 VRF default vrf-id 0
BGP table version 0
RIB entries 0, using 0 bytes of memory
Peers 1, using 24 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc
10.5.5.1 4 65000 0 0 0 0 0 never Active 0 N/A
Total number of neighbors 1
spoke1#The Lab seems to be in a pretty good shape 🙂
Go ahead and try to establish a connection with Spoke#2!
Let’s announce a BGP-Route
On the HUB, we will announce a BGP route (10.5.7.1/32) for testing.
To do this, we will create a dummy interface and assign an IPv4 address.
FRR will then announce this network via BGP to the peers (Spoke#1 and Spoke#2).
Finally, we will verify if we are advertising the route to Spoke#1.
ip link add dummy0 type dummy
ip addr add 10.5.7.1/32 dev dummy0
ip link set dummy0 up
vtysh
show ip bgp neighbors 10.5.5.20 advertised-routes
BGP table version is 1, local router ID is 10.5.5.1, vrf id 0
Default local pref 100, local AS 65000
Status codes: s suppressed, d damped, h history, u unsorted, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
*> 10.5.7.1/32 0.0.0.0 0 32768 i
Total number of prefixes 1
Ok, let’s see if we receive route 10.5.7.1/32 on Spoke#1 and check connectivity:
spoke1## show ip route bgp
Codes: K - kernel route, C - connected, L - local, S - static,
R - RIP, O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
f - OpenFabric, t - Table-Direct,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
B>* 10.5.7.1/32 [20/0] via 10.5.5.1, wg0, weight 1, 00:00:18
spoke1# ping 10.5.7.1
PING 10.5.7.1 (10.5.7.1) 56(84) bytes of data.
64 bytes from 10.5.7.1: icmp_seq=1 ttl=64 time=17.0 msGood! That’s it so far.
We have established a highly secure and scalable network topology across the internet. By leveraging WireGuard for routing transmission and communication, we ensure that this network topology remains exceptionally secure.
You must be logged in to post a comment.