How the SPEED-IX is technically working
The SPEED-IX (AS41441) is supported by its Partners and Members. Currently the traffic exchange point infrastructure SPEED-IX is currently distributed on five sites:
- Serverius at SDC1, Dronten, NL
- Atom86 at Schuberg Philis, Schiphol-Rijk, NL
- Serverius at Nikhef, Amsterdam, NL
- Serverius at SDC2, Meppel, NL
- Serverius at GlobalSwitch, Amsterdam, NL
- Serverius at TCN, Groningen, NL
- The Datacenter Group Amsterdam, NL
The whole network infrastructure is based on the newest Huawei CloudEngine hardware with Intelligent Stack technology and owned CWDM/DWDM equipment which is running on fully redundant fiber channels between all most points of presence.
The available connectivity standards for Member connections are:
- 1 / 10Gb SFP / SFP + (SR / LR / ER / ZR);
- 100/1.000/10.000 Mb (10GBaseT) RJ45;
- 40Gb QSFP + with SR4 / LR4;
- 100Gb QSFP28 + with SR4 / LR4;
Redundant route servers
For the exchange of IP routes between all members of SPEED-IX, we have built two independent Route Server (RS) for the BGPv4 protocol. The RS also support filtering of IP prefixes based on Internet Routing Registry (IRR) policies and other attributes of the BGPv4 Protocol (AS_PATH, Next-hop, etc.). Two route servers are configured on dedicated servers with a Linux-based operating system. To reduce the risks related to bugs in the software we use two different well-known software platforms for BGP routing such as Bird and Quagga
Advanced member panel
The SPEED-IX Member panel is based on the open-source project IXP Manager. Statistics on a members traffic and information on connectivity with other members are collected here. Traffic statistics are grouped by 12 hours, days, weeks and months. A member can analyze not only the bit rate, but also the packet rate, errors, and the number of discarded packets. The “Ports” tab contains information about the parameters of the member’s connection to SPEED-IX. Members can also find detailed information about their own Advertised/Accepted/Not Advertised IP prefixes and a useful Looking Glass tool here.
The infrastructure of the traffic exchange point brings members together into a single broadcast domain (L2-domain), so the risk of broadcast storms with BUM traffic (broadcast, unknown destination address, multicast) is available. At least, a storm can reduce the bandwidth of members channels. In the worst scenario, the connection with the route servers will be lost, BGP sessions will be terminated and the IX infrastructure will be disconnected. To prevent SPEED-IX from being stuck, we use multi-level protection against BUM traffic, limiting the traffic as follows:
- The prohibition on reception on the ports of members all multicast packets, except for protocols and specific types of messages, ensuring the correct operation of network services (LACP, ICMPv6 NS, ICMPv6 NA).
- Restriction on the transmission of broadcast packets (broadcast storm-control / broadcast rate-limit).
- Filtering the ether-type field. It is usually allowed to transfer frames that carry IPv4, IPv6, and ARP.
- Ensuring the reliability of information in ARP-messages (ARP inspection). The member only responds to ARP requests regarding his IP address on a specific interface.
On the third and fourth levels of the OSI-model, we do the filtering of dynamic routing protocols except BGP, and other protocols that carry the threat of users and IX infrastructure itself. We analyze BGP routing information as the prefixes themselves and a set of attributes for the given prefix (community, AS_PATH, Next-hop, etc.).
New users connections
For new IX users there exists standard settings and rules:
- On the port side of the member in SPEED-IX, STP, IP redirects, LLDP, CDP, ARP proxy, and other link-local protocols must be disabled, except ARP and IPv6 ND.
- Allowed the announcement of the Ethernet-frames: 0x0800 – IPv4, 0x0806 – ARP, 0x86dd – IPv6.
- One port – one MAC-address of the member.
- Prohibited announcement of Default-route and BGP full view.
The connection procedure itself is built in a way so that we are able to double-check the accuracy of these settings. First, a new member is connected to a port located in a quarantine VLAN. We analyze the traffic, and if everything is configured correctly, the port is moved to the production VLAN. When a new member is still isolated from the rest, it’s prefixes are not advertised to other members in the IX, and the new members will also receive nothing. If everything is normal, then the session will be translated into production mode.
Connectivity to SPEED-IX
Members can use the following connection options on their SPEED-IX port(s):
Free 10G Shared Peering– Exchange of traffic with all the IX members through route servers.
Free 10G Private Peering– Traffic exchange with individual IX members. In this case, members can make direct BGP-sessions and the traffic flow is not organized through the route server (RS). This method of connecting to the point of traffic exchange is useful when you need to improve connectivity to one or more specific participants in the IX.
Free 1G Private VLAN– Establishing communication through a dedicated VLAN between two or more Members or Partners (to get the free 1G VLAN both Members should use the route servers for bgp peering and do at least 50Mbps of BGP peering traffic in total to at least 10 other peers). This VLAN can be used for private purposes like using additional services from other Members and Partners (VLAN upgrades can be purchased at Partners).
Based on the Internet exchange point, all members can offer users different kinds of additional services, such as:
- Low-latency IP transit service with BGP FlowSpec extensions
- Protected IP transit service with DDoS protection, IP Protection Cloud and Web Application Firewall protection WAF).
- Connection to other exchange points like NL-IX, AMS-IX.