]> ETH Zurich

jelte.vanbommel@inf.ethz.ch

ETH Zurich

wirzf@inf.ethz.ch

ETH Zurich

tilmann.zaeschke@inf.ethz.ch

IRTF PANRG SCION QUIC multipath This document provides informational guidance for using the Multipath Extension for QUIC with the SCION networking technology. SCION is an inter-domain routing protocol that supports path-aware multi-path networking. The multiple paths and their associated path information offered by SCION provide opportunities as well as challenges for combining QUIC-MP with SCION. This document explores various aspects of this combination, such as algorithms for congestion control, RTT estimation, and general application scenarios. In addition, it provides techniques and guidance to maintain the security of QUIC-MP and SCION, and to leverage path-aware multi-path networking with QUIC-MP. About This Document The latest revision of this draft can be found at . Status information for this document may be found at . Discussion of this document takes place on the WG Working Group mailing list (), which is archived at . Subscribe at . Source for this draft and an issue tracker can be found at .

Introduction The Multipath Extension for QUIC , is an extension for the QUIC protocol that enables simultaneous usage of multiple paths for a single QUIC connection. SCION (, ) is an inter-domain routing protocol that offers explicit path selection between two endpoints, typically from a large selection of paths, where paths have detailed information on traversed autonomous systems (ASes), links, router interfaces, and other information. Despite their complementary goals, QUIC-MP and PANs have evolved largely in isolation. QUIC-MP has been designed with traditional IP-based routing in mind, where path changes are typically inferred from endpoint address changes (i.e., 4-tuples), and where routing is opaque to the transport layer. In contrast, Path-Aware Networks (PANs), such as SCION, enable informed path selection based on performance, disjointness, policy, or security requirements. This combination of QUIC-MP and SCION allows for optimizations, for example, for congestion control, RTT estimation, failure recovery, performance, and security. However, the slightly different assumptions on endpoint addresses (4-tuple + path ID vs 4-tuple + AS code + path) and path lifecycles (path abandon vs expiry, etc.) can cause some pitfalls. The purpose of this document is to explore how QUIC-MP and SCION can interoperate, how we can leverage the path awareness offered by SCION, and suggestions on how to overcome challenges. This document lists notable points when using QUIC-MP over SCION (), looks at different usage scenarios (), gives implementations considerations () for library developers of SCION and QUIC-MP, and discusses general security considerations (). While we provide guidelines for these areas, we do not discuss concrete algorithms, APIs, QUIC-MP or SCION implementations, or QUIC-MP user applications; these are considered out of scope. Some considerations are independent of multipathing and may be directly applicable for using over SCION.

Terminology and Conventions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Terminology We assume that the reader is familiar with the terminology used in and . We also draw on the terminology of and of SCION ( and ). For ease of reference, we have included some definitions here, but refer the reader to the references above for complete specifications of the relevant terminology. Autonomous System (AS): An autonomous system is a network under common administrative control. For example, the network of an Internet service provider or organization can constitute an AS. Endpoint: An endpoint is the start or end of a path, as defined in . Inter-AS Link: A direct link between two external interfaces of two ASes. Intra-AS Link: A direct link between two internal interfaces of a single AS. A direct link may contain several internal hops. Link: General term that refers to "inter-AS links" and "intra-AS links". Network Address: In traditional IP networks, this refers to the tuple of address and port of an endpoint. In SCION, the network address consists of an ISD-AS identifier and a host address, typically expressed as [ISD-AS, Host]. Network Path: The network path is the sequence of network elements between the two endpoints (e.g., ASes, interfaces, internal links). In traditional IP networks, the network path is typically opaque to the endpoints. QUIC-MP Path: Consists of the network address at each endpoint and a Path ID (see ). The Path ID allows having multiple logical paths for the same set of network addresses. Path Metadata: Path metadata is additional data that is available to endpoints when they request a selection of paths to a destination. Path metadata is authenticated by the owner of each link, but is otherwise not verified. This data describes properties of traversed ASes and links, such as their identity or MTU. Metadata Extension: SCION offers additional path metadata via extensions. The metadata includes properties such as bandwidth and latency. The extension is widely supported but not further discussed here as it is not specified in or .

Overview of QUIC Multipath in Path-Aware Networks The Multipath Extension for QUIC (QUIC-MP) is primarily designed for traditional IP networks, where each path is identified by a 4-tuple of local and remote IP addresses and ports. Routing is considered opaque to endpoints, and path changes are inferred indirectly through changes in the 4-tuple. However, the path between two endpoints may change unpredictably due to routing dynamics, which is not captured by the 4-tuple. In SCION, endpoints can discover and select explicit network paths, which are described at the level of ASes and inter-AS links, and have associated metadata, such as the MTU. The different underlying assumptions of QUIC-MP and SCION result in some mismatches, for instance:

• Endpoint Ambiguity: PANs such as SCION use composite addresses (e.g., ISD-AS + Host), where the host can be an IP address from a private IP range. This breaks the assumption that IP addresses are unique in the current network.
• Path Identity Mismatch: In QUIC-MP, paths are distinguished by 4-tuples and Path IDs. In SCION, two distinct physical paths may share the same 4-tuple, rendering transport-layer path tracking incomplete.
• Routing and Path Lifecycle: Paths in SCION can expire, be revoked, or be reissued, even when 4-tuple information is unchanged. This can interfere with RTT estimation, congestion control, and path validation logic if not properly accounted for.

However, SCION also opens up opportunities for enhancing multipath transport:

• Explicit path selection enables endpoints to choose disjoint paths, i.e., paths that do not share links or segments, to improve fault tolerance against link failures, or to increase aggregate throughput.
• Path metadata allows endpoints to prioritize paths with more suitable properties. For instance, low-latency and low-hop-count paths can be prioritized for RTT measurements, avoiding wasting resources on probing paths with poorer characteristics. Or, path overlap or disjointness can easily be determined and used for congestion control.
• Path-awareness allows congestion control and RTT estimators to reset only when the underlying network path has actually changed, something not reliably detectable in traditional IP networks, where network path changes may occur due to routing, despite a constant 4-tuple.
• The availability of stable network paths in PANs allows the transport layer to distinguish between actual path changes and transient network conditions, enabling more accurate RTT estimation and congestion control.

Multipath Use Cases and Categorization This section categorizes common use cases for multipath transport and highlights how PANs enhance each scenario. Many of these use cases can be combined to meet complex application requirements.

Fault Tolerance and Availability Multipath transport can improve robustness against network failures in several ways:

• An endpoint can immediately send data over a predetermined backup path if it suspects or obtains a network notification that a primary path is faulty.
• In SCION, the network may send error messages that explicitly specify the faulty link. This allows selecting a backup path that circumvents the faulty link.
• By sending redundant traffic over multiple paths, an application can maintain continuity even if one path becomes unavailable.

The use of backup paths or paths for redundant sending can be further improved by analyzing paths for overlap and selecting disjoint paths. This reduces the likelihood of multiple paths failing simultaneously.

High Throughput An application may aim to maximize the available bandwidth by spreading traffic across multiple paths. To optimize this, an application may:

• Select multiple paths with maximum disjointness.
• Select multiple paths such that disjointness is limited to sub-paths that are expected to have high bandwidth available.
• When congestion is detected, switch over entirely, or shift parts of traffic to an underutilized disjoint path to preserve the throughput.

Low Latency Latency-sensitive applications benefit from selecting the fastest available path at any moment. In PANs, endpoints may estimate latency from explicit metadata or infer it from probing. Because in PANs paths are stable and explicitly selectable, the transport layer can maintain multiple low-latency options in parallel, and either transmit in parallel, or switch traffic to a different path once the latency starts to fluctuate. Latency may be determined by RTT estimation, see . For deadline sensitive applications, an algorithm as described in may be useful. Instead of probing many paths at once, an implementation should probe only the most promising paths (based on the metadata). Probing many paths should also be avoided to avoid overloading individual links, and it may effectively be limited (except traceroute) by the available path IDs and connection IDs, see .

Policy-Driven Routing and Routing Constraints Some applications or deployments may wish to avoid routing traffic through certain ASes, e.g. to ensure path diversity or to enforce routing policies. SCION enables this by making path selection explicit and verifiable.

Anonymity and Traffic Obfuscation Multipath transport can be used to reduce the observability of traffic by distributing it across multiple network paths. In PANs, endpoints can explicitly select disjoint paths to minimize the risk that a single AS observes the full traffic flow. Randomizing path selection and packet scheduling can help obscure traffic patterns. However, traffic characteristics such as packet timing, flow duration, or volume may still be linkable across paths. Mechanisms such as probing should therefore be designed and used such that it avoids creating identifiable patterns.

Gateways and Proxies There are gateways and proxies (including VPN) that translate SCION traffic to IP traffic and back. These are a special case because they are not used together with a QUIC(-MP) implementation, instead they are, and should be, oblivious to QUIC traffic. TODO These, along with NATs, will be discussed in a future version of this document.

Technical Considerations Using QUIC or QUIC-MP over a PAN, such as SCION, changes some of the underlying assumptions. This provides certain benefits, such as additional information and control over paths, but also some challenges.

Addressing SCION uses composite addresses (ISD/AS + IP + port), where the IP address can be from a private IP range. This breaks the assumption that IP addresses are unique in the scope of the network.

Key Implications QUIC-MP relies on the 4-tuple changes to trigger path validation. However, with SCION, the 4-tuple does not uniquely identify an endpoint. Two endpoints with identical IP/port could be in different ASes. An attacker could use endpoints with identical 4-tuple to reroute traffic to a different machine without triggering path validation, see and . The implication of a skipped path validation is that a server can be coaxed into sending QUIC packets to an unexpecting client. This is similar to the attacks described in and .

Recommendations

• To prevent attackers circumventing QUIC path validation, a QUIC-MP implementation MUST make sure that QUIC path validation is triggered when the network address of the destination changes; this includes IP, port and AS number. This protects against several attacks, see and especially .

There are several ways to achieve this, for example:

• Adapt the QUIC-MP library to be aware of the AS number in SCION network addresses.
• If the network address is available as a single "object", the SCION layer can extend this with the AS code and the QUIC-MP implementation must only ensure to compare the whole object instead of port and IP separately.
• The SCION implementation could detect cases where only the AS changes and then mangle the port or IP to trigger a path validation in the QUIC-MP layer. This may be a pragmatic solution but is discouraged because:
  • Managing paths in the SCION layer is not trivial because it requires synchronizing the lifecycle of SCION paths and QUIC-MP paths, e.g., knowing when is a path valid or when is it closed in QUIC-MP.
  • It creates opportunities for memory exhaustion attacks (for storing the mapping of mangled IP/port).
  • It reports a wrong IP/port to the application.

Interoperability of QUIC-MP Path ID and Network Paths The identification of "paths" varies between QUIC, QUIC-MP and SCION.

• uses a 4-tuple of local/remote IP/port to distinguish paths.
• extends the 4-tuple with a path ID to distinguish logical paths (connections).
• SCION can distinguish paths based on the physical inter-domain network path and additional properties, such as an expiration time (the latter may or may not be used to distinguish path instances).

A path change occurs when at least one of the router interfaces changes. The network address may stay the same. With NAT rebinding, as described in , the path can change, but not without changing the SCION network address (IP, port, AS), so this case is not a concern. Path change detection is required to trigger certain actions, such as resetting congestion control or RTT estimation algorithms. See also and . When using a PAN such as SCION, it is important to trigger these actions even if the full network address (4-tuple + AS) stays the same. Alternatively, the system can be implemented in a way so that uncontrolled path changes cannot occur. This is possible because path changes can only be initiated by endpoints. However, this has some limitation if one of the endpoints is not aware of transporting QUIC, for example a SCION gateway or proxy, see . Implementations should try to maintain a 1:1 mapping between QUIC-MP path IDs and SCION network paths. However, this is not always possible or useful:

• A SCION network path may expire. Replacing a path with an identical new path (except for the expiration date), should be allowed without triggering algorithm reset. Alternatively, refresh can be handled by the path selector, see .
• A SCION implementation, when used with QUIC-MP should be configured such that every SCION network path is used for exactly one QUIC-MP path ID. However, it may not always be possible or feasible to configure SCION implementations in this way, for example. when they are part of a SCION gateway or proxy, and are unaware of transporting QUIC, see .
• A SCION path should be allowed to be reused, e.g., they may be assigned to one QUIC path ID, and when that path ID is closed, the path must be allowed to be assigned to another path ID. This should not cause any problems except for the marginal complexity of managing the associated state with a path ID.

Key Implications If a path change occurs undetected, the QUIC-MP layer cannot reset congestion control () or RTT estimation (). This is undesirable but not worse than traditional IP based non-PAN transport where routes can change without the endpoints learning about it.

Recommendations

• Congestion control and RTT estimation algorithms should be designed to gracefully handle path changes that don't trigger a reset, unless it can be guaranteed that both SCION endpoints are configured to prevent automatic path changes.
• Within a QUIC-MP session, every SCION network path should be used only with one path ID. However, it may be reused if the path was abandoned or closed.
• Changes of the network path (while the network address stays the same), except for expired paths being renewed, should trigger algorithm reset (CC, RTT estimate), see .

Analogous to , except for replacing "AS" with "network path". We list them here again because the implications of not following the recommendation are much weaker and may be considered acceptable. Recommendations:

• Adapt the QUIC-MP library to be aware of the full network path, including router interfaces.
• If the network address is available as a single "object", the SCION layer can extend this with the network path (possibly excluding the expiration date), and the QUIC-MP implementation must only ensure to compare the whole object instead of port and IP separately.
• The SCION implementation could detect cases where only the router interfaces change and then mangle the port or IP to trigger a path validation in the QUIC-MP layer. This may be a pragmatic solution but is discouraged, because:
  • Managing paths in the SCION layer is not trivial because it requires synchronizing the lifecycle of SCION paths and QUIC-MP paths, e.g., knowing when is a path valid or when is it closed in QUIC-MP.
  • It creates opportunities for memory exhaustion attacks (for storing the mapping of mangled IP/port).
  • It reports a wrong IP/port to the application.

Initial Handshake requires that there is no connection migration during the initial handshake, and that there are no other packets sent (including probing packets) during the initial handshake, see , paragraphs 2 and 3.

Key Implications Changing the path during handshake would violate :

• The design of QUIC relies on endpoints retaining a stable address for the duration of the handshake. An endpoint MUST NOT initiate connection migration before the handshake is confirmed, as defined in .

Recommendations

• A SCION implementation should not automatically change network paths switch without explicit request by the QUIC(-MP) layer. The only exception allowed is replacing an expiring path with a new path that is identical except for the expiration time. We also need to ensure this for gateways etc., see .

Congestion Control Following , the CC algorithm should be reset when the 4-tuple of a QUIC path changes. With SCION, 4-tuples are not sufficient to identify paths, see . To avoid missing a path change, the SCION implementation should never change a network path unless explicitly instructed by the QUIC-MP implementation, see .

Coupled Congestion Control mentions coupled congestion control algorithms, such as . states:

• One of the prominent problems is that running existing algorithms such as standard TCP independently on each path would give the multipath flow more than its fair share at a bottleneck link traversed by more than one of its subflows.

This can be avoided in PANs, such as SCION, through link-level analysis of paths and selecting paths that do not share a bottleneck link. Instead, this bottleneck knowledge can be used to effectively use separate congestion control for each path. Alternatively, a CC algorithm could be employed that focuses on known shared links (which may be bottlenecks). There are several congestion control algorithms proposed in literature, e.g., LIA, OLIA, BALIA and RSF. These combine congestion control with path selection algorithms. For simplicity, we suggest separating concerns in terms of congestion control and path selection. This allows us to better tailor the solutions to the different usage scenarios. The proposition is to use non-coupled congestion control per path, tailored for each use case in , and use separate independent path selection algorithms. CC algorithms can also benefit from the SCION metadata extension that provides bandwidth and latency data for each link on a network path.

Key Implications A network path change goes unnoticed in case a SCION implementation changes a path that happens to have the same IP/port for both endpoints. Congestion control (CC) algorithms can also benefit from exact knowledge of a path:

• When using multiple paths, a CC algorithm can access path metadata as to if and where the paths overlap and some of the properties of the overlapping sections.
• CC algorithms should be notified of every path change, allowing them to reset only when necessary. A reset may not be necessary if the network path remains the same and only the IP or port of an endpoint changes. This can make sense if any congestion is assumed to be on the network path rather than behind the remote IP/port (e.g., behind a proxy).

See also .

Recommendations

• Congestion control algorithms should be reset when the network path changes (beyond 4-tuple). This is best achieved by ensuring that the network path changes only in conjunction with QUIC path migration events.

Congestion control algorithms can also benefit from exact knowledge of a network path:

• Congestion control algorithms should use the path metadata to detect and, if necessary and possible, avoid overlap with other paths. Congestion control can then be simplified to work independently for each path.
• Path selection algorithms should try to avoid multiple path that share bottleneck links.

RTT Estimation Similarly to congestion control, and following , the RTT estimation algorithm should be reset when the 4-tuple of a QUIC path changes. As described in this can be avoided by forbidding SCION implementations to change a network path unless instructed otherwise by the QUIC-MP implementation.

Key Implications If a path change occurs undetected, the QUIC-MP layer may fail to reset RTT estimation. This is undesirable but not worse than traditional IP based non-PAN transport where routes can change without the endpoints ever learning about it.

Recommendations

• RTT-algorithms should be reset when the network path changes (beyond 4-tuple). This is best achieved by ensuring that the network path changes only when requested by QUIC.

Round-trip time estimation algorithms can also benefit from exact knowledge of a path:

• An implementation may use SCION's SCMP traceroute () to measure the latency of individual links and then use this information to select new network paths that favor low latency links and avoid high latency links. See also .
• An implementation could use the SCION metadata extension to get propagation latency information of links in a path without having to measure it. This latency does not include queueing latency but may in many cases be sufficient for practical use.

MTU Discovery The MTU may be used to calculate the available payload size. SCION inserts an additional header () into each packet. The header size depends on the IP family (e.g., IPv4 vs IPv6 addresses) and on the "length" of the path, i.e., the number of ASes that are traversed. This must be taken into account when calculating the available payload size. The difference between typical MTU (1500 bytes) and QUIC's required packet size (1200 bytes) is sufficient for typical real-world SCION headers. PMTU discovery (, ) can be used to discover or verify MTU sizes. However, the MTU from the path metadata can (at least on the client side) be used to preselect paths with desirable MTU values. In SCION, when an endpoint requests a network path, it will be provided with MTU information for every hop on a path, see also and . However, in SCION, paths are typically only requested by client endpoints, not by server endpoints. There are several ways for a server to determine the MTU. If a server wants to know the MTU, it may:

• Try to determine the MTU from the size of incoming packets.
• Use an algorithm to determine the MTU, see Path MTU Discovery in and .
• Try to look up the path to the client endpoint that is identical to the incoming path. However, this requires time and effort on the server side, and there is no guarantee that the incoming path is available in the local AS.

Also, note that the MTU information is authenticated but not verified, it may be incorrect due to misconfiguration or malicious ASes.

Key Implications and Recommendations PMTU discovery for multi-path may be improved by using path metadata. PMTU will be explored more in detail in a future version of this document (TODO)). It may be possible for a client to send the PMTU size directly to a server, for example as a parameter via the QUIC Transport Parameter Extension, see and .

Retransmission & PTO See and . For retransmission, a SCION client or server may compare available paths and choose one or more paths that have minimum overlap with the current (unreliable) path.

Paths Having Different PMTU Sizes suggests determining a single MTU size in order to simplify retransmission.

Key Implications explains that the benefit of using a single MTU size is to simplify retransmission processing, as the content of lost packets initially sent on one path can be sent on another path without further frame scheduling adaptations.

Recommendations

• On the client, this can be facilitated by computing a viable minimum MTU size from all available network paths. However, it must be considered that these values are not verified.
• On the server, MTU values from path metadata are not available. The server may request these from the local AS, but the exact path may not be available (in SCION, different ASes may offer different sets of paths to their customers). Also, except for initially proposing a preferred address (), new paths must be opened by the client, not the server, see .

Path Selection The path selection component is responsible for requesting paths to a destination, ordering the path based on policy and preferences, using them when new QUIC-paths are opened, and retiring them or listing them for reuse when they are closed.

Dynamic Approach A dynamic approach could start with using low latency paths. If the connection appears to be long lasting, it could start (and keep) adding additional paths as long as the traffic increases. As an example, if the algorithm detects traffic that lasts for at least one second and transfers at least 100MB of traffic, the algorithm could trigger creation of additional QUIC paths.

Bottleneck Detection If live traffic information is not available, bottleneck detection can help to identify links that should be avoided. In PANs, this can be achieved using approaches such as . An alternative is to use SCION's SCMP traceroute command () to measure the latency between two consecutive AS border routers. The measured latency can be compared to earlier measurements or to the latency given in the path metadata. Discrepancies can be an indication of high traffic volume and queueing on the measured link. While traceroute may be useful, it should be used with care:

• traceroute traffic is not congestion controlled.
• It is clearly distinguishable from QUIC traffic, so it may affect anonymity.

Key Implications Path selection is a key feature of SCION and PANs in general. For more details, see and .

Recommendations In order to manage paths effectively, the path selection algorithm probably requires access to the following fields and events:

• initial_max_path_id ()
• MAX_PATH_ID frames (
• PATH_AVAILABLE and PATH_BACKUP, see ,
• PATH_ABANDON .

Moreover, path selection must exclude paths whose MTU is too small to guarantee 1200 bytes MTU payload for QUIC packets. The effective MTU also depends on the length of the paths.

Packet Scheduling Packet scheduling helps to distribute the transfer load efficiently over multiple paths, see also .

Key Implications SCION paths are stable, at least on the inter-AS level, i.e., they cannot change without initiative from the endpoints.

Recommendations Path stability may simplify packet scheduling algorithms because the performance of individual QUIC-paths is more reliable if they cannot unexpectedly be rerouted.

Address Validation Token specifies that a server is expected to send a new address validation token to a client following the successful validation of a new client address. states:

• "Tokens sent in Retry packets SHOULD include information that allows the server to verify that the source IP address and port in client packets remain constant."
• "Tokens sent in NEW_TOKEN frames MUST include information that allows the server to verify that the client IP address has not changed from when the token was issued."

These must be read such that "IP address" becomes "network address", meaning that it includes the ISD/AS code of a peer.

Path ID Allocation SCION application may use many paths. This poses two problems: initial path ID allocation and incremental path ID allocation. See also and . When initializing a connection, servers and clients should be careful to set the maximum path ID high enough to allow as many paths as an application may need, e.g., through the initial_max_path_id transport parameter or the MAX_PATH_ID frame. Afterwards, on a bad connection, paths IDs may still run out earlier than expected, so servers should allocate new path IDs either automatically or on demand when receiving a PATHS_BLOCKED frame.

Key Implications SCION path IDs may run out quickly unless the server provides enough initial path IDs and provides enough path IDs once the initial contingent runs out.

Recommendations

• Ensure that servers and clients are prepared to agree on an initial number of path IDs that is high enough for the user application. This should be configurable.
• Ensure that either the server proactively allocates new path IDs (MAX_PATH_ID frame) when they are running out, or that the server allocates sufficiently many new path IDs once the client sends a PATHS_BLOCKED frame.
• Optional: Allow the client to send PATHS_BLOCKED proactively even before the path IDs are exhausted. Prepare the server to accept early PATHS_BLOCKED frames and respond by allocating a gracious amount of additional path IDs.

Address Spoofing Attackers may create and send packets that look like they originate from somewhere else. There are two main scenarios:

1. An attacker creates a packet with a path that seems to originate outside the local AS.
2. An attacker creates a packet that seems to originate from the local AS but the SCION SRC address does not match the attackers IP.

In both cases the packet may be sent a border router or a local endpoint.

Key Implications states: "If the packet path indicates an origin outside of the local AS, verify that the packet underlay source address matches a known border router in the local AS." For the 2nd scenario, and for the sending to local endpoints, there are no known attacks.

Recommendations There are no known attacks. However, ASes should try to avoid exporting faulty packets and endhosts SCION stacks should avoid forwarding faulty packets to the QUIC layer or application layer. The recommendation is that receivers (endhost SCION stacks, border routers, ...) should drop any packets:

1. Receivers should drop a packet if its path indicates origin outside the local AS but the underlay IP does not match a known border router IP.
2. Receivers should drop a packet if its path indicates origin inside the local AS but the underlay IP does not match the SCION SRC address.

Summary of Recommendations This memo is informational. However, we use imperative language here for recommendations that are relevant to security or performance.

Recommendations for QUIC-MP Implementations

• To prevent attackers circumventing path validation, a QUIC-MP implementation MUST ensure to trigger path validation when the network address of the destination changes; this includes IP, port and AS number. This protects against several attacks, see and especially . There are several ways to achieve this, for example:
  • Adapt the QUIC-MP library to be aware of the AS number in SCION network addresses.
  • If the network address is available as a single "object", the SCION layer can extend this with the AS code, and the QUIC-MP implementation must only ensure to compare the whole object instead of port and IP separately.
  • The SCION implementation could detect cases where only the AS changes and then mangle the port or IP to trigger a path validation in the QUIC-MP layer. This may be a pragmatic solution but is discouraged, because:
    • Managing paths in the SCION layer is not trivial because it requires synchronizing the lifecycle of SCION paths and QUIC-MP paths, e.g., knowing when is a path valid or when is it closed in QUIC-MP.
    • It creates opportunities for memory exhaustion attacks (for storing the mapping of mangled IP/port).
    • It reports a wrong IP/port to the application.
• A QUIC-MP implementations SHOULD be able to recognize network path changes beyond 4-tuple or AS changes. This enables resetting congestion control and RTT algorithms.
• A QUIC-MP implementation, especially congestion control, RTT estimation, and path selection algorithms should be careful to not accept or probe all available SCION network paths. There may be a large number of path available and probing or using them all may amount to a denial of service attack on the receiver.
• To prevent problems with path ID exhaustion with multi-path applications, see :
  • Servers and clients should initialize the connection with sufficiently many path IDs.
  • Servers should proactively allocate new path IDs when they run low.
  • Clients should proactively send PATHS_BLOCKED when path IDs run low, Servers should accept and react to PATHS_BLOCKED before path IDs are actually exhausted.

Recommendations for SCION Implementations

• A SCION implementation SHOULD NOT store or cache paths, especially not on the server side. This prevents memory exhaustion attacks, see {attack-memory-exhaustion}. This also avoids the problem of path lifecycle maintenance, i.e., determining which paths are still alive and which have been closed or abandoned. Sometimes, storing paths is inevitable, see . For security concerns, see also .
• If a SCION implementation stores paths internally, it MUST NOT use IP/port as key to look up paths. IP/port are not unique to identify endpoints.
• When used with QUIC-MP, a SCION implementation MUST NOT change the network paths, possibly with the exception of refreshing expired paths. When a path stops working, the implementation should instead report an error to the QUIC(-MP) layer or time out silently.
• Attackers may create packste with faulty spoofed addresses or paths. While there is no known associated attack, border routers should not forward faulty packets to other ASes and endhost SCION libraries should not forward faulty packets to the QUIC layer or applications. To prevent this, packets receivers, such as border routers or SCION endhost libraries, should drop faulty packets.
  1. Receivers MAY drop a packet if its path indicates origin outside the local AS but the underlay IP does not match a known border router IP.
  2. Receivers MAY drop a packet if its path indicates origin inside the local AS but the underlay IP does not match the SCION SRC address.
  NB: 1. The check for packets that seeminlgy originate outside the local AS is already specified for border routers in .

Recommendations For Both QUIC-MP And SCION Implementations

• A server should return packets on the same path on which they were received.
  • Generally, a server SHOULD respond on the same path on which the data was originally requested, unless the new path has been validated. This ensures that the path does not violate the path policy of the client.
  • A server SHOULD NOT create new paths. This is also stated in . Servers may communicate a preferred address after the initial handshake. However, it is recommended to avoid that because any new path may violate a client's path policy.
  • Returning probing packets on the same network path on which they were received: This greatly simplifies RTT estimation, see .
• Clients may replace expired or soon-to-expire paths with identical paths without performing path migration / validation.
• Within a QUIC-MP session, every SCION network path should be used only with one path ID. However, it may be reused if the path was abandoned or closed by QUIC. This is the responsibility of the path selection algorithm, regardless of whether it is considered part of SCION or part of QUIC-MP.

Other Recommendations

• Endpoints should aim to be located in local networks (local ASes) that have protection against IP spoofing from machines in the same network, see .

Security Considerations The aim is that QUIC-MP over SCION retains all security properties of QUIC-MP and SCION. However, this requires some implementation changes and additional consideration regarding:

• endpoint identity: a 4-tuple is not sufficient to identify an endhost;
• network path authenticity: paths may be forged by malicious clients;
• path probing patterns may expose user intentions or identity.

IP Ambiguity In SCION, ASes are free to use IP addresses from "private" IP ranges. THerfore, IP addresses do not uniquely identify a peer endpoint. To attack a client with IP address X, an attacker could set up an endpoint with identical IP X in a different AS. The attacker can then contact a server endpoint that is also used by the client victim. If the server endpoint stores paths internally with IP addresses as keys, then this would result in a key collision, which can cause two types of problems:

• If the attacker contacts the server after the client, the contact may result in overwriting the existing entry that points to the victim.
• If overwriting is prevented, the attacker may contact the server before the victim, thus potentially preventing the victim from establishing a connection.

The second case is more difficult to achieve. Often, connections may be stored by IP+port, so the attacker must guess the victims port when launching the attack, this is difficult with ephemeral ports.

Mitigation

• A SCION implementation should avoid storing paths while using the IP address as key to look up paths. For QUIC-MP, the QUIC Path ID should be used as key.
• SCION libraries could use IP/port mangling when they detect multiple paths with the same IP/port. However, this may have unintended consequences in the application layer.
• Higher level libraries, such as QUIC(-MP) should be careful to not rely only on IP addresses to trigger path validation or resetting congestion control or RTT estimation algorithms. Instead, QUIC-MP should rely on the QUIC Path ID.

Circumventing path validation is not possible, because that would require injecting a new path, see .

Path Injection There are several potential attacks that build on injecting valid or invalid paths into the server-side software stack. In summary, these attacks can be prevented by the recommendations listed in , specifically we recommend the following where possible:

1. SCION layers should avoid storing/caching paths and network addresses (beyond IP/port) internally. Instead, they should be given to the QUIC(-MP) layer or the application layer. That means that path information would only be accepted and retained if the QUIC(-MP) or application layer decides to do so.
2. SCION layers and QUIC(-MP) layers should interface by using network addresses that include all information that identifies an endpoint, including, for example, AS code. Any change in a network address (including the AS code) should trigger path validation.

Examples of attacks include memory exhaustion attacks, traffic redirection attacks, and traffic amplification attacks.

Traffic Redirection to Different AS An attacker may craft a packet that appears to originate from the same IP/port, but is located in a different AS than an existing connection. If the server's SCION layer stores paths internally, and uses IP/port as key to look them up, then the new paths may replace the existing one, and outgoing traffic is redirected to the new paths and destination. Mitigation:

• The QUIC(-MP) layer MUST trigger path validation if the network address changes, and must consider every attribute of the address, not just IP and port.
• If a packet is rejected by the QUIC(-MP) layer, the SCION layer MUST NOT add it to any local state (including not replacing existing state). This can be achieved trivially by not having state in the SCION layer.

Traffic Redirection over Different Path An attacker may craft a path with a network address that is identical to an existing valid endpoint, but with a different path. The new route may be invalid (e.g., contain nonexistent links) or faulty (contain links that are broken or have high latency or drop rate). The new route may also work fine, but violate the client's path policy or be used for traffic analysis. The attacker injects the crafted path into the server, with the intention that the non-unique IP causes an existing path/connection mapping to be overwritten, and thus replace the victims path with the updated path.

Example of non-unique IPs

This attack requires either spoofing of the client's IP address (when the attacker is in the same AS as the client), spoofing of a border router's IP address (when the attacker is in the server's AS), or injection of a path (which requires control over an AS that is en-route between the client and server). Mitigation: From the recommendations: 1) SCION implementations SHOULD NOT store or cache paths, especially not on the server side. 2) SCION endhost implementations MUST not use IP/port as key to store paths or connections to peers. Both recommendations prevent the attack. Further mitigation comes from the recommendation that path validation MUST always be triggered when the network address or path changes, even if the 4-tuple stays the same.

Traffic Amplification An attacker may establish a connection with a server, request a large amount of data, and then inject a path that redirects to a victim that has the same IP/port, but in a different AS. If the server-side QUIC-MP does not trigger path validation (because IP/port are the same), then it may implicitly accept the new path and send the requested data to a victim. Since injecting a path from on an-path AS is hard (requires control of an AS), the attacker will need two hosts. 1) Host A1 is located in an on-path AS or the server AS. A1 uses the victims IP to initiate a handshake with the server. 2) Host A2 is located in the victims AS. A2 use the secret obtained by A1 to request data from the server. A2 must be in the victim's AS so it can use the same path as the victim. In the GET request, it sets the SCION SRC address to be the victim's IP. If IP spoofing is possible in AS #200 and if border routers and endpoints are on the same network in AS #200, then A2 is not required. Instead, A1 could send the GET request with a spoofed IP of the borderrouter that connects #200 to #300.

Example of traffic amplification attack. #100 is the server AS, #200 is the attacker A1's AS and #300 is the victim's AS. QUIC=Init,Hello,Cert,FIN <-- Path=[#100, #200] Attacker A2 (Request data) QUIC=FIN,GET IP Address=spoofed victim-IP SCION Address=#300,victim-IP Path=[#300, #200, #100] --> QUIC=Data <-- Path=[#100, #200, #300] Client: receives unwanted traffic ]]>

Mitigation: This attack requires spoofing of the victim's IP address. - A QUIC(-MP) library must consider all attributes (not just the 4-tuple) when checking for a change in the network address. This would then trigger path validation, and the attack can be averted. - If a QUIC(-MP) library cannot compare additional attributes (e.g., legacy library), the SCION layer (server side) may have an option to perform port mangling or IP mangling: when the SCION layer detects a new network address that differs only in the AS number from a previously seen address (IP/port are the same), then it should perform IP/port mangling, i.e., reporting a modified IP or port to the QUIC(-MP) layer. This new IP/port would trigger a path validation or algorithm reset where required. Caveats:

• Offering a mangled IP/port to the application may have implications for application correctness, such as displaying an unexpected IP/port.

Memory Exhaustion An attacker may flood a server with packets that each have a different source network address or path. If these are stored in the SCION layer, they may cause memory exhaustion. Mitigation: do not store state in the SCION layer, or implement a way to clean up state without affecting a valid connection. Specifically, the SCION layer must drop a path if the QUIC layer closes the path, abandons the path, or rejects a packet with a new path.

Number of Open Paths The number of open paths should be limited, see . This is important in the context of applications that may open many paths in parallel. Mitigation:

• Same as : endpoints need to limit the maximum number of paths and might consider additional measures to limit the number of concurrent path validation processes, e.g., by pacing them out or limiting the number of path initiation attempts over a certain time period.

Probe Fingerprinting An endpoint may probe multiple paths to determine the best path(s) for a given use case. One example of probing packets is packets that measure round-trip time (RTT). Probing packets may be detected if they are sent in bulk, to the same destination, in regular intervals, and all with slightly different paths attached. This can be used to fingerprint an endpoint or their intentions (applications may have unique intervals defined). This can be mitigated by varying and generally reducing the number of probing packets, and by sending probing packets not en-block but time-shifted.

Additional Points TODO: Complete this section in a future version of this document.

• Use multipathing for anonymity, see .
• See other attacks in ?

IANA Considerations This document has no IANA actions.

References Normative References This document specifies an alternative encapsulation of the Datagram Congestion Control Protocol (DCCP), referred to as DCCP-UDP. This encapsulation allows DCCP to be carried through the current generation of Network Address Translation (NAT) middleboxes without modification of those middleboxes. This document also updates the Session Description Protocol (SDP) information for DCCP defined in RFC 5762. [STANDARDS-TRACK] Hosts are often connected by multiple paths, but TCP restricts communications to a single path per transport connection. Resource usage within the network would be more efficient were these multiple paths able to be used concurrently. This should enhance user experience through improved resilience to network failure and higher throughput. This document outlines architectural guidelines for the development of a Multipath Transport Protocol, with references to how these architectural components come together in the development of a Multipath TCP (MPTCP). This document lists certain high-level design decisions that provide foundations for the design of the MPTCP protocol, based upon these architectural requirements. This document is not an Internet Standards Track specification; it is published for informational purposes. This document defines the core of the QUIC transport protocol. QUIC provides applications with flow-controlled streams for structured communication, low-latency connection establishment, and network path migration. QUIC includes security measures that ensure confidentiality, integrity, and availability in a range of deployment circumstances. Accompanying documents describe the integration of TLS for key negotiation, loss detection, and an exemplary congestion control algorithm. This document describes how Transport Layer Security (TLS) is used to secure QUIC. This document describes loss detection and congestion control mechanisms for QUIC. The goal of this document is to explain the need for congestion control in the Internet, and to discuss what constitutes correct congestion control. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. The User Datagram Protocol (UDP) provides a minimal message-passing transport that has no inherent congestion control mechanisms. This document provides guidelines on the use of UDP for the designers of applications, tunnels, and other protocols that use UDP. Congestion control guidelines are a primary focus, but the document also provides guidance on other topics, including message sizes, reliability, checksums, middlebox traversal, the use of Explicit Congestion Notification (ECN), Differentiated Services Code Points (DSCPs), and ports. Because congestion control is critical to the stable operation of the Internet, applications and other protocols that choose to use UDP as an Internet transport must employ mechanisms to prevent congestion collapse and to establish some degree of fairness with concurrent traffic. They may also need to implement additional mechanisms, depending on how they use UDP. Some guidance is also applicable to the design of other protocols (e.g., protocols layered directly on IP or via IP-based tunnels), especially when these protocols do not themselves provide congestion control. This document obsoletes RFC 5405 and adds guidelines for multicast UDP usage. This document specifies Datagram Packetization Layer Path MTU Discovery (DPLPMTUD). This is a robust method for Path MTU Discovery (PMTUD) for datagram Packetization Layers (PLs). It allows a PL, or a datagram application that uses a PL, to discover whether a network path can support the current size of datagram. This can be used to detect and reduce the message size when a sender encounters a packet black hole. It can also probe a network path to discover whether the maximum packet size can be increased. This provides functionality for datagram transports that is equivalent to the PLPMTUD specification for TCP, specified in RFC 4821, which it updates. It also updates the UDP Usage Guidelines to refer to this method for use with UDP datagrams and updates SCTP. The document provides implementation notes for incorporating Datagram PMTUD into IETF datagram transports or applications that use datagram transports. This specification updates RFC 4960, RFC 4821, RFC 6951, RFC 8085, and RFC 8261. RFC 5033 discusses the principles and guidelines for standardizing new congestion control algorithms. This document obsoletes RFC 5033 to reflect changes in the congestion control landscape by providing a framework for the development and assessment of congestion control mechanisms, promoting stability across diverse network paths. This document seeks to ensure that proposed congestion control algorithms operate efficiently and without harm when used in the global Internet. It emphasizes the need for comprehensive testing and validation to prevent adverse interactions with existing flows. In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. Informative References Otto-von-Guericke University Magdeburg Otto-von-Guericke University Magdeburg This document proposes the Deadline-aware Multipath Transport Protocol (DMTP) concept as an extension to the Multipath Extension for QUIC (QUIC-MULTIPATH) as well as QUIC itself. This extension aims to support data streams with strict latency requirements by enabling the signaling of a stream's deadline and ideally by combining multipath scheduling, congestion control adaptations, and optional forward error correction (FEC). Moreover, DMTP leverages the path identifiers introduced by the Multipath Extension for QUIC to distinguish different end-to-end paths as they may be offered in a Path Aware Network (PAN) such as SCION. This allows an application to select its preferred paths while maintaining interoperability with standard endpoints using the Multipath Extension for QUIC. In combination, DMTP enables endpoints to exchange and schedule deadline-aware streams across multiple network paths. Additionally, this proposal also maintains compatibility with QUIC itself, in order to deliver its benefits - albeit with limited effectiveness - even in scenarios where only a single path can or should be used. Fastly Google Ericsson This document specifies an extension to QUIC that enables an endpoint to request its peer change its behavior when sending or delaying acknowledgments. Note to Readers Discussion of this draft takes place on the QUIC working group mailing list (quic@ietf.org), which is archived at https://mailarchive.ietf.org/arch/search/?email_list=quic. Source code and issues list for this draft can be found at https://github.com/quicwg/ack-frequency. Working Group information can be found at https://github.com/quicwg. Alibaba Inc. Uber Technologies Inc. University of Mons (UMONS) UCLouvain and WELRI Private Octopus Inc. Ericsson This document specifies a multipath extension for the QUIC protocol to enable the simultaneous usage of multiple paths for a single connection. It introduces explicit path identifiers to create, delete, and manage multiple paths. This document does not specify address discovery or management, nor how applications using QUIC schedule traffic over multiple paths. Often endpoints are connected by multiple paths, but communications are usually restricted to a single path per connection. Resource usage within the network would be more efficient were it possible for these multiple paths to be used concurrently. Multipath TCP is a proposal to achieve multipath transport in TCP. New congestion control algorithms are needed for multipath transport protocols such as Multipath TCP, as single path algorithms have a series of issues in the multipath context. One of the prominent problems is that running existing algorithms such as standard TCP independently on each path would give the multipath flow more than its fair share at a bottleneck link traversed by more than one of its subflows. Further, it is desirable that a source with multiple paths available will transfer more traffic using the least congested of the paths, achieving a property called "resource pooling" where a bundle of links effectively behaves like one shared link with bigger capacity. This would increase the overall efficiency of the network and also its robustness to failure. This document presents a congestion control algorithm that couples the congestion control algorithms running on different subflows by linking their increase functions, and dynamically controls the overall aggressiveness of the multipath flow. The result is a practical algorithm that is fair to TCP at bottlenecks while moving traffic away from congested links. This document defines an Experimental Protocol for the Internet community. Path properties express information about paths across a network and the services provided via such paths. In a path-aware network, path properties may be fully or partially available to entities such as endpoints. This document defines and categorizes path properties. Furthermore, the document identifies several path properties that might be useful to endpoints or other entities, e.g., for selecting between paths or for invoking some of the provided services. This document is a product of the Path Aware Networking Research Group (PANRG). SCION Association SCION Association Anapaya Systems This document describes the Control Plane of the path-aware, inter- domain network architecture SCION (Scalability, Control, and Isolation On Next-generation networks). A fundamental characteristic of SCION is that it gives path control to SCION-capable endpoints that can choose between multiple path options, thereby enabling the optimization of network paths. The SCION Control Plane is responsible for discovering these paths and making them available to the endpoints. The SCION Control Plane creates and securely disseminates path segments between SCION Autonomous Systems (AS) which can then be combined into forwarding paths to transmit packets in the data plane. This document describes mechanisms of path exploration through beaconing and path registration. In addition, it describes how Endpoints construct end-to-end paths by combining path segments obtained through a path lookup process. This document contains new approaches to secure path aware networking. It is not an Internet Standard, has not received any formal review of the IETF, nor was the work developed through the rough consensus process. The approaches in this work are offered to the community for its consideration in the further evolution of the Internet. Independent SCION Association Independent Anapaya Systems This document describes the Data Plane of SCION (Scalability, Control, and Isolation On Next-generation networks), a path-aware, inter-domain network architecture. Unlike IP-based forwarding, SCION embeds inter-domain forwarding directives in the packet header, enabling endpoints to construct and select end-to-end paths from segments discovered by the Control Plane. The role of the Data Plane is to combine such segments into end-to-end paths, and to forward data according to the specified path. This document describes the SCION packet format, header structure, and extension headers. It also describes the cryptographic mechanisms used for path authorization, processing at routers including a life of a packet example. This document contains new approaches to secure path aware networking. It is not an Internet Standard, has not received any formal review of the IETF, nor was the work developed through the rough consensus process. The approaches offered in this work are offered to the community for its consideration in the further evolution of the Internet.

Acknowledgments Thanks to the Path Aware Networking Research Group for discussion and feedback. Specifically, we would like to thank Kevin Meynell and Nicola Rustignoli from the Scion Association for their valuable input on several iterations of this document.

Change Log Changes made to drafts since initial submission. This section is to be removed before publication.

draft-zaeschke-scion-quic-multipath-01 Major changes:

• Added suggestions for SCION level spoofing protection.
• Reverted and because the attack can only be mitigated by SCION aware QUIC implementations. The previously proposed new rules do not help.
• "Path Injection": clarify that path injection in SCION is, depending on the situation, either impossible or only marginally useful for an attacker. Rewrite sections and .
• New section "IP Ambiguity" in security considerations: .
• Completed section on Address Validation Token, see .
• Added recommendation to avoid using all SCION paths.
• Added section on path ID allocation and exhaustion.

Minor changes:

• Nits and wording improvements
• Reviewed use of normative language
• Added Changelog
• Moved memory exhaustion attack into own section.

draft-zaeschke-scion-quic-multipath-00 Initial submission.