Advanced Routing, Congestion Control and Quality-of-Service Mechanisms in HPC Fabrics such as Intel® Omnipath®

There are two main areas to cover in this project and they have been divided into two work-packages (WP), each one subdivided in two tasks.

WP1. Topologies, adaptive routing, and congestion control: Adaptive routing allows exploiting, on-the-fly, alternative routes in the presence of congestion in the network. Congestion management on the other hand tries to prevent congestion or alleviate the impact of its negative effects, especially Head-of-Line (HoL) blocking, mainly by dividing switch buffers into several queues to separately store different traffic flows. This WP will evaluate the interaction between current adaptive routings and queuing schemes, as adaptive routing will increase the mobility and dynamics of congested hot-spots in the network (adaptively diverting traffic to avoid a congested area of the network may lead to congestion in other areas). Then, new approaches to adaptive routing will be explored in order to avoid diverting traffic spoiling the separation of traffic flows in the queues. Two lines of studies will be performed, corresponding to different types of network topologies, which can be arranged in two tasks:

– Task 1.1. Congestion control and adaptive routing for fat-tress. In this task, the study will focus on the family of fat-tree topologies. The team has a long track in this topic, its researchers have developed routing algorithms and queuing schemes already in use on IBA-based fabrics with fat-tree topologies. This background, as well as other previous proposals, will be taken into account to define new solutions that combine adaptive routing and queuing schemes for fat-trees. It will be also analyzed the feasibility of the proposed solutions in the framework of Intel Omnipath technologies.

– Task 1.2. Congestion control and adaptive routing for emergent topologies. In this task, the study will focus on emergent topologies such as SlimFly, Dragonfly, Megafly, etc. The team also has a long track in this topic, its researchers have developed routing algorithms and queuing schemes already in use on IBA-based fabrics with these types of novel topologies. This background, as well as other previous proposals for these topologies and the insights and conclusions drawn from Task 1.1, will be taken into account to define new solutions that combine adaptive routing and queuing schemes for these topologies. It will be also analyzed the feasibility of the proposed solutions in the framework of Intel Omnipath technologies.

WP2. QoS Enhancement Leveraging Current OPA Hardware HPC interconnection networks should provide QoS capabilities to the different traffic flows in the fabric. This is a requirement so some applications are able to request to the fabric some guarantees or best-efforts on response time (latency) or bandwidth. OPA already supports some QoS support using different resources and techniques (VCs, traffic classes, arbitration, etc.). This task will focus on the best strategies to configure all those available parameters in OPA to best fulfill the QoS requirements by at-scale applications. Moreover, potential enhancements to the baseline arbitration schemes present in OPA will be also evaluated, and their appropriateness to meet QoS requirements in the fabric. This WP has been also divided in two tasks:

– Task 2.1 Study of the current Omnipath QoS support. The objective of this task is to study the different mechanisms that OmniPath provides in order to propose a general framework for providing QoS to the applications over computing systems using this interconnect technology. The aim is to determine the best way to configure this OPA support in order to provide QoS in terms of different QoS metrics, such as latency or bandwidth. These mechanisms will be evaluated under the usual traffic used in this kind of environments.

– Task 2.2 Development of enhancements to the OPA QoS support. The purpose of this task is to study possible enhancements to the arbitration schemes present in OPA. The comparison point may be a baseline where there is no strict QoS guarantees but mere differentiated services (simple distinction between higher-priority flows versus low-priority). In this case the arbitration schemes that are required will be much simpler yet still meeting the QoS requirements in at-scale systems.