Scaling internet routers using optics

In this paper, the authors set out to design a 100TB/s router, motivated by the increasing demand for high router capacity which are not being met by current router design developments. They target three main issues with existing technologies: high power consumption, unpredictable performance and poor scalability. To overcome power consumption issues, they argue for optical switching; to over come high power consumption and unpredictable performance, they propose a solution that is based on a previously proposed load-balanced switch, but modified to handle four main problems: (i) need for rapid re-configuration of switch fabric; (ii) possible mis-sequencing of packets; (iii) throughput can be made arbitrarily small; and (iv) breaks down when linecards fails.

The basic architecture is based on a load-balanced switch, which has a load-balancing stage implemented by a switch, followed by a single stage of buffer, and then followed by another switching stage to the final outputs. The switching is done in a deterministic manner, which makes it easily implemented in optics by a arrayed waveguide grating router, thus eliminating the need for rapid reconfiguration. The load-balancing is performed by the input linecard, using a Full Ordered Frames First (FOFF) scheme that spreads input packets over all intermediate linecards through the switching inputs. This also mitigates the possible mis-sequencing of packets and prevents pathological traffic patterns from arbitrarily affecting throughput. The actual switching of the packets from input to output is performed by intermediate linecards, which places them in the appropriate queue so that they will end up in the right output after the second switching stage. This is possible since switching is done in a deterministic manner. Finally, by partioning the switching into multiple stages and using a pre-determined routing of the optical switches, it can adapt to linecard additions/deletions.

I unfortunately do not follow some parts of the paper. For example, I am rather confused by the role that linecards play. The authors mention both input linecards and intermediate linecards. Are these a conceptual distinction, or does the actual physical implementation use separate linecards for two different purposes? This was especially confusing to me since some diagrams show a linecard doing both of these things. If so, how does the linecard know from the inputs what is intermediate packets and what are meant for outgoing?

The authors also list some open challenges, such as high speed address lookups due to processing and memory speed bottlenecks.

I felt that the paper started off being fairly easy to follow. The introduction motivated and introduced the problem pretty well. However, the technical sections were very hard to understand, possible (probably) because this is an area I had no exposure to previously, and many of the terms and concepts, such as crossbar, are alien to me. The various sections seem rather disjointed, perhaps because they were written by different people? By the time I got to the end of the paper, I felt rather exhausted, and yet had the feeling that I did not really got too much out of the paper.

Supporting real-time applications in an integrated services packet network

As its title suggests, this paper proposes an architecture and mechanism for providing service guarantees. It provides two classes of services: guaranteed delays, which is suitable for clients that are intolerant to highly varying delays and fixes its operation based on an assumed delay, and predictive delays, which is suitable for clients that are tolerant to varying delays and is able to adapt to it. 

The mechanism to support such an architecture is based on a hierarchy of scheduling algorithms. For the guaranteed class of services, each client is given its own flow, while for the predictive class of services, all clients are aggregated into a virtual flow. Within that virtual flow, clients that request a particular delay constraint are grouped into the same priority class. The scheduling mechanism performs WFQ across all flows, including the virtual flow. This provides isolation of each flow, assuring that a misbehaving flow would have limited impact on the other flows. Within the virtual flow, FIFO/FIFO+ together with priority is used to schedule all the individual flows. This provides sharing of the link in order to distribute delay.

There are two further  (among others) interesting details. First, FIFO+ extends FIFO by attempting to emulate FIFO in aggregate across all hops to provide more effective sharing. However, this requires modification of the packet header to include some state information about the delay seen by the packet up to the current hop.  Second, for the predictive service, clients have to specify their traffic generation patterns using parameters of a token bucket filter, which provides some idea of the average and peak rates (or the amount of burstiness). This is to be used in conjunction with delay and loss requirements to appropriately assign its  priority class. This seems to be a hybrid of the explicit and implicit service selection schemes mentioned in the previous paper. The network is inferring the appropriate service class using a set of application provided parameters, thus avoiding the need to know service requirements of each application class. The application is also explicitly requesting for certain service requirements, but in a more flexible and extensible manner.

The paper does assume a play-back application as the one that requires real-time service. In a real-time playback as in telephony, there is very limited buffering due to tight latency constraints. Hence, it is not clear to me how adaptive the schemes are with respect to being able to shift the playback time. (Constrast this to streaming recorded content).  Also, my experience with video is that it is not too tolerant to packet losses, although various things such as error correction and concealment can be done to somewhat mitigate it. Given a choice between loss, delay and bandwidth, it is usually always much better to take a hit in bandwidth, because the resulting video experience would be better. (Delay and loss are somewhat equivalent in a real-time scenario)

Fundamental design issues for the future Internet

In this paper, the author's thesis is that due to the emergence of multimedia applications such as voice/video telephony with very different delay/bandwidth service requirements from traditional data applications, the following list of things need to be done:

1. IP should extend its service model, since services which are aligned with service requirements increases overall utility of the network.
2. The application should request the appropriate service instead of being it being estimated by the network, to maintain separation between network and application layers.
3. Admission control, instead of over-provisioning, should be adopted to prevent network overload, since it is economically more sound.

The author looks at the Internet from the high-level, and use macro-type of analysis to make his arguments. To do so, a framework is introduced whereby the design objective is to maximize the total utility of users of the Internet, subject to physical constraints of the network. This is not meant as a way to optimize resource allocation or choose network parameters, but rather as a tool to characterize various design choices. Unfortunately, the outcome of the analysis sometimes depend on the various cost and utility parameters, such as the relative cost of over-provisioning and changing network mechanisms, and utility function when certain service requirements are not met. For example, in trying to show that service models should be extended, the argument is assuming that over-provisioning cannot be done and the cost of adding a service model is negligible. 

The author argues for an explicit choosing of service by an application, rather than the service being implicitly assigned by the network, based on a layer separation argument. While an implicit assignment might be bad, it is not clear if an explicit assignment would work in practice either, due to issues such as incentives (and authentication, accounting, billing to establish financial incentives). In other words, even if it was established that separate services is the way to go, there might not be a good way to actually assign a service to a packet.

The discussion on admission controls seems to me to be the most sound. I like the use of the utility maximizing framework here, since the outcome is independent of quantities, but just depends on the characteristics of the utility function. The logical outcome of best-effort service for data, and admission control for real-time applications was very satisfying. Unfortunately, it is not clear to me how admission control is to be done for the Internet, since the user (or gateway) has to in effect probe the state of the network, which might not be easy or stationary. Also, such a mechanism can be abused to "excommunicate" users - a chilling thought.

I'm not sure how different the Internet was during the 1990s, but with hindsight, it is interesting to note that various things we accept today, such as spam, web-crawling, IP (video)telephony, were mentioned in this paper.

Congestion control for high bandwidth-delay product networks

This paper proposes eXplicit control protocol (XCP) to address some of the performance shortcomings of TCP, particularly as delay/bandwidth increases. For example, with high RTT, TCP's policy of additive increase increases bandwidth utilization very slowly. Another scenario is with short flows, which is unable to use available bandwidth faster than slow start allows. The authors also point out that with TCP uses an implicit congestion feedback through packet loss, which only indicates the presence/absence of congestion and not its amount, thus sources have to keep pushing the link to congestion before backing off continuously.

As implied by its name, XCP provides feedback about the state of congestion at the routers to senders, by indicating to the senders how much to adjust its congestion window by. Two key elements of XCP are the efficiency controller and the fairness controller, which operate independently. The efficiency controller aims at maximizing link utilization, and uses a feedback controller based on spare bandwidth and queue size to set the aggregate change in rate. The parameters of this feedback controller are chosen based on the feedback delay to ensure that the system is stable. The fairness controller then parcels out the aggregate change to various packets to achieve fairness, based on an additive-increase-multiplicative-decrease mechanism. As the authors noted, XCP can converge to fairness much faster, since the adjustment is done with each RTT instead with each packet loss.

This paper is full of useful ideas that can be applicable in other problems as well. First, there is the use of control theory to examine how a network (the plant) should be controlled and parameters should be chosen. Second, the use of a fluid model to allow the analysis of protocols by approximating a discrete system with a continuous one. (How good the approximation is would be debatable though.)  Third, the mention of a shadow pricing model is an interesting way of determining how to allocate resources based on how much users are willing to pay for them.

On the other hand, the policing of misbehaving sources based on border agents sounds like an afterthought to address the vulnerabilities of the system to non-compliant senders. While it is possible (but expensive) to have gateways to shut down the bad sources, it would be far better to have a scheme which provides no ineherent incentives to a source to misbehave - i.e. the compliant sender policy should also be one that gives the sender the highest throughput.

The experimental results are also to be expected. If a router is able to control how sources should adjust their transmission rate, then it is easy to believe that it would be able to achieve high link utilization.