Monday, November 3, 2008

DNS performance and the effectiveness of caching

This paper presents a study in which traces of DNS lookups and responses, as well as TCP starts and ends, were collected for analysis and simulations for testing out of ideas. To be honest, it is not clear to me why some of the analysis they would be interesting, but it is probably just me unable to appreciate what they have done. The more interesting portions are the experiments where the traces are used to drive simulations in which various parameters or ideas can be tested. The main results seem to be that reducing TLL of adress records down to a few hundred seconds has limited ill effect on cache hit rates. Also, while sharing DNS cache among clients might be useful, it saturates at about 10-20 clients.

One concern I have is with the time period in which data was collected. Its not clear why data was not collected over the same time period (i.e. the same week) in two different years (perhaps a deadline?) The number of students/users on campus on Dec 4 to Dec 11 (during the regular semester) would be different from Jan 3 to Jan 10 (corresponding to an optional month-long independent activities period). The number of lookups is higher in mit-dec00, but of course this could be in part due to increasing hosts and websites. The other problem with different periods would be that the type of students could be different, e.g. more international students staying behind in Jan, which might contribute to a different set of domains being looked up. This is of course speculative, but it would be more convincing (to me) to keep it the same period, especially in some of the trends that the authors claim in the paper. With just two time-points that are not even at the same time of the year, I would not believe any of the trends stated in the paper.

The other issue for me is that the authors claim that in hindsight, certain collection could be done differently. For example, 128 bytes collected per packet is insufficient; how did this affect the analysis? Also, the collection of UDP flows as well could verify some of the claims made. I imagine that the collection of data packets is not a trivial task, but if all the collection software/hardware is already in place, what would prevent the authors from re-collecting the data? The only reason I can think of is bureaucratic hoops that the authors would have to jump through for each of the collection.

No comments: