An Efficient and Resilient Approach to Filtering and Disseminating Streaming Data

This chapter discusses an efficient and resilient approach to filtering and disseminating streaming data. It considers techniques for creating a resilient and efficient content distribution network for such dynamically changing streaming data. The chapter addresses the problem of maintaining the coherency of dynamic data items in a network of repositories: Data disseminated to one repository is filtered by that repository and disseminated to repositories dependent on it. The method is resilient to link failures and repository failures. This resiliency implies that data fidelity is not lost even when the repository from which a user obtains data experiences failures. Experimental evaluation, using real world traces of streaming data, demonstrates that (1) the cost of adding this redundancy is low, and (2) surprisingly, in many cases, adding resiliency-enhancing features actually improves the fidelity provided by the system even in cases when there are no failures. To enhance fidelity, the chapter proposes efficient techniques for filtering data arriving at one repository and for scheduling the dissemination of filtered data to another repository. The results show that the combination of resiliency enhancing and efficiency improving techniques in fact help derive the potential that push based systems are said to have in delivering 100% fidelity. Without them, computational and communication delays inherent in dissemination networks can lead to a large fidelity loss even in push based dissemination.