Dissertation: A Data-Descriptive Feedback Framework for Data Stream Management Systems

I advanced to candidacy on May 5th, 2010. I expect to defend in 2012.

Abstract

Data Stream Management Systems (DSMSs) provide support for continuous query evaluation over data streams. Data streams provide processing challenges due to their unbounded nature and varying characteristics, such as rate and density fluctuations. DSMSs need to adapt stream processing to these changes within certain constraints, such as available computational resources and minimum latency requirements in producing results. The proposed research develops an inter-operator feedback framework, where opportunities for run-time adaptation of stream processing are expressed in terms of descriptions of substreams and actions applicable to thee substreams, called feedback punctuations. Both the discovery of adaptation opportunities and the exploitation of these opportunities are performed in the query operators. DSMSs are also concerned with state management, in particular, state derived from tuple processing. The proposed research also introduces a framework that provides execution guarantees that bound state accumulation in continuous query evaluation for systems with and without inter-operator feedback. This research provides both theoretical and design contributions, as well as evaluation of the proposed techniques.

Committee

• Prof. David Maier, Chair
• Jonathan Goldstein (Microsoft)
• Prof. Leonard Shapiro
• Prof. Kristin Tufte
• Prof. Christopher Monsere, Graduate Office Representative

Related publications

• Rafael J. Fernández-Moctezuma, David Maier, and Kristin A. Tufte.
  Towards Execution Guarantees for Stream Queries.
  In Proceedings of the Third International Workshop on Scalable Stream Processing Systems (SSPS), Atlanta, GA, USA, April 23, 2010.
  [PDF]
• Rafael J. Fernández-Moctezuma, Kristin A. Tufte, and Jin Li.
  Inter-Operator Feedback in Data Stream Management Systems via Punctuation.
  In Proceedings of the Fourth Biennial Conference on Innovative Data Systems Research (CIDR). Asilomar, Pacific Grove, CA. January 4-7, 2009.
  [PDF] [Acceptance rate: 27%]

Links

• Proposal
• NiagaraST Project
• Inter-Operator Contracts reference implementation (C#)