Linear Road implementation on PC

SCSQ (pronounced 'sisque', Supercomputer Stream Query processor) is a Data Stream Management System (DSMS) that enables queries over high-volume distributed streams.It enables high level specification of distributed stream queries that filter, transform, and join data from different kinds of distributed streaming data sources. The SCSQ prototype that runs on a variety of hardware platforms, from Windows to IBM BlueGene massively parallel computers.

The performance of the SCSQ prototype has been evaluated using the Linear Road Benchmark for data stream management systems. Linear Road simulates an expressway system with dynamically varying toll rates producing data streams to be processed by a DSMS. The performance of a DSMS is measured as its L-rating, which is the number of expressways the system can process in real-time. It is endorsed by several universities, including Brandeis, Brown, MIT, and Stanford.

New: The highly scalable parallel implementation of Linear Road SCSQ-PLR now achieves L=64, which is substantially improved scalability over previously published results for the Linear Road Benchmark:

E. Zeitler and T.Risch: Scalable Splitting of Massive Data Streams, to be presented at Proc. 15th Conf. on Database Systems for Advanced Application, DASFAA 2010., Tokyo, Japan, 1-4 April, 2010 (abstract). Addendum: Algebraic derivation of Maxtree

The downloadable and ready-to-run SCSQ-LR implements Linear Road with SCSQ on a single PC running Windows. It passes the Linear Road benchmark for L=1.5, i.e. SCSQ-LR can process data from 1.5 expressways in real-time on a stationary PC (Fujitsu-Siemens 2.21 GHz AMD64 4200+ with 2.87 GB of RAM) or a laptop PC (LG, 1.73 GHz, 1 GB RAM). It handles L=1.0 on a Thinkpad notebook X40 (1.2GHz Pentium M, 760MB RAM).

The SCSQ-LR implementation is described in the MSc Thesis report:
Benchmarking the performance of a data stream management system.

Read more about SCSQ in:

G.Gidofalvi, T.B. Pedersen, T.Risch, and E.Zeitler: Highly Scalable Trip Grouping for Large Scale Collective Transportation Systems, Proc. 11th International Conference on Extending Database Technology, EDBT 2008 , Nantes, France, March 2008.

E.Zeitler and T.Risch: Using stream queries to measure communication performance of a parallel computing environment. First International Workshop on Distributed Event Processing, Systems and Applications (DEPSA), Toronto, Canada, June 29, 2007.