Search · four archives
Search · four archives
16 papers · ranked by Valyu relevance
Teng Li, Zhiyuan Xu, Jian Tang, Yanzhi Wang
In this paper, we focus on general-purpose Distributed Stream Data Processing Systems (DSDPSs), which deal with processing of unbounded streams of continuous data at scale distributedly in real or near-real time. A fundamental prob lem in a DSDPS is the scheduling problem (i.e., assigning workload to workers/machines)…
Marcos Dias de Assunção, Alexandre da Silva Veith, Rajkumar Buyya
Under several emerging application scenarios, such as in smart cities, operational monitoring of large infrastructure, wearable assistance, and Internet of Things, continuous data streams must be processed under very short delays. Several solutions, including multiple software engines, have been developed for…
Anshu Shukla, Yogesh Simmhan
Distributed Stream Processing frameworks are being commonly used with the evolution of Internet of Things(IoT). These frameworks are designed to adapt to the dynamic input message rate by scaling in/out.Apache Storm, originally developed by Twitter is a widely used stream processing engine while others includes Flink…
Tarek Stolz, István Koren, Liam Tirpitz, Sandra Geisler
With the increasing prevalence of IoT environments, the demand for processing massive distributed data streams has become a critical challenge. Data Stream Processing on the Edge (DSPoE) systems have emerged as a solution to address this challenge, but they often struggle to cope with the heterogeneity of hardware and…
Milinda Pathirage, Beth Plale
— To stay competitive in today's data driven economy, enterprises large and small are turning to stream processing platforms to process high volume, high velocity, and diverse streams of data (fast data) as they arrive. Low-level programming models provided by the popular systems of today suffer from lack of…
Genoveva Vargas‐Solar, Javier A. Espinosa-Oviedo
This paper introduces H-STREAM, a big stream/data processing pipelines evaluation engine that proposes stream processing operators as micro-services to support the analysis and visualisation of Big Data streams stemming from IoT (Internet of Things) environments. H-STREAM micro-services combine stream processing and…
Suyeon Lee, Sungyong Park
—This paper presents LMStream, which ensures bounded latency while maximizing the throughput on the GPUenabled micro-batch streaming systems. The main ideas behind LMStream's design can be summarized as two novel mechanisms: (1) dynamic batching and (2) dynamic operation-level query planning. By controlling the…
Cristian Ramon-Cortes, Francesc Lordan, Jorge Ejarque, Rosa M. Badía
In the past years, e-Science applications have evolved from large-scale simulations executed in a single cluster to more complex workflows where these simulations are combined with High-Performance Data Analytics (HPDA). To implement these workflows, developers are currently using different patterns; mainly task-based…
Shantenu Jha, Daniel S. Katz, André Luckow, Omer Rana + 2 more
'Yogesh Simmhan' 'Neil Chue Hong'] A common feature across many science and engineering applications is the amount and diversity of data and computation that must be integrated to yield insights. Data sets are growing larger and becoming distributed; and their location, availability and properties are often…
Arun Kejariwal, Sanjeev R. Kulkarni, Karthik Ramasamy
Velocity is one of the 4 Vs commonly used to characterize Big Data [27]. In this regard, Forrester remarked the following in Q3 2014 [94]: "The high velocity, white-water flow of data from innumerable real-time data sources such as market data, Internet of Things, mobile, sensors, clickstream, and even transactions…
Ovidiu-Cristian Marcu, Pascal Bouvry
—Real-time Big Data architectures evolved into specialized layers for handling data streams' ingestion, storage, and processing over the past decade. Layered streaming architectures integrate pull-based read and push-based write RPC mechanisms implemented by stream ingestion/storage systems. In addition, stream…
Atul Sandur, ChanHo Park, Stavros Volos, Gul Agha + 1 more
—Rapid detection and mitigation of issues that impact performance and reliability is paramount for large-scale online services. For real-time detection of such issues, datacenter operators use a stream processor and analyze streams of monitoring data collected from servers (referred to as data source nodes) and their…
Maninder Singh, Mohammad A. Hoque, Sasu Tarkoma
The immense growth of data demands switching from traditional data processing solutions to systems, which can process a continuous stream of real time data. Various applications employ stream processing systems to provide solutions to emerging Big Data problems. Open-source solutions such as Storm, Spark Streaming, and…
Zhida Chen, Gao Cong, Zhenjie Zhang, Tom Z. J. Fu + 1 more
—Huge amount of data with both space and text information, e.g., geo-tagged tweets, is flooding on the Internet. Such spatio-textual data stream contains valuable information for millions of users with various interests on different keywords and locations. Publish/subscribe systems enable efficient and effective…
Shilpa Chaturvedi, Sahil Tyagi, Yogesh Simmhan
Distributed Stream Processing Systems (DSPS) like Apache Storm and Spark Streaming enable composition of continuous dataflows that execute persistently over data streams. They are used by Internet of Things (IoT) applications to analyze sensor data from Smart City cyber-infrastructure, and make active utility…
Sylvain Hallé, Raphaël Khoury, Sébastien Gaboury
Current runtime verification tools seldom make use of multi-threading to speed up the evaluation of a property on a large event trace. In this paper, we present an extension to the BeepBeep 3 event stream engine that allows the use of multiple threads during the evaluation of a query. Various parallelization strategies…