21 papers · ranked by Valyu relevance
Ayush Singhal, Rakesh Pant, Pradeep K. Sinha
The demand for stream processing is increasing at an unprecedented rate. Big data is no longer limited to processing of big volumes of data. In most real-world scenarios, the need for processing stream data as it comes can only meet the business needs. It is required for trading, fraud detection, system monitoring…
Marios Fragkoulis, Paris Carbone, Vasiliki Kalavri, Asterios Katsifodimos
'Asterios Katsifodimos'] Abstract Stream processing has been an active research field for more than 20 years, but it is now witnessing its prime time due to recent successful efforts by the research community and numerous worldwide open-source communities. This survey provides a comprehensive overview of fundamental…
Tongya Zheng, Gang Chen, Xinyu Wang, Chun Chen + 2 more
'Sihui Luo'] Abstract Human beings keep exploring the physical space using information means. Only recently, with the rapid development of information technologies and the increasing accumulation of data, human beings can learn more about the unknown world with data-driven methods. Given data timeliness, there is a…
Xikui Wang, Michael J. Carey, Vassilis J. Tsotras
Today, data is being actively generated by a variety of devices, services, and applications. Such data is important not only for the information that it contains, but also for its relationships to other data and to interested users. Most existing Big Data systems focus on passively answering queries from users, rather…
Haruna Isah, Farhana Zulkernine
—An essential part of building a data-driven organization is the ability to handle and process continuous streams of data to discover actionable insights. The explosive growth of interconnected devices and the social Web has led to a large volume of data being generated on a continuous basis. Streaming data sources…
Maninder Singh, Mohammad A. Hoque, Sasu Tarkoma
The immense growth of data demands switching from traditional data processing solutions to systems, which can process a continuous stream of real time data. Various applications employ stream processing systems to provide solutions to emerging Big Data problems. Open-source solutions such as Storm, Spark Streaming, and…
Lianjie Zhou, Nengcheng Chen, Zeqiang Chen, Leonhard M. Reindl
The efficient data access of streaming vehicle data is the foundation of analyzing, using and mining vehicle data in smart cities, which is an approach to understand traffic environments. However, the number of vehicles in urban cities has grown rapidly, reaching hundreds of thousands in number. Accessing the mass…
Radhya Sahal, Saeed H. Alsamhi, John G. Breslin, Muhammad Intizar Ali
'Muhammad Intizar Ali'] Forestry 4.0 is inspired by the Industry 4.0 concept, which plays a vital role in the next industrial generation revolution. It is ushering in a new era for efficient and sustainable forest management. Environmental sustainability and climate change are related challenges to promote sustainable…
Cun Ji, Qingshi Shao, Jiao Sun, Shijun Liu + 6 more
'Chenglei Yang' 'Yunchuan Sun' 'Antonio Jara' 'Shengling Wang'] Despite having played a significant role in the Industry 4.0 era, the Internet of Things is currently faced with the challenge of how to ingest large-scale heterogeneous and multi-type device data. In response to this problem we present a heterogeneous…
Amber Spackman Jones, Jeffery S. Horsburgh, Stephanie L. Reeder, Maurier Ramírez + 1 more
'Maurier Ramírez' 'Juan Caraballo'] It is common for hydrology researchers to collect data using in situ sensors at high frequencies, for extended durations, and with spatial distributions that produce data volumes requiring infrastructure for data storage, management, and sharing. The availability and utility of these…
Minh Duc Cao, Devika Ganesamoorthy, Alysha G. Elliott, Huihui Zhang + 2 more
The recently introduced Oxford Nanopore MinION platform generates DNA sequence data in real-time. This opens immense potential to shorten the sample-to-results time and is likely to lead to enormous benefits in rapid diagnosis of bacterial infection and identification of drug resistance. However, there are very few…
Siniša Veseli, John Hammonds, Steven Henke, Hannah Parraga + 3 more
'Barbara Frosik' 'Nicholas Schwarz' 'M. Yamamoto'] A computing framework for real-time analysis of X-ray detector data streamed directly to Python applications via the EPICS pvAccess protocol is described.
Suluk Chaikhan, Suphakant Phimoltares, Chidchanok Lursinsap, Mohamed Hammad
'Mohamed Hammad'] Big streaming data environment concerns a complicated scenario where data to be processed continuously flow into a processing unit and certainly cause a memory overflow problem. This obstructs the adaptation of deploying all existing classic sorting algorithms because the data to be sorted must be…
Francesco Versaci, Luca Pireddu, Gianluigi Zanetti
Modern sequencing machines produce order of a terabyte of data per day, which need subsequently to go through a complex processing pipeline. The standard workflow begins with a few independent, shared-memory tools, which communicate by means of intermediate files. Given the constant increase of the amount of data…
Giuseppe Coviello, Kunal Rao, Murugan Sankaradas, Srimat Chakradhar
The exponential growth in smart sensors and rapid progress in 5G networks is creating a world awash with data streams. However, a key barrier to building performant multi-sensor, distributed stream processing applications is high programming complexity. We propose DataX, a novel platform that improves programmer…
Páll Melsted, Bjarni V. Halldórsson
Several applications in bioinformatics, such as genome assemblers and error corrections methods, rely on counting and keeping track of k-mers (substrings of length k). Histograms of k-mer frequencies can give valuable insight into the underlying distribution and indicate the error rate and genome size sampled in the…
Roye Rozov, Gil Goldshlager, Eran Halperin, Ron Shamir
We present Faucet, a 2-pass streaming algorithm for assembly graph construction. Faucet builds an assembly graph incrementally as each read is processed. Thus, reads need not be stored locally, as they can be processed while downloading data and then discarded. We demonstrate this functionality by performing streaming…
Benjamin Coleman, Benito Geordie, Li Chou, R. A. Leo Elworth + 2 more
The rise of whole-genome shotgun sequencing (WGS) has enabled numerous breakthroughs in large-scale comparative genomics research. However, the size of genomic datasets has grown exponentially over the last few years, leading to new challenges for traditional streaming algorithms. Modern petabyte-sized genomic datasets…
Jamshed Khan, Rob Patro, Prashant Pandey
Hash tables are fundamental to computational genomics, where keys are often k-mers—fixed-length substrings that exhibit a “streaming” property: consecutive k-mers share k−1 nucleotides and are processed in order. Existing static data structures exploit this locality but cannot support dynamic updates, while…
Authors not listed
Bipolar electrode (BPE) systems have been attracting increasing attention owing to their cost-effectiveness and wireless design. Recently, we reported the concept of streaming potential-driven BPEs for the oxidative electropolymerization of aromatic monomers without the need for an electric power supply. However, our…
Authors not listed
Machine learning models are transforming data-driven research across scientific disciplines, yet their deployment as accessible and reliable web services remains a significant challenge. We introduce the NERDD framework, a scalable, maintainable, and secure microservices platform designed to support the sustainable…