PlumX Metrics
Embed PlumX Metrics

Orchestrating scheduling, grouping and parallelism to enhance the performance of distributed stream computing system

Expert Systems with Applications, ISSN: 0957-4174, Vol: 254, Page: 124346
2024
  • 0
    Citations
  • 0
    Usage
  • 5
    Captures
  • 1
    Mentions
  • 0
    Social Media
Metric Options:   Counts1 Year3 Year

Metrics Details

  • Captures
    5
  • Mentions
    1
    • News Mentions
      1
      • News
        1

Most Recent News

Recent Findings in Data Streaming Described by Researchers from China University of Geosciences (Orchestrating Scheduling, Grouping and Parallelism To Enhance the Performance of Distributed Stream Computing System)

2024 NOV 14 (NewsRx) -- By a News Reporter-Staff News Editor at Information Technology Daily -- Investigators publish new report on Information Technology - Data

Article Description

In a big data stream computing environment, the arrival rate of data streams usually fluctuates over time, posing a great challenge to the elasticity of system. The performance of stream computing system is crucial, especially when dealing with unbounded and fluctuating data streams. Most prior studies have primarily focused on one or two aspects to enable elasticity, often lacking prompt and comprehensive performance optimization. This limitation could lead to a tuning bottleneck, preventing the system’s performance from consistently reaching its optimal state. Additionally, many stream computing systems are not intelligently adaptive in real time due to the challenges of manual parameter reconfiguration for fluctuating streams. To better address these issues, we propose a framework named Sgp-Stream, which orchestrates scheduling, grouping and parallelism (Sgp). To enhance the system performance. We conduct the following research: (1) Running experiments to evaluate the impact of different factors such as scheduling, grouping and parallelism on system performance. Results show that factors at a single level usually have an upper limit on tuning system performance, and better overall performance can be achieved by coordinating multi-level factors. (2) Establishing quantitative models for stream application that consider computational cost and communication cost, multi-dimensional featured data stream, data center resources, and latency & throughput performance. (3) Demonstrating the effectiveness of the proposed runtime-aware data stream grouping based on smooth weighted polling, elastic adaptive scheduling based on Linear Deterministic Greedy and elastic scaling strategy based on Gradient Descent in Sgp-Stream, for continuous performance optimization.(4) Evaluating the application latency, throughput and resource utilization objectives using a real-world elastic stream computing system and twitter data set. Experimental results show that, compared to existing state-of-the-art works, the proposed Sgp-Stream outperforms them by reducing latency by 26%–48%, improving throughput by 14%–20%, and increasing resource utilization rate by 15%–21%, especially under increasing data stream input rates.

Provide Feedback

Have ideas for a new metric? Would you like to see something else here?Let us know