A Framework for Statistical Analysis of Datasets on Heterogeneous Clusters
- 1 September 2005
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 15525244,p. 1-9
Abstract
This paper proposes a framework for the statistical analysis of multiple related datasets on heterogeneous clusters. The set of processors assigned to the framework are partitioned into groups according to rack locations, with the group sizes being chosen to match the degree of concurrency in the analysis procedure. The datasets are initially divided among the groups. Dynamic loop scheduling is employed to address load imbalance arising from the differences in computational powers of groups, the variability of dataset sizes, and the unpredictable irregularities in the cluster environment. Results of preliminary tests indicate the effectiveness of the framework in fitting gamma-ray burst datasets with vector functional coefficient autoregressive time series models on 64 processors of a heterogeneous general-purpose Linux clusterKeywords
This publication has 9 references indexed in Scilit:
- A note on multi-step forecasting with functional coefficient autoregressive modelsInternational Journal of Forecasting, 2005
- Performance of scheduling scientific applications with adaptive weighted factoring2005
- Load balancing highly irregular computations with the adaptive factoring2002
- Load-sharing in heterogeneous systems via weighted factoring1996
- Balancing processor loads and exploiting data locality in N-body simulations1995
- Trapezoid self-scheduling: a practical scheduling scheme for parallel compilersIEEE Transactions on Parallel and Distributed Systems, 1993
- Factoring: a method for scheduling parallel loopsCommunications of the ACM, 1992
- Guided Self-Scheduling: A Practical Scheduling Scheme for Parallel SupercomputersIEEE Transactions on Computers, 1987
- Allocating Independent Subtasks on Parallel ProcessorsIEEE Transactions on Software Engineering, 1985