Efficient organization of large multidimensional arrays
Authors:
- 17 December 2002
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 328-336
Abstract
Large multidimensional arrays are widely used in scientific and engineering database applications. The authors present methods of organizing arrays to make their access on secondary and tertiary memory devices fast and efficient. They have developed four techniques for doing this: (1) storing the array in multidimensional “chunks” to minimize the number of blocks fetched, (2) reordering the chunked array to minimize seek distance between accessed blocks, (3) maintaining redundant copies of the array, each organized for a different chunk size and ordering and (4) partitioning the array onto platters of a tertiary memory device so as to minimize the number of platter switches. The measurements on real data obtained from global change scientists show that accesses on arrays organized using these techniques are often an order of magnitude faster than on the unoptimized dataKeywords
This publication has 10 references indexed in Scilit:
- The POSTGRES next generation database management systemCommunications of the ACM, 1991
- Linear clustering of objects with multiple attributes1990
- A new vector quantization clustering algorithmIEEE Transactions on Acoustics, Speech, and Signal Processing, 1989
- A virtual memory system for picture processingCommunications of the ACM, 1984
- The Grid FileACM Transactions on Database Systems, 1984
- Picture paging for efficient image processing1980
- An Algorithm for Vector Quantizer DesignIEEE Transactions on Communications, 1980
- Storage reorganization techniques for matrix computation in a paging environmentCommunications of the ACM, 1979
- Preserving Proximity in ArraysSIAM Journal on Computing, 1975
- Organizing matrices and matrix operations for paged memory systemsCommunications of the ACM, 1969