Andrew Washbrook
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Research Interests
Andrew Washbrook has a total of 37 co-authors. Here are the top co-authors by number of shared publications:
R Walker
Ludwig-Maximilians-Universitat Muchen, Munchen, Germany
2 shared publicationsA. Tamerus
High Performance Computing Service, The University of Cambridge, Roger Needham Building, 7 JJ Thomson Avenue, Cambridge CB3 0RB, United Kingdom
1 shared publicationB O'Brien
SUPA, School of Physics and Astronomy, The University of Edinburgh, James Clerk Maxwell Building, Mayfield Road, Edinburgh, EH9 3JZ, UK
1 shared publicationChristopher K R T Jones
University of Edinburgh
1 shared publicationD Crooks
Kelvin Building, Physics and Astronomy, University of Glasgow, Glasgow, G12 8QQ
1 shared publicationD Emeilyanov
Rutherford Appleton Laboratory
1 shared publication
Latest Publications
- Open Access
- 17 September 2019
- Published by EDP Sciences in EPJ Web of Conferences
- Vol. 214, 04043
Using ZFS to manage Grid storage and improve middleware resilience
ZFS is a powerful storage management technology combining filesystem, volume management and software RAID technology into a single solution. The WLCG Tier-2 computing at Edinburgh was an early adopter of ZFS on Linux, with this technology being used to manage all of our storage systems including servers with aging components. Our experiences of ZFS deployment have been shared with the Grid storage community which has led to additional sites adopting this technology. ZFS is highly configurable therefore allowing systems to be tuned to give best performance under diverse workloads. This paper highlights our experiences in tuning our systems for best performance when combining ZFS with DPM storage management. This has resulted in reduced system load and better data throughput. This configuration also provides the high redundancy required for managing older storage servers. We also demonstrate how ZFS can be combined with Distributed Replicated Block Device (DRBD) technology to provide a performant and resilient hypervisor solution to host multiple production Grid services.
- 17 September 2019
- Published by EDP Sciences in EPJ Web of Conferences
- Vol. 214, 04043
Open AccessUsing ZFS to manage Grid storage and improve middleware resilience
ZFS is a powerful storage management technology combining filesystem, volume management and software RAID technology into a single solution. The WLCG Tier-2 computing at Edinburgh was an early adopter of ZFS on Linux, with this technology being used to manage all of our storage systems including servers with aging components. Our experiences of ZFS deployment have been shared with the Grid storage community which has led to additional sites adopting this technology. ZFS is highly configurable therefore allowing systems to be tuned to give best performance under diverse workloads. This paper highlights our experiences in tuning our systems for best performance when combining ZFS with DPM storage management. This has resulted in reduced system load and better data throughput. This configuration also provides the high redundancy required for managing older storage servers. We also demonstrate how ZFS can be combined with Distributed Replicated Block Device (DRBD) technology to provide a performant and resilient hypervisor solution to host multiple production Grid services.
- Open Access
- 1 October 2017
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 898 (6) , 062054
Evaluation of ZFS as an efficient WLCG storage backend
A ZFS based software raid system was tested for performance against a hardware raid system providing storage based on the traditional Linux file systems XFS and EXT4. These tests were done for a healthy raid array as well as for a degraded raid array and during the rebuild of a raid array. It was found that ZFS performs better in almost all test scenarios. In addition, distinct features of ZFS were tested for WLCG data storage use, like compression and higher raid levels with triple redundancy information. The long term reliability was observed after converting all production storage servers at the Edinburgh WLCG Tier-2 site to ZFS, resulting in about 1.2PB of ZFS based storage at this site.
- 1 October 2017
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 898 (6) , 062054
Open AccessEvaluation of ZFS as an efficient WLCG storage backend
A ZFS based software raid system was tested for performance against a hardware raid system providing storage based on the traditional Linux file systems XFS and EXT4. These tests were done for a healthy raid array as well as for a degraded raid array and during the rebuild of a raid array. It was found that ZFS performs better in almost all test scenarios. In addition, distinct features of ZFS were tested for WLCG data storage use, like compression and higher raid levels with triple redundancy information. The long term reliability was observed after converting all production storage servers at the Edinburgh WLCG Tier-2 site to ZFS, resulting in about 1.2PB of ZFS based storage at this site.
- Open Access
- 23 December 2015
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 664 (9) , 092026
Acceleration of ensemble machine learning methods using many-core devices
We present a case study into the acceleration of ensemble machine learning methods using many-core devices in collaboration with Toshiba Medical Visualisation Systems Europe (TMVSE). The adoption of GPUs to execute a key algorithm in the classification of medical image data was shown to significantly reduce overall processing time. Using a representative dataset and pre-trained decision trees as input we will demonstrate how the decision forest classification method can be mapped onto the GPU data processing model. It was found that a GPU-based version of the decision forest method resulted in over 138 times speed-up over a single-threaded CPU implementation with further improvements possible. The same GPU-based software was then directly applied to a suitably formed dataset to benefit supervised learning techniques applied in High Energy Physics (HEP) with similar improvements in performance.
- 23 December 2015
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 664 (9) , 092026
Open AccessAcceleration of ensemble machine learning methods using many-core devices
We present a case study into the acceleration of ensemble machine learning methods using many-core devices in collaboration with Toshiba Medical Visualisation Systems Europe (TMVSE). The adoption of GPUs to execute a key algorithm in the classification of medical image data was shown to significantly reduce overall processing time. Using a representative dataset and pre-trained decision trees as input we will demonstrate how the decision forest classification method can be mapped onto the GPU data processing model. It was found that a GPU-based version of the decision forest method resulted in over 138 times speed-up over a single-threaded CPU implementation with further improvements possible. The same GPU-based software was then directly applied to a suitably formed dataset to benefit supervised learning techniques applied in High Energy Physics (HEP) with similar improvements in performance.
- Open Access
- 23 December 2015
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 664 (2) , 022041
Development of site-oriented Analytics for Grid computing centres
The field of analytics, the process of analysing data to visualise meaningful patterns and trends, has become increasingly important in scientific computing as the volume and variety of data available to process has significantly increased. There is now ongoing work in the High Energy Physics (HEP) community in this area, for example in the augmentation of systems management at WLCG computing sites. We report on work evaluating the feasibility of distributed site-oriented analytics using the Elasticsearch, Logstash and Kibana software stack and demonstrate functionality by the application of two workflows that give greater insight into site operations.
- 23 December 2015
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 664 (2) , 022041
Open AccessDevelopment of site-oriented Analytics for Grid computing centres
The field of analytics, the process of analysing data to visualise meaningful patterns and trends, has become increasingly important in scientific computing as the volume and variety of data available to process has significantly increased. There is now ongoing work in the High Energy Physics (HEP) community in this area, for example in the augmentation of systems management at WLCG computing sites. We report on work evaluating the feasibility of distributed site-oriented analytics using the Elasticsearch, Logstash and Kibana software stack and demonstrate functionality by the application of two workflows that give greater insight into site operations.
- Open Access
- 23 December 2015
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 664 (2) , 022034
Evaluation of containers as a virtualisation alternative for HEP workloads
In this paper the emerging technology of Linux containers is examined and evaluated for use in the High Energy Physics (HEP) community. Key technologies required to enable containerisation will be discussed along with emerging technologies used to manage container images. An evaluation of the requirements for containers within HEP will be made and benchmarking will be carried out to asses performance over a range of HEP workflows. The use of containers will be placed in a broader context and recommendations on future work will be given.
- 23 December 2015
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 664 (2) , 022034
Open AccessEvaluation of containers as a virtualisation alternative for HEP workloads
In this paper the emerging technology of Linux containers is examined and evaluated for use in the High Energy Physics (HEP) community. Key technologies required to enable containerisation will be discussed along with emerging technologies used to manage container images. An evaluation of the requirements for containers within HEP will be made and benchmarking will be carried out to asses performance over a range of HEP workflows. The use of containers will be placed in a broader context and recommendations on future work will be given.
- Open Access
- 11 June 2014
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 513 (5) , 052037
Many-core on the Grid: From exploration to production
High Energy Physics experiments have successfully demonstrated that many-core devices such as GPUs can be used to accelerate critical algorithms in their software. There is now increasing community interest for many-core devices to be made available on the LHC Computing Grid infrastructure. Despite anticipated usage there is no standard method available to run many-core applications in distributed computing environments and before many-core resources are made available on the Grid a number of operational issues such as job scheduling and resource discovery will need to be addressed. The key challenges for Grid-enabling many-core devices will be discussed.
- 11 June 2014
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 513 (5) , 052037
Open AccessMany-core on the Grid: From exploration to production
High Energy Physics experiments have successfully demonstrated that many-core devices such as GPUs can be used to accelerate critical algorithms in their software. There is now increasing community interest for many-core devices to be made available on the LHC Computing Grid infrastructure. Despite anticipated usage there is no standard method available to run many-core applications in distributed computing environments and before many-core resources are made available on the Grid a number of operational issues such as job scheduling and resource discovery will need to be addressed. The key challenges for Grid-enabling many-core devices will be discussed.
- Open Access
- 11 June 2014
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 513 (3) , 032104
Leveraging HPC resources for High Energy Physics
High Performance Computing (HPC) supercomputers provide unprecedented computing power for a diverse range of scientific applications. The most powerful supercomputers now deliver petaflop peak performance with the expectation of "exascale" technologies available in the next five years. More recent HPC facilities use x86-based architectures managed by Linux-based operating systems which could potentially allow unmodified HEP software to be run on supercomputers. There is now a renewed interest from both the LHC experiments and the HPC community to accommodate data analysis and event simulation production on HPC facilities. This study provides an outline of the challenges faced when incorporating HPC resources for HEP software by using the HECToR supercomputer as a demonstrator.
- 11 June 2014
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 513 (3) , 032104
Open AccessLeveraging HPC resources for High Energy Physics
High Performance Computing (HPC) supercomputers provide unprecedented computing power for a diverse range of scientific applications. The most powerful supercomputers now deliver petaflop peak performance with the expectation of "exascale" technologies available in the next five years. More recent HPC facilities use x86-based architectures managed by Linux-based operating systems which could potentially allow unmodified HEP software to be run on supercomputers. There is now a renewed interest from both the LHC experiments and the HPC community to accommodate data analysis and event simulation production on HPC facilities. This study provides an outline of the challenges faced when incorporating HPC resources for HEP software by using the HECToR supercomputer as a demonstrator.
- Open Access
- 13 December 2012
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 396 (3) , 032115
Multi-core job submission and grid resource scheduling for ATLAS AthenaMP
AthenaMP is the multi-core implementation of the ATLAS software framework and allows the efficient sharing of memory pages between multiple threads of execution. This has now been validated for production and delivers a significant reduction on the overall application memory footprint with negligible CPU overhead. Before AthenaMP can be routinely run on the LHC Computing Grid it must be determined how the computing resources available to ATLAS can best exploit the notable improvements delivered by switching to this multi-process model. A study into the effectiveness and scalability of AthenaMP in a production environment will be presented. Best practices for configuring the main LRMS implementations currently used by grid sites will be identified in the context of multi-core scheduling optimisation.
- 13 December 2012
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 396 (3) , 032115
Open AccessMulti-core job submission and grid resource scheduling for ATLAS AthenaMP
AthenaMP is the multi-core implementation of the ATLAS software framework and allows the efficient sharing of memory pages between multiple threads of execution. This has now been validated for production and delivers a significant reduction on the overall application memory footprint with negligible CPU overhead. Before AthenaMP can be routinely run on the LHC Computing Grid it must be determined how the computing resources available to ATLAS can best exploit the notable improvements delivered by switching to this multi-process model. A study into the effectiveness and scalability of AthenaMP in a production environment will be presented. Best practices for configuring the main LRMS implementations currently used by grid sites will be identified in the context of multi-core scheduling optimisation.
- Open Access
- 23 December 2011
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 331 (2) , 022031
Algorithm Acceleration from GPGPUs for the ATLAS Upgrade
Feasibility studies into the use of General Purpose CPUs have been performed on two key algorithms in the ATLAS High Level Trigger. A GPGPU based version of the Z-finder vertex finding algorithm resulted in over 35 times speed-up over serial CPU execution in the best case scenario, whilst a speed-up of over 5 times was observed from a GPGPU-based Kalman filter track finder routine. The approaches taken for converting these algorithms for execution on GPU devices is described.
- 23 December 2011
- Published by IOP Publishing in Journal of Physics: Conference Series
- Vol. 331 (2) , 022031
Open AccessAlgorithm Acceleration from GPGPUs for the ATLAS Upgrade
Feasibility studies into the use of General Purpose CPUs have been performed on two key algorithms in the ATLAS High Level Trigger. A GPGPU based version of the Z-finder vertex finding algorithm resulted in over 35 times speed-up over serial CPU execution in the best case scenario, whilst a speed-up of over 5 times was observed from a GPGPU-based Kalman filter track finder routine. The approaches taken for converting these algorithms for execution on GPU devices is described.