- 12 May 2025
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Abstract
Inspired by prior work suggesting undetected errors were becoming a problem on the Internet, we set out to create a measurement system to detect errors that the TCP checksum missed. We designed a client-server framework in which the servers sent known files to clients. We then compared the received data with the original file to identify undetected errors introduced by the network. We deployed this measurement framework on various public testbeds. Over the course of 9 months, we transferred a total of 26 petabytes of data. Scaling the measurement framework to capture a large number of errors proved to be a challenge. This paper focuses on the challenges encountered during the deployment of the measurement system. We also present the interim results, which suggest that the error problems seen in prior works may be caused by two distinct processes: (1) errors that slip past TCP and (2) file system failures. The interim results also suggest that the measurement system needs to be adjusted to collect exabytes of measurement data, rather than the petabytes that prior studies predicted.Keywords
Funding Information
- National Science Foundation (2019163,2126148)
This publication has 21 references indexed in Scilit:
- Science DMZ Networks: How Different Are They Really?2024
- Is It Time to Upgrade From CRC-32?2024
- In-Network Caching Assisted Error Recovery For File Transfers2022
- Analyzing WLCG File Transfer Errors Through Machine LearningComputing and Software for Big Science, 2022
- Silent Data Corruption Estimation and Mitigation Without Fault InjectionIEEE Canadian Journal of Electrical and Computer Engineering, 2022
- Be SMART, Save I/O: A Probabilistic Approach to Avoid Uncorrectable Errors in Storage Systems2022
- Concurrent and Robust End-to-End Data Integrity Verification Scheme for Flash-Based Storage DevicesIEEE Access, 2022
- FABRIC: A National-Scale Programmable Experimental Network InfrastructureIEEE Internet Computing, 2019
- Towards Securing Data Transfers Against Silent Data Corruption2019
- A Comprehensive Tutorial on Science DMZIEEE Communications Surveys & Tutorials, 2018