EcoVal: An Efficient Data Valuation Framework for Machine Learning
Open Access
- 24 August 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 2866-2875
Abstract
Quantifying the value of data within a machine learning workflow can play a pivotal role in making more strategic decisions in machine learning initiatives. The existing Shapley value based frameworks for data valuation in machine learning are computationally expensive as they require considerable amount of repeated training of the model to obtain the Shapley value. In this paper, we introduce an efficient data valuation framework EcoVal, to estimate the value of data for machine learning models in a fast and practical manner. Instead of directly working with individual data sample, we determine the value of a cluster of similar data points. This value is further propagated amongst all the member cluster points. We show that the overall value of the data can be determined by estimating the intrinsic and extrinsic value of each data. This is enabled by formulating the performance of a model as aproduction function, a concept which is popularly used to estimate the amount of output based on factors like labor and capital in a traditional free economic market. We provide a formal proof of our valuation technique and elucidate the principles and mechanisms that enable its accelerated performance. We demonstrate the real-world applicability of our method by showcasing its effectiveness for both in-distribution and out-of-sample data. This work addresses one of the core challenges of efficient data valuation at scale in machine learning models. The code is available at https://github.com/respai-lab/ecoval.Keywords
This publication has 10 references indexed in Scilit:
- Nonrivalry and the Economics of DataAmerican Economic Review, 2020
- A Marketplace for Data2019
- Shapley Meets Uniform: An Axiomatic Framework for Attribution in Online Advertising2019
- A Feature Selection Method Based on Shapley Value to False Alarm Reduction in ICUs A Genetic-Algorithm Approach2018
- Centralizing Inventory in Supply Chains by Using Shapley Value to Allocate the ProfitsManufacturing & Service Operations Management, 2011
- An axiomatic approach to the concept of interaction among players in cooperative gamesInternational Journal of Game Theory, 1999
- On weighted Shapley valuesInternational Journal of Game Theory, 1987
- Value Theory Without EfficiencyMathematics of Operations Research, 1981
- The Law of Diminishing Returns1974
- 17. A Value for n-Person Games1953