Brian Pulfer
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Brian Pulfer has a total of 13 co-authors. Here are the top co-authors by number of shared publications:
- Yury Belousov
Department of Computer Science University of Geneva Switzerland
9 shared publications - Roman Chaban
Department of Computer Science University of Geneva Switzerland
6 shared publications - Slava Voloshynovskiy
University of Geneva
6 shared publications - Joakim Tutt
Department of Computer Science University of Geneva Switzerland
5 shared publications - Olga Taran
University Health Network
5 shared publications Slava Voloshynovskiy
Department of Computer Science University of Geneva Switzerland
3 shared publications
Latest Publications
- Open Access
Beyond Classification: Evaluating Diffusion Denoised Smoothing for Security-Utility Trade off
While foundation models demonstrate impressive performance across various tasks, they remain vulnerable to adversarial inputs. Current research explores various approaches to enhance model robustness, with Diffusion Denoised Smoothing emerging as a particularly promising technique. This method employs a pretrained diffusion model to preprocess inputs before model inference. Yet, its effectiveness remains largely unexplored beyond classification. We aim to address this gap by analyzing three datasets with four distinct downstream tasks under three different adversarial attack algorithms. Our findings reveal that while foundation models maintain resilience against conventional transformations, applying high-noise diffusion denoising to clean images without any distortions significantly degrades performance by as high as 57%. Low-noise diffusion settings preserve performance but fail to provide adequate protection across all attack types. Moreover, we introduce a novel attack strategy specifically targeting the diffusion process itself, capable of circumventing defenses in the low-noise regime. Our results suggest that the trade-off between adversarial robustness and performance remains a challenge to be addressed.
Open AccessBeyond Classification: Evaluating Diffusion Denoised Smoothing for Security-Utility Trade off
While foundation models demonstrate impressive performance across various tasks, they remain vulnerable to adversarial inputs. Current research explores various approaches to enhance model robustness, with Diffusion Denoised Smoothing emerging as a particularly promising technique. This method employs a pretrained diffusion model to preprocess inputs before model inference. Yet, its effectiveness remains largely unexplored beyond classification. We aim to address this gap by analyzing three datasets with four distinct downstream tasks under three different adversarial attack algorithms. Our findings reveal that while foundation models maintain resilience against conventional transformations, applying high-noise diffusion denoising to clean images without any distortions significantly degrades performance by as high as 57%. Low-noise diffusion settings preserve performance but fail to provide adequate protection across all attack types. Moreover, we introduce a novel attack strategy specifically targeting the diffusion process itself, capable of circumventing defenses in the low-noise regime. Our results suggest that the trade-off between adversarial robustness and performance remains a challenge to be addressed.
- Open Access
Robustness Tokens: Towards Adversarial Robustness of Transformers
Recently, large pre-trained foundation models have become widely adopted by machine learning practitioners for a multitude of tasks. Given that such models are publicly available, relying on their use as backbone models for downstream tasks might result in high vulnerability to adversarial attacks crafted with the same public model. In this work, we propose Robustness Tokens, a novel approach specific to the transformer architecture that fine-tunes a few additional private tokens with low computational requirements instead of tuning model parameters as done in traditional adversarial training. We show that Robustness Tokens make Vision Transformer models significantly more robust to white-box adversarial attacks while also retaining the original downstream performances.
Open AccessRobustness Tokens: Towards Adversarial Robustness of Transformers
Recently, large pre-trained foundation models have become widely adopted by machine learning practitioners for a multitude of tasks. Given that such models are publicly available, relying on their use as backbone models for downstream tasks might result in high vulnerability to adversarial attacks crafted with the same public model. In this work, we propose Robustness Tokens, a novel approach specific to the transformer architecture that fine-tunes a few additional private tokens with low computational requirements instead of tuning model parameters as done in traditional adversarial training. We show that Robustness Tokens make Vision Transformer models significantly more robust to white-box adversarial attacks while also retaining the original downstream performances.
- 21 November 2024
- Published by Springer Nature
- p. 110-127
Robustness Tokens: Towards Adversarial Robustness of Transformers
- 21 November 2024
- Published by Springer Nature
- p. 110-127
Robustness Tokens: Towards Adversarial Robustness of Transformers
- Open Access
Assessing the Viability of Synthetic Physical Copy Detection Patterns on Different Imaging Systems
This paper explores the potential of synthetic physical Copy Detection Patterns (CDP) to improve the robustness of anti-counterfeiting systems. By leveraging synthetic physical CDP, we aim at enhancing security and cost-effectiveness across various real-world applications. Our research demonstrates that synthetic CDP offer substantial improvements in authentication accuracy compared to one based on traditional digital templates. We conducted extensive tests using both a scanner and a diverse range of mobile phones, validating our approach through ROC analysis. The results indicate that synthetic CDP can reliably differentiate between original and fake samples, making this approach a viable solution for real-world applications, though requires an additional research to make this technology scalable across a variety of imaging devices.
Open AccessAssessing the Viability of Synthetic Physical Copy Detection Patterns on Different Imaging Systems
This paper explores the potential of synthetic physical Copy Detection Patterns (CDP) to improve the robustness of anti-counterfeiting systems. By leveraging synthetic physical CDP, we aim at enhancing security and cost-effectiveness across various real-world applications. Our research demonstrates that synthetic CDP offer substantial improvements in authentication accuracy compared to one based on traditional digital templates. We conducted extensive tests using both a scanner and a diverse range of mobile phones, validating our approach through ROC analysis. The results indicate that synthetic CDP can reliably differentiate between original and fake samples, making this approach a viable solution for real-world applications, though requires an additional research to make this technology scalable across a variety of imaging devices.
- 2 February 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Forensics and Security
- Vol. 19 (15566013) , 3395-3408
A Machine Learning-Based Digital Twin for Anti-Counterfeiting Applications With Copy Detection Patterns
In this paper, we present a new approach to model a printing-imaging channel using a machine learning-based “digital twin” for copy detection patterns (CDP). The CDP are considered as modern anti-counterfeiting features in multiple applications. Our digital twin is formulated within the information-theoretic framework of TURBO initially developed for high energy physics simulations, using variational approximations of mutual information for both encoder and decoder in the bidirectional exchange of information. This model extends various architectural designs, including paired pix2pix and unpaired CycleGAN, for image-to-image translation. Applicable to any type of printing and imaging devices, the model needs only training data comprising digital templates sent to a printing device and data acquired by an imaging device. The data can be paired, unpaired, or hybrid, ensuring architectural flexibility and scalability for multiple practical setups. We explore the influence of various architectural factors, metrics, and discriminators on the overall system’s performance in generating and predicting printed CDP from their digital versions and vice versa. We also performed a comparison with several state-of-the-art methods for image-to-image translation applications. The simulation code and extended results are publicly available at https://gitlab.unige.ch/sip-group/digital-twin.
- 2 February 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Forensics and Security
- Vol. 19 (15566013) , 3395-3408
A Machine Learning-Based Digital Twin for Anti-Counterfeiting Applications With Copy Detection Patterns
In this paper, we present a new approach to model a printing-imaging channel using a machine learning-based “digital twin” for copy detection patterns (CDP). The CDP are considered as modern anti-counterfeiting features in multiple applications. Our digital twin is formulated within the information-theoretic framework of TURBO initially developed for high energy physics simulations, using variational approximations of mutual information for both encoder and decoder in the bidirectional exchange of information. This model extends various architectural designs, including paired pix2pix and unpaired CycleGAN, for image-to-image translation. Applicable to any type of printing and imaging devices, the model needs only training data comprising digital templates sent to a printing device and data acquired by an imaging device. The data can be paired, unpaired, or hybrid, ensuring architectural flexibility and scalability for multiple practical setups. We explore the influence of various architectural factors, metrics, and discriminators on the overall system’s performance in generating and predicting printed CDP from their digital versions and vice versa. We also performed a comparison with several state-of-the-art methods for image-to-image translation applications. The simulation code and extended results are publicly available at https://gitlab.unige.ch/sip-group/digital-twin.
- 29 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Forensics and Security
- Vol. 19 (15566013) , 3124-3134
Authentication of Copy Detection Patterns: A Pattern Reliability Based Approach
Copy Detection Pattern (CDP) technology is a promising anti-counterfeiting solution for the protection of physical goods. In recent years, it has been shown that this technology is threatened by powerful deep learning attacks that are able to bypass original authentication schemes. In this paper, we tackle this problem by proposing a new CDP authentication scheme based on statistical knowledge discovered about the printing and imaging process. The novelty of our approach lies in providing means to measure the reliability of each local pattern appearing in the CDP. This allows to define new authentication measures to better differentiate original CDP from fakes. Our results show that this new system is capable of performing reliable CDP authentication with smartphones without the need for heavyweight machine learning tools requiring massive data entries.
- 29 January 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Forensics and Security
- Vol. 19 (15566013) , 3124-3134
Authentication of Copy Detection Patterns: A Pattern Reliability Based Approach
Copy Detection Pattern (CDP) technology is a promising anti-counterfeiting solution for the protection of physical goods. In recent years, it has been shown that this technology is threatened by powerful deep learning attacks that are able to bypass original authentication schemes. In this paper, we tackle this problem by proposing a new CDP authentication scheme based on statistical knowledge discovered about the printing and imaging process. The novelty of our approach lies in providing means to measure the reliability of each local pattern appearing in the CDP. This allows to define new authentication measures to better differentiate original CDP from fakes. Our results show that this new system is capable of performing reliable CDP authentication with smartphones without the need for heavyweight machine learning tools requiring massive data entries.
- Open Access
- 2 May 2023
- Published by Springer Nature in Empirical Software Engineering
- Vol. 28 (3) , 1-26
Model vs system level testing of autonomous driving systems: a replication and extension study
Offline model-level testing of autonomous driving software is much cheaper, faster, and diversified than in-field, online system-level testing. Hence, researchers have compared empirically model-level vs system-level testing using driving simulators. They reported the general usefulness of simulators at reproducing the same conditions experienced in-field, but also some inadequacy of model-level testing at exposing failures that are observable only in online mode. In this work, we replicate the reference study on model vs system-level testing of autonomous vehicles while acknowledging several assumptions that we had reconsidered. These assumptions are related to several threats to validity affecting the original study that motivated additional analysis and the development of techniques to mitigate them. Moreover, we also extend the replicated study by evaluating the original findings when considering a physical, radio-controlled autonomous vehicle. Our results show that simulator-based testing of autonomous driving systems yields predictions that are close to the ones of real-world datasets when using neural-based translation to mitigate the reality gap induced by the simulation platform. On the other hand, model-level testing failures are in line with those experienced at the system level, both in simulated and physical environments, when considering the pre-failure site, similar-looking images, and accurate labels.
- 2 May 2023
- Published by Springer Nature in Empirical Software Engineering
- Vol. 28 (3) , 1-26
Open AccessModel vs system level testing of autonomous driving systems: a replication and extension study
Offline model-level testing of autonomous driving software is much cheaper, faster, and diversified than in-field, online system-level testing. Hence, researchers have compared empirically model-level vs system-level testing using driving simulators. They reported the general usefulness of simulators at reproducing the same conditions experienced in-field, but also some inadequacy of model-level testing at exposing failures that are observable only in online mode. In this work, we replicate the reference study on model vs system-level testing of autonomous vehicles while acknowledging several assumptions that we had reconsidered. These assumptions are related to several threats to validity affecting the original study that motivated additional analysis and the development of techniques to mitigate them. Moreover, we also extend the replicated study by evaluating the original findings when considering a physical, radio-controlled autonomous vehicle. Our results show that simulator-based testing of autonomous driving systems yields predictions that are close to the ones of real-world datasets when using neural-based translation to mitigate the reality gap induced by the simulation platform. On the other hand, model-level testing failures are in line with those experienced at the system level, both in simulated and physical environments, when considering the pre-failure site, similar-looking images, and accurate labels.
- Open Access
Mathematical model of printing-imaging channel for blind detection of fake copy detection patterns
Nowadays, copy detection patterns (CDP) appear as a very promising anti-counterfeiting technology for physical object protection. However, the advent of deep learning as a powerful attacking tool has shown that the general authentication schemes are unable to compete and fail against such attacks. In this paper, we propose a new mathematical model of printing-imaging channel for the authentication of CDP together with a new detection scheme based on it. The results show that even deep learning created copy fakes unknown at the training stage can be reliably authenticated based on the proposed approach and using only digital references of CDP during authentication.
Open AccessMathematical model of printing-imaging channel for blind detection of fake copy detection patterns
Nowadays, copy detection patterns (CDP) appear as a very promising anti-counterfeiting technology for physical object protection. However, the advent of deep learning as a powerful attacking tool has shown that the general authentication schemes are unable to compete and fail against such attacks. In this paper, we propose a new mathematical model of printing-imaging channel for the authentication of CDP together with a new detection scheme based on it. The results show that even deep learning created copy fakes unknown at the training stage can be reliably authenticated based on the proposed approach and using only digital references of CDP during authentication.
- Open Access
- 12 December 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Digital twins of physical printing-imaging channel
In this paper, we address the problem of modeling a printing-imaging channel built on a machine learning approach a.k.a. digital twin for anti-counterfeiting applications based on copy detection patterns (CDP). The digital twin is formulated on an information-theoretic framework called Turbo that uses variational approximations of mutual information developed for both encoder and decoder in two-directional information passage. The proposed model generalizes several state-of-the-art architectures such as adversarial autoencoder (AAE) [1], CycleGAN [2] and adversarial latent space auto-encoder (ALAE) [3]. This model can be applied to any type of printing and imaging and it only requires training data consisting of digital templates or artworks that are sent to a printing device and data acquired by an imaging device. Moreover, these data can be paired, unpaired or hybrid paired-unpaired which makes the proposed architecture very flexible and scalable to many practical setups. We demonstrate the impact of various architectural factors, metrics and discriminators on the overall system performance in the task of generation/prediction of printed CDP from their digital counterparts and vice versa. We also compare the proposed system with several state-of-the-art methods used for image-to-image translation applications. The code and extended results of the simulation are publicly available 1 .
- 12 December 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Open AccessDigital twins of physical printing-imaging channel
In this paper, we address the problem of modeling a printing-imaging channel built on a machine learning approach a.k.a. digital twin for anti-counterfeiting applications based on copy detection patterns (CDP). The digital twin is formulated on an information-theoretic framework called Turbo that uses variational approximations of mutual information developed for both encoder and decoder in two-directional information passage. The proposed model generalizes several state-of-the-art architectures such as adversarial autoencoder (AAE) [1], CycleGAN [2] and adversarial latent space auto-encoder (ALAE) [3]. This model can be applied to any type of printing and imaging and it only requires training data consisting of digital templates or artworks that are sent to a printing device and data acquired by an imaging device. Moreover, these data can be paired, unpaired or hybrid paired-unpaired which makes the proposed architecture very flexible and scalable to many practical setups. We demonstrate the impact of various architectural factors, metrics and discriminators on the overall system performance in the task of generation/prediction of printed CDP from their digital counterparts and vice versa. We also compare the proposed system with several state-of-the-art methods used for image-to-image translation applications. The code and extended results of the simulation are publicly available 1 .
- Open Access
- 12 December 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Printing variability of copy detection patterns
Copy detection pattern (CDP) is a novel solution for products’ protection against counterfeiting, which gains its popularity. CDP attracts industries due to its numerous benefits in comparison to alternative protection techniques. Besides its attractiveness, there is an essential gap in the fundamental analysis of CDP authentication performance in large-scale industrial applications. It concerns variability of CDP parameters under different production conditions that include a type of printer, substrate, printing resolution, etc. Since digital off-set printing represents great flexibility in terms of product personalized in comparison with the traditional off-set printing, it looks very interesting to address the above concerns for digital off-set printers that are used by several companies for the CDP protection of physical objects. In this paper, we thoroughly investigate certain factors impacting CDP. The experimental results obtained during our study reveal some previously unknown results and raise new and even more challenging questions. The results prove that it is a matter of great importance to choose carefully the substrate or printer for CDP production. This paper presents a new dataset produced by two industrial HP Indigo printers. The deviations of printed CDP from their digital templates, from which they have been produced, are chosen as a simple measure of similarity in our study. We found several particularities that might be of interest for large-scale industrial applications.
- 12 December 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Open AccessPrinting variability of copy detection patterns
Copy detection pattern (CDP) is a novel solution for products’ protection against counterfeiting, which gains its popularity. CDP attracts industries due to its numerous benefits in comparison to alternative protection techniques. Besides its attractiveness, there is an essential gap in the fundamental analysis of CDP authentication performance in large-scale industrial applications. It concerns variability of CDP parameters under different production conditions that include a type of printer, substrate, printing resolution, etc. Since digital off-set printing represents great flexibility in terms of product personalized in comparison with the traditional off-set printing, it looks very interesting to address the above concerns for digital off-set printers that are used by several companies for the CDP protection of physical objects. In this paper, we thoroughly investigate certain factors impacting CDP. The experimental results obtained during our study reveal some previously unknown results and raise new and even more challenging questions. The results prove that it is a matter of great importance to choose carefully the substrate or printer for CDP production. This paper presents a new dataset produced by two industrial HP Indigo printers. The deviations of printed CDP from their digital templates, from which they have been produced, are chosen as a simple measure of similarity in our study. We found several particularities that might be of interest for large-scale industrial applications.