Kenneth Chaney
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Kenneth Chaney has a total of 38 co-authors. Here are the top co-authors by number of shared publications:
- Kostas Daniilidis
University of Pennsylvania
10 shared publications - Alex Zihao Zhu
University of Pennsylvania
4 shared publications - Ziyun Wang
University of Pennsylvania
4 shared publications - Friedhelm Hamann
Technical University of Berlin
2 shared publications - Guillermo Gallego
Technical University of Berlin
2 shared publications - Ioannis Asmanis
University of Pennsylvania
2 shared publications
Latest Publications
- 28 October 2024
- Published by Springer Nature
- p. 18-37
Motion-Prior Contrast Maximization for Dense Continuous-Time Motion Estimation
- 28 October 2024
- Published by Springer Nature
- p. 18-37
Motion-Prior Contrast Maximization for Dense Continuous-Time Motion Estimation
- Open Access
Motion-prior Contrast Maximization for Dense Continuous-Time Motion Estimation
Current optical flow and point-tracking methods rely heavily on synthetic datasets. Event cameras are novel vision sensors with advantages in challenging visual conditions, but state-of-the-art frame-based methods cannot be easily adapted to event data due to the limitations of current event simulators. We introduce a novel self-supervised loss combining the Contrast Maximization framework with a non-linear motion prior in the form of pixel-level trajectories and propose an efficient solution to solve the high-dimensional assignment problem between non-linear trajectories and events. Their effectiveness is demonstrated in two scenarios: In dense continuous-time motion estimation, our method improves the zero-shot performance of a synthetically trained model on the real-world dataset EVIMO2 by 29%. In optical flow estimation, our method elevates a simple UNet to achieve state-of-the-art performance among self-supervised methods on the DSEC optical flow benchmark. Our code is available at https://github.com/tub-rip/MotionPriorCMax.
Open AccessMotion-prior Contrast Maximization for Dense Continuous-Time Motion Estimation
Current optical flow and point-tracking methods rely heavily on synthetic datasets. Event cameras are novel vision sensors with advantages in challenging visual conditions, but state-of-the-art frame-based methods cannot be easily adapted to event data due to the limitations of current event simulators. We introduce a novel self-supervised loss combining the Contrast Maximization framework with a non-linear motion prior in the form of pixel-level trajectories and propose an efficient solution to solve the high-dimensional assignment problem between non-linear trajectories and events. Their effectiveness is demonstrated in two scenarios: In dense continuous-time motion estimation, our method improves the zero-shot performance of a synthetically trained model on the real-world dataset EVIMO2 by 29%. In optical flow estimation, our method elevates a simple UNet to achieve state-of-the-art performance among self-supervised methods on the DSEC optical flow benchmark. Our code is available at https://github.com/tub-rip/MotionPriorCMax.
- Open Access
EvAC3D: From Event-based Apparent Contours to 3D Models via Continuous Visual Hulls
3D reconstruction from multiple views is a successful computer vision field with multiple deployments in applications. State of the art is based on traditional RGB frames that enable optimization of photo-consistency cross views. In this paper, we study the problem of 3D reconstruction from event-cameras, motivated by the advantages of event-based cameras in terms of low power and latency as well as by the biological evidence that eyes in nature capture the same data and still perceive well 3D shape. The foundation of our hypothesis that 3D reconstruction is feasible using events lies in the information contained in the occluding contours and in the continuous scene acquisition with events. We propose Apparent Contour Events (ACE), a novel event-based representation that defines the geometry of the apparent contour of an object. We represent ACE by a spatially and temporally continuous implicit function defined in the event x-y-t space. Furthermore, we design a novel continuous Voxel Carving algorithm enabled by the high temporal resolution of the Apparent Contour Events. To evaluate the performance of the method, we collect MOEC-3D, a 3D event dataset of a set of common real-world objects. We demonstrate the ability of EvAC3D to reconstruct high-fidelity mesh surfaces from real event sequences while allowing the refinement of the 3D reconstruction for each individual event.
Open AccessEvAC3D: From Event-based Apparent Contours to 3D Models via Continuous Visual Hulls
3D reconstruction from multiple views is a successful computer vision field with multiple deployments in applications. State of the art is based on traditional RGB frames that enable optimization of photo-consistency cross views. In this paper, we study the problem of 3D reconstruction from event-cameras, motivated by the advantages of event-based cameras in terms of low power and latency as well as by the biological evidence that eyes in nature capture the same data and still perceive well 3D shape. The foundation of our hypothesis that 3D reconstruction is feasible using events lies in the information contained in the occluding contours and in the continuous scene acquisition with events. We propose Apparent Contour Events (ACE), a novel event-based representation that defines the geometry of the apparent contour of an object. We represent ACE by a spatially and temporally continuous implicit function defined in the event x-y-t space. Furthermore, we design a novel continuous Voxel Carving algorithm enabled by the high temporal resolution of the Apparent Contour Events. To evaluate the performance of the method, we collect MOEC-3D, a 3D event dataset of a set of common real-world objects. We demonstrate the ability of EvAC3D to reconstruct high-fidelity mesh surfaces from real event sequences while allowing the refinement of the 3D reconstruction for each individual event.
- 13 November 2022
- Published by Springer Nature
- p. 284-299
EvAC3D: From Event-Based Apparent Contours to 3D Models via Continuous Visual Hulls
- 13 November 2022
- Published by Springer Nature
- p. 284-299
EvAC3D: From Event-Based Apparent Contours to 3D Models via Continuous Visual Hulls
- Open Access
- 1 March 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in Field Robotics
- Vol. 2, 147-171
Semantic Keypoint-Based Pose Estimation from Single RGB Frames
- 1 March 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in Field Robotics
- Vol. 2, 147-171
Open AccessSemantic Keypoint-Based Pose Estimation from Single RGB Frames
- 7 October 2020
- Published by Springer Nature
- p. 366-382
Spike-FlowNet: Event-Based Optical Flow Estimation with Energy-Efficient Hybrid Neural Networks
- 7 October 2020
- Published by Springer Nature
- p. 366-382
Spike-FlowNet: Event-Based Optical Flow Estimation with Energy-Efficient Hybrid Neural Networks
- Open Access
Spike-FlowNet: Event-based Optical Flow Estimation with Energy-Efficient Hybrid Neural Networks
Event-based cameras display great potential for a variety of tasks such as high-speed motion detection and navigation in low-light environments where conventional frame-based cameras suffer critically. This is attributed to their high temporal resolution, high dynamic range, and low-power consumption. However, conventional computer vision methods as well as deep Analog Neural Networks (ANNs) are not suited to work well with the asynchronous and discrete nature of event camera outputs. Spiking Neural Networks (SNNs) serve as ideal paradigms to handle event camera outputs, but deep SNNs suffer in terms of performance due to the spike vanishing phenomenon. To overcome these issues, we present Spike-FlowNet, a deep hybrid neural network architecture integrating SNNs and ANNs for efficiently estimating optical flow from sparse event camera outputs without sacrificing the performance. The network is end-to-end trained with self-supervised learning on Multi-Vehicle Stereo Event Camera (MVSEC) dataset. Spike-FlowNet outperforms its corresponding ANN-based method in terms of the optical flow prediction capability while providing significant computational efficiency.
Open AccessSpike-FlowNet: Event-based Optical Flow Estimation with Energy-Efficient Hybrid Neural Networks
Event-based cameras display great potential for a variety of tasks such as high-speed motion detection and navigation in low-light environments where conventional frame-based cameras suffer critically. This is attributed to their high temporal resolution, high dynamic range, and low-power consumption. However, conventional computer vision methods as well as deep Analog Neural Networks (ANNs) are not suited to work well with the asynchronous and discrete nature of event camera outputs. Spiking Neural Networks (SNNs) serve as ideal paradigms to handle event camera outputs, but deep SNNs suffer in terms of performance due to the spike vanishing phenomenon. To overcome these issues, we present Spike-FlowNet, a deep hybrid neural network architecture integrating SNNs and ANNs for efficiently estimating optical flow from sparse event camera outputs without sacrificing the performance. The network is end-to-end trained with self-supervised learning on Multi-Vehicle Stereo Event Camera (MVSEC) dataset. Spike-FlowNet outperforms its corresponding ANN-based method in terms of the optical flow prediction capability while providing significant computational efficiency.
- 31 July 2020
- Published by Wiley in Developmental Dynamics
- Vol. 249 (9) , 1147-1165
Tissue‐specific expression of ribosomal protein paralogue eRpL22‐like in Drosophila melanogaster eye development
Background: Differences in core or tissue‐specific ribosomal protein (Rp) composition within ribosomes contribute to ribosome heterogeneity and functional variability. Yet, the degree to which ribosome heterogeneity modulates development is unknown. The Drosophila melanogaster eRpL22 family contains structurally diverse paralogues, eRpL22 and eRpL22‐like. Unlike ubiquitously expressed eRpL22, eRpL22‐like expression is tissue‐specific, notably within the male germline and the eye. We investigated expression within the developing eye to uncover tissue/cell types where specific paralogue roles might be defined.Results: Immunohistochemistry analysis confirms ubiquitous eRpL22 expression throughout eye development. In larvae, eRpL22‐like is ubiquitously expressed, but highly enriched in the peripodial epithelium (PE). In early pupae, eRpL22‐like is broadly distributed in multiple cell types, but later, is primarily enriched in interommatidial hair cells (IoHC). Adult patterns include the ring of accessory cells around ommatidia. Adult retinae IoHC patterning phenotypes (shown by scanning electron microscopy) may be linked to RNAi‐mediated eRpL22‐like depletion within larval PE. Immunoblots and polysome profile analyses show multiple variants of eRpL22‐like across development, with the variant at the expected molecular mass co‐sedimenting with active ribosomes.Conclusion: Our data reveal differential patterns of eRpL22‐like expression relative to eRpL22 and suggest a specific role for eRpL22‐like in developmental patterning of the eye.
- 31 July 2020
- Published by Wiley in Developmental Dynamics
- Vol. 249 (9) , 1147-1165
Tissue‐specific expression of ribosomal protein paralogue eRpL22‐like in Drosophila melanogaster eye development
Background: Differences in core or tissue‐specific ribosomal protein (Rp) composition within ribosomes contribute to ribosome heterogeneity and functional variability. Yet, the degree to which ribosome heterogeneity modulates development is unknown. The Drosophila melanogaster eRpL22 family contains structurally diverse paralogues, eRpL22 and eRpL22‐like. Unlike ubiquitously expressed eRpL22, eRpL22‐like expression is tissue‐specific, notably within the male germline and the eye. We investigated expression within the developing eye to uncover tissue/cell types where specific paralogue roles might be defined.Results: Immunohistochemistry analysis confirms ubiquitous eRpL22 expression throughout eye development. In larvae, eRpL22‐like is ubiquitously expressed, but highly enriched in the peripodial epithelium (PE). In early pupae, eRpL22‐like is broadly distributed in multiple cell types, but later, is primarily enriched in interommatidial hair cells (IoHC). Adult patterns include the ring of accessory cells around ommatidia. Adult retinae IoHC patterning phenotypes (shown by scanning electron microscopy) may be linked to RNAi‐mediated eRpL22‐like depletion within larval PE. Immunoblots and polysome profile analyses show multiple variants of eRpL22‐like across development, with the variant at the expected molecular mass co‐sedimenting with active ribosomes.Conclusion: Our data reveal differential patterns of eRpL22‐like expression relative to eRpL22 and suggest a specific role for eRpL22‐like in developmental patterning of the eye.
- Open Access
Minibatch Processing in Spiking Neural Networks
Spiking neural networks (SNNs) are a promising candidate for biologically-inspired and energy efficient computation. However, their simulation is notoriously time consuming, and may be seen as a bottleneck in developing competitive training methods with potential deployment on neuromorphic hardware platforms. To address this issue, we provide an implementation of mini-batch processing applied to clock-based SNN simulation, leading to drastically increased data throughput. To our knowledge, this is the first general-purpose implementation of mini-batch processing in a spiking neural networks simulator, which works with arbitrary neuron and synapse models. We demonstrate nearly constant-time scaling with batch size on a simulation setup (up to GPU memory limits), and showcase the effectiveness of large batch sizes in two SNN application domains, resulting in $\approx$880X and $\approx$24X reductions in wall-clock time respectively. Different parameter reduction techniques are shown to produce different learning outcomes in a simulation of networks trained with spike-timing-dependent plasticity. Machine learning practitioners and biological modelers alike may benefit from the drastically reduced simulation time and increased iteration speed this method enables. Code to reproduce the benchmarks and experimental findings in this paper can be found at https://github.com/djsaunde/snn-minibatch.
Open AccessMinibatch Processing in Spiking Neural Networks
Spiking neural networks (SNNs) are a promising candidate for biologically-inspired and energy efficient computation. However, their simulation is notoriously time consuming, and may be seen as a bottleneck in developing competitive training methods with potential deployment on neuromorphic hardware platforms. To address this issue, we provide an implementation of mini-batch processing applied to clock-based SNN simulation, leading to drastically increased data throughput. To our knowledge, this is the first general-purpose implementation of mini-batch processing in a spiking neural networks simulator, which works with arbitrary neuron and synapse models. We demonstrate nearly constant-time scaling with batch size on a simulation setup (up to GPU memory limits), and showcase the effectiveness of large batch sizes in two SNN application domains, resulting in $\approx$880X and $\approx$24X reductions in wall-clock time respectively. Different parameter reduction techniques are shown to produce different learning outcomes in a simulation of networks trained with spike-timing-dependent plasticity. Machine learning practitioners and biological modelers alike may benefit from the drastically reduced simulation time and increased iteration speed this method enables. Code to reproduce the benchmarks and experimental findings in this paper can be found at https://github.com/djsaunde/snn-minibatch.
- 23 January 2019
- Published by Springer Nature
- p. 711-714
Unsupervised Event-Based Optical Flow Using Motion Compensation
- 23 January 2019
- Published by Springer Nature
- p. 711-714
Unsupervised Event-Based Optical Flow Using Motion Compensation