Deep learning, built on two decades of signal processing
I worked at Ford Motor Company from 1998 to 2023. From 2015 to 2023 I was an AI/ML research scientist at Greenfield Labs in Palo Alto, leading research on end-to-end deep neural networks for autonomous vehicle control, automatic speech recognition and natural language understanding, signal separation, and sound source identification — along with the GUI tooling that let other researchers design, train and test those networks in PyTorch and TensorFlow/Keras.
The two decades before that were digital signal processing applied to acoustics and vibration. My PhD and post-doctoral work were in active control of sound and vibration, which left me with a great deal of hands-on experience in real-time DSP, acoustic and vibration measurement, instrumentation, and sensor and actuator development. That background is the reason the machine learning work has always been grounded in the physics of the signal rather than in the framework of the month.
At Ford I led the development of numerous signal processing tools and metrics for speech signal quality and objective NVH measurement, and repeatedly built new processing approaches when the problem at hand was too complex for off-the-shelf analysis software. Much of that work went into user-friendly tools that let other engineers do their own work faster. I authored or co-authored twenty Research & Advanced Engineering technical reports and multiple Corporate Engineering Test Procedures, and received the Henry Ford Technological Award, four Global Customer Satisfaction Awards, a PD Engineering Excellence Award and eight others.
I retired from Ford in 2023 — from the company, not from the work. I now run Charette AI Group, researching and building applications in AI, machine learning and signal processing. Most of what has come out of it so far is free to download; commercialization runs through partnerships, which keeps the focus on the research and development itself.
I remain open to select work — consulting, advisory or research engagements — where analytical depth, machine learning and signal processing expertise, and a hands-on approach are what the problem actually calls for. Email is the best way to reach me.
Charette AI Group
Research in artificial intelligence applications, machine learning development and digital signal processing. Six projects so far — most of them free downloads, one still in development.
pySPWB
FreeSignal processing workbench — spectra, transfer functions, coherence, spectrograms and adaptive filtering. Runs on Windows, macOS and Linux, with or without Python installed.
RV Backup Helper
In developmentAn object recognition system integrated into the backup camera systems used on RVs.
CloakClip
FreeEncrypt and decrypt text with a shared password, straight from the clipboard — for private messages through chat, email or notes. Windows and macOS.
SAE Fractional Calculator
FreeDesktop calculator for imperial measurements: yards, feet, inches and fractions on an exact 1/32″ grid. Windows and macOS.
HTML Font Toolbar
FreeObsidian plugin — a floating toolbar that styles selected text with clean inline HTML: colors, highlights, sizes, fonts and alignment merged into a single span.
HTML Table Toolbar
FreeObsidian plugin — a Word-style toolbar for real HTML tables: rows and columns, merged and split cells, header rows, alignment, borders and cell shading.
Twenty-five years, eight roles
During my time at Ford Motor Company I had the chance to occupy a variety of positions and responsibilities. Highlights, most recent first:
Ford Motor Company
May 1998 – July 2023-
AI/ML Research Scientist
Led research in:
- End-to-end deep neural network controls for autonomous vehicles, i.e., steering and throttle.
- A variety of graphical user interface tools for deep neural network architecture design, training and testing, using Python with PyTorch or TensorFlow/Keras.
- Deep neural networks for natural language understanding and automatic speech recognition.
- Sound source identification using DNNs.
- Sensor signal processing, e.g., LiDAR, cameras, etc.
I also participated in evaluations of strategic technologies from Silicon Valley startups, and supported recruiting efforts to build a strong team in a very competitive talent landscape.
-
SPAVR Research Engineer
Team leader for the development and implementation of the Corporate Engineering Test Procedure (CETP) for hands-free speech quality processing of vehicle infotainment systems. The ability to do in-house evaluation of hands-free phone quality was a Ford first, and the effort resulted in innovative hands-free noise reduction algorithm improvements for in-vehicle speech quality processing.
Also lead developer of the Ford signal processing and advanced speech analysis toolbox (MATLAB based). The toolbox contains multiple user interfaces that manage, process and create (1 patent) large numbers of speech utterances with various background noise levels and types, supporting the development of speech recognition algorithms and the testing of speech recognition engines. Every interface was designed to be user-friendly and to make engineers more efficient at speech processing and speech data management.
Supported and analyzed all of the suppliers' noise suppression software evaluations, and used the results to guide the speech processing selection for SYNC Gen III, i.e., QNX.
-
S&R Core Engineer
Responsible for the development of innovative testing procedures, objective metrics, data acquisition and processing, and reporting for a wide variety of core production programs — sliding wedge rattle, window scrape, side door chuck, lift gates, and others. Developed and adapted objective metrics, methods and tools to the requirements of each project, and performed both road and lab measurements using the proving ground and the Hydraulic Road Simulator.
-
VSHC Core Engineer
Principal developer for the Vehicle Sensitivity Health Chart tools and infrastructure. Created the first comprehensive implementation of Vehicle Sensitivity data management, processing and reporting, reducing the time to process full Vehicle Sensitivity data from days to minutes.
-
Closures Lab Leader
Planned and scheduled the closures lab's noise and vibration measurements, processed the data, and adapted the signal processing and metrics where needed. This included in-vehicle microphone and accelerometer instrumentation, and measurements on several harsh road surfaces at various speeds.
-
Sound Quality Lab Leader
Responsible for the development and implementation of the Sound Quality lab's Corporate Engineering Test Procedures (CETP), along with data processing, acquisition, archiving and management. Proactively created batch processing software that cut lab turnaround time — processing all the SQ closures data became at least five times faster.
-
NOMAD Team Leader
Led the development and implementation of assembly plant end-of-line acoustic and vibration analysis tools and software for NVH/S&R Objective Measurements, Analysis and Diagnostic — the NOMAD project.
-
S&R Core Engineer
Principal technical developer for various NVH/S&R tools. Received the Henry Ford Technological Award for the technical contributions behind these tools (4 patents plus a CETP).
-
NVH Engineer
Lead engineer for Buzz, Squeak & Rattle (BSR) measurements and diagnostics across multiple vehicle programs — sliding wedge rattle, window scrape, side doors, lift gates, door chuck. Developed and adapted objective metrics, methods and tools to each project's needs.
Active control of sound and vibration
-
Postdoctoral research
I worked on multiple projects in active control of sound and vibration:
- Led the development and lab experimentation of an active trim panel for the reduction of separation flow noise in airplane cockpits (NASA funded project).
- Lead developer of various real-time controllers — signal processing on DSP boards with GUI interfaces — and multiple data acquisition systems for the Vibration and Acoustic Laboratories research group (VAL).
- Led the signal processing development for noise source separation and identification in motor vehicle cabins using the Least Mean Square (LMS) algorithm.
- Developed electro-mechanical adaptive vibration absorbers for sound minimization, designing and implementing two different types of tunable absorber. The lab work demonstrated that the greatest sound reduction from a vibration absorber is not necessarily obtained when the absorber is tuned to minimize vibration, but rather through an acoustic cost function such as sound radiation minimization — which leaves the absorber detuned relative to the structure's vibration.
Degrees
PhD, Mechanical Engineering
Active Control of Sound Radiation from Plane Structures Using PVDF Volume Displacement Sensors — Innovative volume displacement sensors made of shaped PVDF film strips were developed for beams and plates with arbitrary boundary conditions. Multiple variational models of piezoelectric actuation and PVDF film sensing were developed and experimentally verified, then implemented in active control systems minimizing the volume displacement of the structure.
MS, Mechanical Engineering
Bending Energy Dissipation of Simplified Single-Layer Stranded Cable — A simplified single-layer stranded cable model was developed to compute the energy dissipated by friction between strand components. Relative displacement between wire and core was determined from contact mechanics, and predicted values compared against tests on overhead transmission line conductors in free-field conditions.
BS, Mechanical Engineering
Scholarships
- 1994–95FCAR Scholarship
- 1993Sherbrooke University Institutional Scholarship
- 1990–92FCAR Scholarship
What I work with
Tools and domains I have used in production research work.
Machine learning & AI
Signal processing & acoustics
Instrumentation & data
Practice
Awards
- 2022AI/ML Virtual Conference Excellence Award (Visual Path Navigation)
- 2020AI/ML Virtual Conference Excellence Award (One-Shot Learning)
- 2020AI/ML Virtual Conference Excellence Award (Platooning)
- 2019R&AE Recognition Award for Girls Who Code program support
- 2018R&AE Recognition Award for Girls Who Code program support
- 2015Global Customer Satisfaction Award
- 2013I.T. Innovation Contest Award
- 2012Peer Recognition Award
- 2011PD Engineering Excellence Award
- 2010Global Customer Satisfaction Award
- 2009VE Quality Award
- 2007Global Customer Satisfaction Award
- 2005VASE Technical Achievement Award
- 2002Global Customer Satisfaction Award
- 1999Henry Ford Technological Award
Patents & publications
Twenty-nine granted U.S. patents and seventy-one publications going back to 1993. Titles marked PDF link to the full document.
Patents 29 granted
- Efficient Neural Networks
- Vehicle Neural Network Enhancement
- Joint Automatic Speech Recognition and Text-to-Speech Conversion Using Adversarial Neural Networks
- Hierarchical Encoder for Speech Conversion System
- Vehicle Neural Network
- Natural Speech Data Generation Systems and Methods
- Vehicle Language Processing
- Neural Network Generative Modeling to Transform Speech Utterances and Augment Training Data
- Vehicle Language Processing
- Vehicle Language Processing
- Multi-Tier Network for Task-Oriented Deep Neural Network
- Vehicle Window Transmittance Control Apparatus and Method
- Adaptive Vehicle State-Based Hands-Free Phone Noise Reduction with Learning Capability
- Acoustic and Domain-Based Speech Recognition for Vehicles
- Multi-Sensor Precipitation Classification Apparatus and Method
- Vehicle Having Dynamic Acoustic Model Switching to Improve Noisy Speech Recognition
- Autonomous Police Vehicle
- Autonomous-Vehicle-Control System and Method Incorporating Occupant Preferences
- Vehicle-Window-Transmittance Control Apparatus and Method
- Detecting Physical Threats Approaching a Vehicle
- Method and Apparatus for Tuning Speech Recognition Systems to Accommodate Ambient Noise
- Acoustic Impulse Response Simulation
- Method and Apparatus for Objective Measurement of Noise
- Method and System for Assessing a Refrigerant Charge Level in a Vehicle Air Conditioning System
- Method and System to Detect Unwanted Noise
- Sound Detector Device
- Method and Apparatus for Identifying Sound in a Composite Sound Signal
- System for Diagnosing Unexpected Noise in a Multi-Component Assembly
- Method for Determining and Reproducing Noise Inducing Vibrations in a Multi-Component Assembly
Articles & preprints 16
- A Follow-the-Leader Strategy using Hierarchical Deep Neural Networks with Grouped Convolutions
- Hierarchical Sequence to Sequence Voice Conversion with Limited Data
- Hierarchical Multi-task Deep Neural Network Architecture for End-to-End Driving
- The Impact of Microphone Location and Beamforming on In-Vehicle Speech Recognition
- Validation of In-Vehicle Speech Recognition Using Synthetic Mixing
- Quantifying Hands-Free Call Quality in an Automobile
- Minimization of Sound Radiation from Plates using Adaptive Tuned Vibration Absorbers
- Squeaks and Rattles Portable Excitation System
- Bending Energy Dissipation of Simplified Single-Layer Stranded Cable
- Active Control of Sound Radiation from a Plate using a PVDF Volume Displacement Sensor
- Dynamic Effects of Piezoelectric Actuators on the Vibrational Response of a Plate
- Active Control of Finite Beam Volume Velocity using Shaped PVDF Sensor
- Variational Analysis of a Thin Finite Beam Excitation with a Single Asymmetric Piezoelectric Actuator Including Bonding Layer and Dynamical Effects
- Active Control of Planar Structures Volume Displacement using Piezoelectric Transducers
- Asymmetric Actuation and Sensing of a Beam using Piezoelectric Materials
- Analysis of Stranded Cable Damping Characteristics under Cycling Bending
Technical reports 20 · Ford Corporate Library
- Deep Generative Models for Data Synthesis
- Deep Neural Networks for End-to-End Controls
- Deep Neural Networks for Sound and Speech Applications
- 1/10th Scale Vehicle Educational Framework for Autonomous Vehicles and ADAS Technologies
- Detection of Lane-Splitting Motorcycles
- Multiple Microphones for Automatic Speech Recognition Improvement
- Mobile Far-End Sound Quality: An On-Road System for Recording and Analyzing Phone Calls
- Understanding the Lombard Effect and its Impact on In-Vehicle Speech Recognition
- Effects of Narrow Band Bluetooth on the VoCon Embedded Automatic Speech Recognition System
- Close Speech Utterances Convolution with Cabin Impulse Response Validation using ASR Word Error Rate
- SYNC Microphone Location Optimization for P552 Vehicle and Recommendation for all Vehicle Platforms
- SYNC Microphone Location Optimization for CD-Car
- Development of Audio Chambers to Quantify Laboratory Repeatability and Reproducibility of In-Car Hands-Free Call Testing
- SYNC Microphone Location Optimization for B-Car
- Mobile Phone Mean Opinion Score (MOS) Listening Study
- 3QUEST Target Setting Using Vehicle Benchmarking
- SYNC Microphone Location Optimization in Vehicle Cabin
- Preliminary Evaluation of Hosiden Microphone Array for Vehicle SYNC Systems
- Ford Vehicles Benchmarking using 3QUEST Testing
- SYNC Microphone Signal Measurements
Conferences 24
- One-Shot Learning for Autonomous Vehicles
- A Follow-the-Leader Strategy using Hierarchical Deep Neural Networks with Grouped Convolutions
- Key-Value Retrieval Networks for Task-Oriented Dialogue
- The Impact of Microphone Location and Beamforming on In-Vehicle Speech Recognition
- Validation of In-Vehicle Speech Recognition Using Synthetic Mixing
- Background Noise Effects on Automotive Hands-free Phone Call Quality and Automatic Speech Recognition
- Signal Processing for Noise, Vibration and Harshness (NVH) Applications
- Squeaks and Rattles Automated Measurements at Assembly Plants
- Idle Vibration Objective Monitoring System for Assembly Plants
- Adaptive Vibration Absorbers for Control of Sound Radiation from Panels
- Control of Structural Sound Radiation using Multiple Detuned Vibration Absorbers
- Test of a Piezoceramic Actuator for the Active Control of Power Transformer Vibration
- Control of Sound Radiation from Panels using Multiple Globally Detuned Vibration Absorbers
- Active Enhancement of the Transmission Loss of a Panel via Minimization of the Volume Velocity: Theoretical and Experimental Results
- Active Control of Plate Volume Displacement: Simulation and Experimental Results
- Volume Velocity Sensors for Plates
- Active Control of Plate Volume Displacement using Shaped Strips of PVDF Film as Sensor
- Active Control of Sound by Minimization of Volume Velocity on Finite Beam
- Active Control of Plate Volume Velocity using Shaped PVDF Sensors
- Active Control of Structural Volume Velocity using Shaped PVDF Sensors
- Active Control of Plate Volume Velocity using Shaped PVDF Sensors
- Contrôle actif du bruit par minimisation du débit volumique de structure : analyse et expérience
- Asymmetric Actuation and Sensing of a Beam using Piezoelectric Materials
- Asymmetric Piezoelectric Actuation and Sensing of a Beam
Workshops 11
- Quantification of the Impact of Lombard Speech on In-Vehicle Speech Recognition
- Speech Recognition Robustness Studies at Ford Motor Company
- Reduction of Flow Noise in an Airplane Cockpit using an Active Trim Panel
- Design and Implementation of Adaptable Tuned Vibration Absorbers for Control of Sound Radiation
- Active Control of a Cockpit Trim Panel
- LabVIEW-Based Interfaces for DSP Boards
- Design and Implementation of Mechanical and Solid State Adaptable Tuned Vibration Absorbers for Control of Sound Radiation
- Noise Source Separation System using the LMS Algorithm
- Adaptively Tuned/Detuned Vibration Absorbers for Control of Sound Radiation from Structures
- Adaptive Tuned Vibration Absorbers for Sound Minimization
- Short Survey of the Current Projects Related to Active Control of Sound in Aircraft at VAL