EECE 522 Estimation Theory
This Course is Offered Spring of Even Years;
Next Offered Spring 2018
Addresses the theory and practice of estimating parameters for discrete-time
signals embedded in noise. Topics include:
- Classical Estimation (Deterministic Parameter)
- Cramer-Rao Lower Bound
- Minimum Variance Unbiased Estimation
- Least Squares Estimation
- Maximum Likelihood Estimation
- Bayesian Estimation (Random Parameter)
- Minimum Mean Square Estimation
- Maximum A Posteriori Estimation
- Optimal Filtering
- Wiener Filtering
- Kalman Filtering
- Radar, Sonar, and Emitter Location
- Communication Systems
This course is not for the
Must Have a Basic Understanding of:
Linear Algebra or Matrix Theory (see textbook appendix
& Reserve Books #1, 4)
Probability Theory and Random Functions (see textbook
appendix & Reserve Books #1, 2, 3)
Probability Density Functions
Joint, Marginal, Conditional Versions
Mean and Variance of Random Variables
Wide-Sense Stationary Random Processes
Correlation Function & Covariance Matrix
Power Spectral Density
Digital Signal Processing (see Reserve Book #2)
- Fourier Transform for Discrete-Time Signals
- Discrete-Time Filters (Mostly FIR - not design, but
operation via convolution)
- Fundamentals of Statistical Signal Processing, Volume I:
by Steven Kay (Published by
Other Books of Interest
- Parameter Estimation - H. Sorenson
- Covers same ground as textbook but in a different
order; thus, provides an interesting alternative view.
- Has appendices on Matrices and Probability Theory - a
little more detailed than textbook.
Signal Processing: Discrete Spectral Analysis, Detection, and
Estimation - M. Schwartz and L. Schaw
Ch. 2 Reviews Digital Signal Processing
Ch. 3 reviews Random Discrete-Time Signals
Ch. 6 gives concise coverage of Parameter Estimation (Classical
and Bayesian) as well as Wiener Filter
Ch. 7 covers Kalman Filters and has example of Aircraft Tracking
- Introduction to Random Signal Analysis and Kalman
Filtering - R. Brown
- Gives a good overview of probability and random
- Several Chapters on Kalman Filter
- Estimation Theory and Applications - N.
- An older book on estimation, but still might have
useful perspectives on parameter estimation
- BUT... mostly focused on state-estimation (e.g.,
Kalman Filter type stuff)
- HOWEVER... has a good section on Matrix Algebra and
- Applied Optimal Estimation - A. Gelb
- "THE BIBLE" for Kalman Filters - on the bookshelf of
virtually everyone working with Kalman Filters!
- Data Analysis: A Bayesian Tutorial - D.
- An Excellent, down-to-earth book on Bayesian
- Starts with Bayesian approach and shows how it
"degenerates" into classical methods (ML & LS)
- Mostly deals with problems of the scientific data
analysis sort but still very good for signal processing types
Relevant Papers & Other Material
For most of these
you can find them in the library.
I'll try to post most of them on
Only by reading papers in the area
can you really get a feeling for how this stuff works!
The following link gives some advice
on how to read technical papers:
How To Read Papers
- D. Torrieri, "Statistical Theory of Passive Location
Systems," IEEE Transactions on Aerospace and Electronic Systems, pp.
183 - 198, March 1984
- W. Gardner, "Likelihood Sensitivity and the Cramer-Rao
Bound," IEEE Transactions on Information Theory, p. 491, July 1979
- J. Cadzow, "Least Squares, Modeling, and Signal
Processing," Digital Signal Processing, pp. 2 - 20, 1994
- W. Press et al., "Ch. 15 Modeling of Data", in
Numerical Recipes in C, 2nd Edition, Cambridge Press
- S. Stein, "Differential Delay/Doppler ML Estimation with
Unknown Signals," IEEE Transactions on Signal Processing, pp. 2717 -
2719, August 1993
- T. Berger and R. Blahut, "Coherent Estimation of
Differential Delay and Differential Doppler," Proceedings of the 1984
Conference on Information Sciences and Systems, Princeton University, pp. 537
- 541, 1984
- M. Fowler, “Analysis of Passive Emitter Location using
Terrain Data,” IEEE Transactions on Aerospace and Electronic Systems,
pp. 495 – 507, April 2001.
- K. Becker, "An Efficient Method of Passive Emitter
Location," IEEE Transactions on Aerospace and Electronic Systems, pp.
1019 – 1104, Oct. 1992
- P. Chestnut, "Emitter Location Accuracy using TDOA and
Differential Doppler," IEEE Transactions on Aerospace and Electronic
Systems, pp. 214 - 218.
- M. Fowler, “Air‑to‑Air Passive Location,” U.S. Patent
#5,870,056 Issued 2/9/1999
- D. Rife and Boorstyn, "Single-Tone Parameter Estimation from
Discrete-Time Observations," IEEE Transactions on Information Theory, pp.
591 - 598, Sept. 1974.
- S. Tretter, "Estimating the Frequency of a Noisy Sinusoid by Linear
Regression," IEEE Transactions on Information Theory, pp. 832 - 835, Nov.
- S. Kay, "A Fast Accurate Single Frequency Estimator," IEEE
Transactions on Acoustics, Speech, and Signal Processing , pp. 1987 - 1990,
download, print out, and bring to the relevant class - see Course Schedule above
These notes are
complete versions of my class notes.
- You'll only need to fill in
certain spoken information during class you deem important.
- This will free you up for
in-class thinking (come ready to do some!)
There also a few "reading notes"
that supplement the textbook's coverage... these are now posted on BB.
New PDFs of PPT Charts
Notes #1a Probability Review
Notes #1b Vectors and Matrices Review (See Reading Notes on BB)
Notes #2: Ch 1 Intro to Est
Notes #3: Ch 2 MVUE
Notes #4: Ch 3 Cramer Rao Bound Pt. A
Notes #5: Ch 3 Cramer
Rao Bound Pt. B
Notes #6: Ch 3 Cramer
Rao Bound Pt. C
Notes #7: Ch 3 Cramer
Rao Bound Pt. D
Notes #8: Ch 3 CRLB Examples
Notes #9: CRLB Example for Doppler Location (See Reading Notes on BB)
Notes #10 Ch_4 Linear
Notes #11 Ch_6 BLUE
Notes #12 Ch7A
Notes #13 Ch7B
Notes #14 Ch7C
Notes #15 ML Example - Revised
Notes #16 Ch8A
Notes #17 Ch8B
Notes #18 Ch8C
Notes #19 Ch8D
Notes #20 LS Single Platform (See Reading Notes on BB)
#21 Doppler Tracking (See Reading Notes on BB)
#22 Results for 2 RVs (Pre-Ch. 10) (See Reading Notes on BB)
Notes #23 Ch10A
Notes #24 Ch10B
(See Reading Notes on BB)
Notes #25 Ch11A
Notes #26 Ch11B
Notes #26a Recursive Bayesian
Notes #27 Ch12A
Notes #28 Ch12B
Notes #28a Wiener Filter for Deblurring Images
Notes #29 Ch13A
Notes #30 Ch13B
Notes #31 Ch13C
Notes #32 Ch13D
- Will be posted on
Blackboard (if you don't know how to get access to it ask me)
- Will be posted on
Blackboard (if you don't know how to get access to it ask me)
A significant portion of your grade will be
based on a project. It is important to start early.
Things you can do early-on are:
- Understand the Signal Model and Project Issues
- Derive/Analyze Cramer-Rao bounds for your problem
- Write simulation code to generate the data
Things you can do by mid-semester are:
- Estimator derivation and analysis (most projects will
use classical methods, all of which we will have studied by Spring Break)
- Coding of estimator
- Start your analyses of effects and/or trade-offs
Things you can do by end of semester are:
- Complete your analyses of effects and/or trade-offs
- Complete your simulations
- Analyze your results
- Write your report
Here are three files to help you get started.
- The first gives a list of project suggestions.
- The second gives details on how to do your report.
- The third is a MSWord template that will help you format your report to
professional publication standards
MATLAB Handouts & Links
Links of Interest
DSP Tutorials and Reference Material
Some interactive demos of DSP
concepts (e.g., filter design)
Some basics of DSP theory and
The Scientist and Engineer's Guide to Digital Signal Processing
A freely downloadable DSP Book!!!!
Provides coverage at the level assumed as a pre-requisite for EE522 - so it's a
good place to start if you need a refresher.
Signals & Systems Demos (Johns Hopkins University)
A neat set of java applets that
demonstrate continuous-time & discrete-time signal processing at the level
assumed as a pre-requisite for EE522 - so it's a good place to start if you need
Estimation Oriented Material
An overview of many different ways
to estimate frequency.
Blind SNR Estimation
Discusses how to estimate the SNR of
Back to Prof. Fowler's Homepage