On the Calculation of Mutual Information
1970; Society for Industrial and Applied Mathematics; Volume: 19; Issue: 1 Linguagem: Inglês
10.1137/0119020
ISSN1095-712X
Autores Tópico(s)Complex Systems and Time Series Analysis
ResumoPrevious article Next article On the Calculation of Mutual InformationTyrone E. DuncanTyrone E. Duncanhttps://doi.org/10.1137/0119020PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAbout[1] I. M. Gel'fand and , A. M. Yaglom, Computation of the amount of information about a stochastic function contained in another such function, Uspehi Mat. Nauk (N.S.), 12 (1957), 3–52, English transl., Amer. Math. Soc. Transl. (2), 12 (1959), pp. 199–246 MR0085153 Google Scholar[2] Chiang Tse-Pei, A remark on defining the quantity of information, Teor. Veroyatnost. i Primenen, 3 (1958), 99–103, English transl., Amer. Math. Soc. Transl. (2), 12 (1959), pp. 247–250 MR0094270 0080.12103 Google Scholar[3] Albert Perez, Notions généralisées d'incertitude, d'entropie et d'information du point de vue de la théorie de martingalesTransactions of the first Prague conference on information theory, statistical decision functions, random processes held at Liblice near Prague from November 28 to 30, 1956, Publishing House of the Czechoslovak Academy of Sciences, Prague, 1957, 183–208 MR0099889 0102.13204 Google Scholar[4] Claude E. Shannon and , Warren Weaver, The Mathematical Theory of Communication, The University of Illinois Press, Urbana, Ill., 1949vi+117 MR0032134 0041.25804 Google Scholar[5] Tyrone E. Duncan, Likelihood functions for stochastic signals in white noise, Information and Control, 16 (1970), 303–310 10.1016/S0019-9958(70)90150-6 MR0282435 CrossrefGoogle Scholar[6] Kiyosi Itô, On a formula concerning stochastic differentials, Nagoya Math. J., 3 (1951), 55–65 MR0044063 0045.07603 CrossrefGoogle Scholar[7] J. L. Doob, Stochastic processes, John Wiley & Sons Inc., New York, 1953viii+654 MR0058896 0053.26802 Google Scholar[8] Paul-A. Meyer, Probability and potentials, Blaisdell Publishing Co. Ginn and Co., Waltham, Mass.-Toronto, Ont.-London, 1966xiii+266 MR0205288 0138.10401 Google Scholar[9] H. L. Van Trees, Detection, Estimation, and Modulation Theory, Part I, John Wiley, New York, 1968 0202.18002 Google Scholar[10] Charles R. Baker, Mutual information for Gaussian processes, SIAM J. Appl. Math., 19 (1970), 451–458 10.1137/0119044 MR0266672 0207.50102 LinkISIGoogle Scholar[11] M. S. Pinsker, Information and information stability of random variables and processes, Translated and edited by Amiel Feinstein, Holden-Day Inc., San Francisco, Calif., 1964xii+243 MR0213190 0125.09202 Google Scholar[12] R. E. Kalman and , R. S. Bucy, New results in linear filtering and prediction theory, Trans. ASME Ser. D. J. Basic Engrg., 83 (1961), 95–108 MR0234760 CrossrefGoogle Scholar[13] R. E. Kalman, J. L. Bodganoff and , F. Kozin, New methods in Wiener filtering theory, Proc. First Symposium on Engineering Applications of Random Function Theory and Probability, John Wiley, New York, 1963, 270–388 0218.93019 Google Scholar Previous article Next article FiguresRelatedReferencesCited ByDetails Friendly spectrum jamming against MIMO eavesdroppingWireless Networks, Vol. 28, No. 6 | 27 April 2022 Cross Ref Optimal Sensor Gain Control for Minimum-Information Estimation of Continuous-Time Gauss-Markov Processes2022 American Control Conference (ACC) | 8 Jun 2022 Cross Ref Networked Filtering with Feedback for Continuous-Time Observations2022 American Control Conference (ACC) | 8 Jun 2022 Cross Ref Multi-Intensity Optimization-Based CT and Cone Beam CT Image RegistrationElectronics, Vol. 11, No. 12 | 13 June 2022 Cross Ref Patient‐specific synthetic magnetic resonance imaging generation from cone beam computed tomography for image guidance in liver stereotactic body radiation therapyPrecision Radiation Oncology, Vol. 6, No. 2 | 11 June 2022 Cross Ref Comparison of Methane Emission Patterns from Dairy Housings with Solid and Slatted Floors at Two LocationsAgronomy, Vol. 12, No. 2 | 3 February 2022 Cross Ref Measuring the Rate of Information Exchange in Point-Process Data With Application to Cardiovascular VariabilityFrontiers in Network Physiology, Vol. 1 | 28 January 2022 Cross Ref On the Relation Between Information and Power in Stochastic Thermodynamic EnginesIEEE Control Systems Letters, Vol. 6 | 1 Jan 2022 Cross Ref An Information-Theoretic Framework to Measure the Dynamic Interaction Between Neural Spike TrainsIEEE Transactions on Biomedical Engineering, Vol. 68, No. 12 | 1 Dec 2021 Cross Ref Usage of Mutual Information as Similarity Measures for Stochastic Binding Images2021 International Conference on Information Technology and Nanotechnology (ITNT) | 20 Sep 2021 Cross Ref On the Achievable Rate and Capacity for a Sample-Based Practical Photon-Counting ReceiverIEEE Transactions on Communications, Vol. 69, No. 9 | 1 Sep 2021 Cross Ref Estimation, Survey onEncyclopedia of Systems and Control | 4 August 2021 Cross Ref Information thermodynamics for interacting stochastic systems without bipartite structureJournal of Statistical Mechanics: Theory and Experiment, Vol. 2019, No. 11 | 4 November 2019 Cross Ref Generic Variance Bounds on Estimation and Prediction Errors in Time Series Analysis: An Entropy Perspective2019 IEEE Information Theory Workshop (ITW) | 1 Aug 2019 Cross Ref An Elementary Proof of a Classical Information-Theoretic Formula2019 IEEE International Symposium on Information Theory (ISIT) | 1 Jul 2019 Cross Ref Ambiguity and Information Processing in a Model of Intermediary Asset PricingSSRN Electronic Journal | 1 Jan 2019 Cross Ref On Continuous-Time Gaussian ChannelsEntropy, Vol. 21, No. 1 | 14 January 2019 Cross Ref Multiscale network dynamics between heart rate and locomotor activity are altered in schizophreniaPhysiological Measurement, Vol. 39, No. 11 | 30 October 2018 Cross Ref A Survey on MIMO Transmission With Finite Input Signals: Technical Challenges, Advances, and Future TrendsProceedings of the IEEE, Vol. 106, No. 10 | 1 Oct 2018 Cross Ref Mutual Information, Relative Entropy and Estimation Error in Semi-Martingale ChannelsIEEE Transactions on Information Theory, Vol. 64, No. 10 | 1 Oct 2018 Cross Ref Nonlinear Filtering and Information Geometry: A Hilbert Manifold ApproachInformation Geometry and Its Applications | 4 November 2018 Cross Ref Statistical Approach for Performance Analysis of Multipath Scattering EnvironmentWireless Personal Communications, Vol. 98, No. 1 | 18 August 2017 Cross Ref Optimal sensor design and zero-delay source coding for continuous-time vector Gauss-Markov processes2017 IEEE 56th Annual Conference on Decision and Control (CDC) | 1 Dec 2017 Cross Ref A View of Information-Estimation Relations in Gaussian NetworksEntropy, Vol. 19, No. 8 | 9 August 2017 Cross Ref Finding Homologous PointsTheory and Applications of Image Registration | 7 July 2017 Cross Ref Semidefinite Programming Approach to Gaussian Sequential Rate-Distortion Trade-OffsIEEE Transactions on Automatic Control, Vol. 62, No. 4 | 1 Apr 2017 Cross Ref Extensions of the I-MMSE Relationship to Gaussian Channels With Feedback and MemoryIEEE Transactions on Information Theory, Vol. 62, No. 10 | 1 Oct 2016 Cross Ref Information, Estimation, and Lookahead in the Gaussian ChannelIEEE Transactions on Signal Processing, Vol. 64, No. 14 | 15 Jul 2016 Cross Ref Mutual information, relative entropy and estimation error in semi-martingale channels2016 IEEE International Symposium on Information Theory (ISIT) | 1 Jul 2016 Cross Ref Informative windowed forecasting of continuous-time linear systems for mutual information-based sensor planningAutomatica, Vol. 57 | 1 Jul 2015 Cross Ref Diffusion channel with Poisson reception process: capacity results and applications2015 IEEE International Symposium on Information Theory (ISIT) | 1 Jun 2015 Cross Ref Information geometric nonlinear filteringInfinite Dimensional Analysis, Quantum Probability and Related Topics, Vol. 18, No. 02 | 11 June 2015 Cross Ref Input-Dependent Frequency Modulation of Cortical Gamma Oscillations Shapes Spatial Synchronization and Enables Phase CodingPLOS Computational Biology, Vol. 11, No. 2 | 13 February 2015 Cross Ref Survey on EstimationEncyclopedia of Systems and Control | 19 February 2015 Cross Ref Maximum work extraction and implementation costs for nonequilibrium Maxwell's demonsPhysical Review E, Vol. 90, No. 4 | 13 October 2014 Cross Ref Minimax Filtering Regret via Relations Between Information and EstimationIEEE Transactions on Information Theory, Vol. 60, No. 8 | 1 Aug 2014 Cross Ref Support Recovery for the Drift Coefficient of High-Dimensional DiffusionsIEEE Transactions on Information Theory, Vol. 60, No. 7 | 1 Jul 2014 Cross Ref Mismatched quantum filtering and entropic information2014 IEEE International Symposium on Information Theory | 1 Jun 2014 Cross Ref Analysis of Mismatched Estimation Errors Using Gradients of Partition FunctionsIEEE Transactions on Information Theory, Vol. 60, No. 4 | 1 Apr 2014 Cross Ref SIGNAL EXTRACTION AND RATIONAL INATTENTIONEconomic Inquiry, Vol. 52, No. 2 | 25 February 2014 Cross Ref Relations Between Information and Estimation in the Presence of FeedbackInformation and Control in Networks | 1 Jan 2014 Cross Ref Information theoretic conditions for tracking in leader-follower systems with communication constraintsJournal of Control Theory and Applications, Vol. 11, No. 3 | 4 July 2013 Cross Ref Minimax filtering regret via relations between information and estimation2013 IEEE International Symposium on Information Theory | 1 Jul 2013 Cross Ref Pointwise relations between information and estimation in the Poisson channel2013 IEEE International Symposium on Information Theory | 1 Jul 2013 Cross Ref The role of lookahead in estimation under Gaussian noise2013 IEEE International Symposium on Information Theory | 1 Jul 2013 Cross Ref Directed Information, Causal Estimation, and Communication in Continuous TimeIEEE Transactions on Information Theory, Vol. 59, No. 3 | 1 Mar 2013 Cross Ref On MMSE Crossing Properties and Implications in Parallel Vector Gaussian ChannelsIEEE Transactions on Information Theory, Vol. 59, No. 2 | 1 Feb 2013 Cross Ref Survey on EstimationEncyclopedia of Systems and Control | 4 April 2014 Cross Ref Parametric Regularity of the Conditional Expectations via the Malliavin Calculus and ApplicationsMalliavin Calculus and Stochastic Analysis | 9 January 2013 Cross Ref Optimal Coding and Control for Linear Gaussian Systems Over Gaussian Channels Under Quadratic CostStochastic Networked Control Systems | 9 April 2013 Cross Ref ReferencesSystem Parameter Identification | 1 Jan 2013 Cross Ref On information, estimation and lookahead2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton) | 1 Oct 2012 Cross Ref On MMSE, mutual information and Popov function in continuous-time multivariable linear systems2012 IEEE International Conference on Control Applications | 1 Oct 2012 Cross Ref Control-Theoretic Approach to Communication With FeedbackIEEE Transactions on Automatic Control, Vol. 57, No. 10 | 1 Oct 2012 Cross Ref Pointwise Relations Between Information and Estimation in Gaussian NoiseIEEE Transactions on Information Theory, Vol. 58, No. 10 | 1 Oct 2012 Cross Ref Explicit Forward Recursive Estimators for Markov Modulated Markov ProcessesStochastic Models, Vol. 28, No. 3 | 1 Jul 2012 Cross Ref Pointwise relations between information and estimation in Gaussian noise2012 IEEE International Symposium on Information Theory Proceedings | 1 Jul 2012 Cross Ref Mutual Information, Relative Entropy, and Estimation in the Poisson ChannelIEEE Transactions on Information Theory, Vol. 58, No. 3 | 1 Mar 2012 Cross Ref Feature Selection and Heterogeneous DescriptorsImage Registration | 1 Jan 2012 Cross Ref Mutual information, relative entropy, and estimation in the Poisson channel2011 IEEE International Symposium on Information Theory Proceedings | 1 Jul 2011 Cross Ref Information theoretic limits on learning stochastic differential equations2011 IEEE International Symposium on Information Theory Proceedings | 1 Jul 2011 Cross Ref Optimum Estimation via Gradients of Partition Functions and Information Measures: A Statistical-Mechanical PerspectiveIEEE Transactions on Information Theory, Vol. 57, No. 6 | 1 Jun 2011 Cross Ref Information Capacity of Mobile Communication Systems with MMSE DFE and OFDM Receivers2011 IEEE 73rd Vehicular Technology Conference (VTC Spring) | 1 May 2011 Cross Ref The Relationship Between Causal and Noncausal Mismatched Estimation in Continuous-Time AWGN ChannelsIEEE Transactions on Information Theory, Vol. 56, No. 9 | 1 Sep 2010 Cross Ref Continuous trajectory planning of mobile sensors for informative forecastingAutomatica, Vol. 46, No. 8 | 1 Aug 2010 Cross Ref Optimum estimation via partition functions and information measures2010 IEEE International Symposium on Information Theory | 1 Jun 2010 Cross Ref The relationship between causal and non-causal mismatched estimation in continuous-time AWGN channelsIEEE Information Theory Workshop 2010 (ITW 2010) | 1 Jan 2010 Cross Ref Mutual Information for Stochastic Signals and LÉvy ProcessesIEEE Transactions on Information Theory, Vol. 56, No. 1 | 1 Jan 2010 Cross Ref Mutual information for stochastic differential equations driven by fractional Brownian motionRandom Operators and Stochastic Equations, Vol. 18, No. 1 | 1 Jan 2010 Cross Ref Hessian and Concavity of Mutual Information, Differential Entropy, and Entropy Power in Linear Vector Gaussian ChannelsIEEE Transactions on Information Theory, Vol. 55, No. 8 | 1 Aug 2009 Cross Ref Directed information and causal estimation in continuous time2009 IEEE International Symposium on Information Theory | 1 Jun 2009 Cross Ref Directed information, causal estimation, and communication in continuous time2009 7th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks | 1 Jun 2009 Cross Ref Informed tradersProceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, Vol. 465, No. 2104 | 19 December 2008 Cross Ref Scanning and Sequential Decision Making for Multidimensional Data—Part II: The Noisy CaseIEEE Transactions on Information Theory, Vol. 54, No. 12 | 1 Dec 2008 Cross Ref Interactive Statistical Mechanics and Nonlinear FilteringJournal of Statistical Physics, Vol. 133, No. 4 | 24 September 2008 Cross Ref Mutual Information for Stochastic Signals and Fractional Brownian MotionIEEE Transactions on Information Theory, Vol. 54, No. 10 | 1 Oct 2008 Cross Ref Control of Continuous-Time Linear Gaussian Systems Over Additive Gaussian Wireless Fading Channels: A Separation PrincipleIEEE Transactions on Automatic Control, Vol. 53, No. 4 | 1 May 2008 Cross Ref Mutual Information and Conditional Mean Estimation in Poisson ChannelsIEEE Transactions on Information Theory, Vol. 54, No. 5 | 1 May 2008 Cross Ref Continuous motion planning for information forecast2008 47th IEEE Conference on Decision and Control | 1 Jan 2008 Cross Ref On the connections between information and estimation theory: from AWGN Channels to dynamical systems with feedback2008 Information Theory and Applications Workshop | 1 Jan 2008 Cross Ref Scanning, Filtering and Prediction for Random Fields Corrupted by Gaussian Noise2007 IEEE International Symposium on Information Theory | 1 Jun 2007 Cross Ref Some relations between mutual information and estimation error in Wiener spaceThe Annals of Applied Probability, Vol. 17, No. 3 | 1 Jun 2007 Cross Ref Dual Nonlinear Filters and Entropy ProductionNigel J. NewtonSIAM Journal on Control and Optimization, Vol. 46, No. 5 | 2 November 2007AbstractPDF (290 KB)Estimation and mutual information2007 46th IEEE Conference on Decision and Control | 1 Jan 2007 Cross Ref Scanning and Sequential Decision Making for Multidimensional Data2007 Information Theory and Applications Workshop | 1 Jan 2007 Cross Ref Dual Kalman–Bucy Filters and Interactive Entropy ProductionNigel J. NewtonSIAM Journal on Control and Optimization, Vol. 45, No. 3 | 31 July 2006AbstractPDF (188 KB)Robustness and information processingReview of Economic Dynamics, Vol. 9, No. 1 | 1 Jan 2006 Cross Ref Gradient of mutual information in linear vector Gaussian channelsIEEE Transactions on Information Theory, Vol. 52, No. 1 | 1 Jan 2006 Cross Ref On Mutual Information, Likelihood Ratios, and Estimation Error for the Additive Gaussian ChannelIEEE Transactions on Information Theory, Vol. 51, No. 9 | 1 Sep 2005 Cross Ref Mutual Information and Minimum Mean-Square Error in Gaussian ChannelsIEEE Transactions on Information Theory, Vol. 51, No. 4 | 1 Apr 2005 Cross Ref Information and Entropy Flow in the Kalman?Bucy FilterJournal of Statistical Physics, Vol. 118, No. 1-2 | 1 Jan 2005 Cross Ref Additive non-Gaussian noise channels: mutual information and conditional mean estimationProceedings. International Symposium on Information Theory, 2005. ISIT 2005. | 1 Jan 2005 Cross Ref Filtering and Estimation, NonlinearWiley Encyclopedia of Electrical and Electronics Engineering | 27 December 1999 Cross Ref Bounds on the information rate of intertransition-time-restricted binary signaling over an AWGN channelIEEE Transactions on Information Theory, Vol. 45, No. 6 | 1 Sep 1999 Cross Ref Entropy Inequalities and Entropy Dynamics in Nonlinear Filtering of Diffusion ProcessesStochastic Analysis, Control, Optimization and Applications | 1 Jan 1999 Cross Ref Fading channels: information-theoretic and communications aspectsIEEE Transactions on Information Theory, Vol. 44, No. 6 | 1 Oct 1998 Cross Ref Achievable performance over the correlated Rician channelIEEE Transactions on Communications, Vol. 42, No. 11 | 1 Nov 1994 Cross Ref Lower and upper bounds on the minimum mean-square error in composite source signal estimationIEEE Transactions on Information Theory, Vol. 38, No. 6 | 1 Nov 1992 Cross Ref On information transfer by envelope-constrained signals over the AWGN channelIEEE Transactions on Information Theory, Vol. 34, No. 3 | 1 May 1988 Cross Ref On the Capacity of Peak Power Constrained Gaussian ChannelsPerformance Limits in Communication Theory and Practice | 1 Jan 1988 Cross Ref FilteringStabilization of Control Systems | 1 Jan 1987 Cross Ref The adaptive lqg problem, IIStochastics, Vol. 16, No. 1-2 | 1 Feb 1986 Cross Ref Mutual information in stationary channels with additive noiseIEEE Transactions on Information Theory, Vol. 31, No. 5 | 1 Sep 1985 Cross Ref Measures of information for continuous observationsIEEE Transactions on Information Theory, Vol. 29, No. 2 | 1 Mar 1983 Cross Ref Information and filteringInformation Sciences, Vol. 18, No. 2 | 1 Jul 1979 Cross Ref A lower bound on filtering error with application to phase demodulationIEEE Transactions on Information Theory, Vol. 25, No. 4 | 1 Jul 1979 Cross Ref Detection, Mutual Information and Feedback Encoding: Applications of Stochastic CalculusCommunication Systems and Random Process Theory | 1 Jan 1978 Cross Ref Information and distortion in reduced-order filter designIEEE Transactions on Information Theory, Vol. 23, No. 2 | 1 Mar 1977 Cross Ref An alternative expression of the mutual information for Gaussian processes (Corresp.)IEEE Transactions on Information Theory, Vol. 22, No. 5 | 1 Sep 1976 Cross Ref Mutual information in Gaussian channelsJournal of Multivariate Analysis, Vol. 4, No. 1 | 1 Mar 1974 Cross Ref Mutual information for locally infinitely divisible random processesLithuanian Mathematical Transactions, Vol. 14, No. 1 | 1 Jan 1974 Cross Ref Mutual information for stochastic differential equationsInformation and Control, Vol. 19, No. 3 | 1 Oct 1971 Cross Ref Mutual information of the white Gaussian channel with and without feedbackIEEE Transactions on Information Theory, Vol. 17, No. 4 | 1 Jul 1971 Cross Ref On measures of information for continuous stochastic processesStochastic Processes in Classical and Quantum Systems Cross Ref Some Martingales from a Fractional Brownian Motion and ApplicationsProceedings of the 44th IEEE Conference on Decision and Control Cross Ref Control Over Wireless Communication Channel for Continuous-Time SystemsProceedings of the 44th IEEE Conference on Decision and Control Cross Ref Bounds on the capacity of a binary input AWGN channel with intertransition duration restrictions17th Convention of Electrical and Electronics Engineers in Israel Cross Ref Lower and Upper Bounds on the Minimum Mean Square Error in Composite Source EstimationProceedings. 1991 IEEE International Symposium on Information Theory Cross Ref Mutual information and MMSE in gaussian channelsInternational Symposium onInformation Theory, 2004. ISIT 2004. Proceedings. Cross Ref Mutual information and conditional mean estimation in Poisson channelsManufacturing Engineer Cross Ref Volume 19, Issue 1| 1970SIAM Journal on Applied Mathematics1-265 History Submitted:21 May 1968Published online:12 July 2006 InformationCopyright © 1970 Society for Industrial and Applied MathematicsPDF Download Article & Publication DataArticle DOI:10.1137/0119020Article page range:pp. 215-220ISSN (print):0036-1399ISSN (online):1095-712XPublisher:Society for Industrial and Applied Mathematics
Referência(s)